SMOPD-C: An autonomous vertical partitioning technique for distributed databases on cluster computers

2014 ◽  
Author(s):  
Liangzhe Li ◽  
Le Gruenwald
Keyword(s):  
Distributed Databases ◽  
Vertical Partitioning ◽  
Cluster Computers ◽  
Partitioning Technique

Network search

2018 ◽  
Author(s):  
Mark Newman
Keyword(s):  
Web Search ◽  
Distributed Databases ◽  
Small World ◽  
Peer Networks ◽  
Model Parameters ◽  
Breadth First Search ◽  
Ranking Algorithms ◽  
Network Search ◽  
Peer To Peer Networks ◽  
Search Processes

This chapter gives a discussion of search processes on networks. It begins with a discussion of web search, including crawlers and web ranking algorithms such as PageRank. Search in distributed databases such as peer-to-peer networks is also discussed, including simple breadth-first search style algorithms and more advanced “supernode” approaches. Finally, network navigation is discussed at some length, motivated by consideration of Milgram's letter passing experiment. Kleinberg's variant of the small-world model is introduced and it is shown that efficient navigation is possible only for certain values of the model parameters. Similar results are also derived for the hierarchical model of Watts et al.

scholarly journals Modelling and optimization of information systems with distributed databases

2021 ◽  
Vol 1155 (1) ◽  
pp. 012062
Author(s):  
I Ya Lvovich ◽  
Ya E Lvovich ◽  
A P Preobrazhenskiy ◽  
Yu P Preobrazhenskiy ◽  
O N Choporov
Keyword(s):  
Information Systems ◽  
Distributed Databases ◽  
Modelling And Optimization

scholarly journals State Management for Cloud-Native Applications

Electronics ◽  
2021 ◽  
Vol 10 (4) ◽  
pp. 423
Author(s):  
Márk Szalay ◽  
Péter Mátray ◽  
László Toka
Keyword(s):  
Large Scale ◽  
Distributed Databases ◽  
Access Time ◽  
Replica Placement ◽  
State Management ◽  
Placement Decisions ◽  
Dynamic Replication ◽  
Cloud Databases ◽  
Placement Algorithm ◽  
The One

The stateless cloud-native design improves the elasticity and reliability of applications running in the cloud. The design decouples the life-cycle of application states from that of application instances; states are written to and read from cloud databases, and deployed close to the application code to ensure low latency bounds on state access. However, the scalability of applications brings the well-known limitations of distributed databases, in which the states are stored. In this paper, we propose a full-fledged state layer that supports the stateless cloud application design. In order to minimize the inter-host communication due to state externalization, we propose, on the one hand, a system design jointly with a data placement algorithm that places functions’ states across the hosts of a data center. On the other hand, we design a dynamic replication module that decides the proper number of copies for each state to ensure a sweet spot in short state-access time and low network traffic. We evaluate the proposed methods across realistic scenarios. We show that our solution yields state-access delays close to the optimal, and ensures fast replica placement decisions in large-scale settings.

Query Optimization in the Databases Distributed by Means of Product Equivalence Relations

1988 ◽  
Vol 11 (3) ◽  
pp. 241-265
Author(s):  
W. Marek ◽  
C. Rauszer
Keyword(s):  
Query Optimization ◽  
Selection Process ◽  
Distributed Databases ◽  
Optimization Techniques ◽  
Equivalence Relations ◽  
Database Operations ◽  
By Products

In this paper, we address the problem of query optimization in distributed databases. We show that horizontal partitions of databases, generated by products of equivalence relations, induce optimization techniques for the basic database operations (i.e., the selection, projection, and join operators). In the case of selection, our method allows for restriction of the number of blocks to be searched in the selection process and subsequent simplification of the selection formula at each block. For the natural join operation, we propose an algorithm that reduces the computation of fragments. Proofs of the correctness of our algorithms are also included.

scholarly journals Communication Failure Resilient Distributed Neural Network for Edge Devices

Electronics ◽  
2021 ◽  
Vol 10 (14) ◽  
pp. 1614
Author(s):  
Jonghun Jeong ◽  
Jong Sung Park ◽  
Hoeseok Yang
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
High Performance ◽  
State Of The Art ◽  
Wearable Devices ◽  
Communication Failure ◽  
Canadian Institute ◽  
Multiple Devices ◽  
Knowledge Distillation ◽  
Partitioning Technique

Recently, the necessity to run high-performance neural networks (NN) is increasing even in resource-constrained embedded systems such as wearable devices. However, due to the high computational and memory requirements of the NN applications, it is typically infeasible to execute them on a single device. Instead, it has been proposed to run a single NN application cooperatively on top of multiple devices, a so-called distributed neural network. In the distributed neural network, workloads of a single big NN application are distributed over multiple tiny devices. While the computation overhead could effectively be alleviated by this approach, the existing distributed NN techniques, such as MoDNN, still suffer from large traffics between the devices and vulnerability to communication failures. In order to get rid of such big communication overheads, a knowledge distillation based distributed NN, called Network of Neural Networks (NoNN), was proposed, which partitions the filters in the final convolutional layer of the original NN into multiple independent subsets and derives smaller NNs out of each subset. However, NoNN also has limitations in that the partitioning result may be unbalanced and it considerably compromises the correlation between filters in the original NN, which may result in an unacceptable accuracy degradation in case of communication failure. In this paper, in order to overcome these issues, we propose to enhance the partitioning strategy of NoNN in two aspects. First, we enhance the redundancy of the filters that are used to derive multiple smaller NNs by means of averaging to increase the immunity of the distributed NN to communication failure. Second, we propose a novel partitioning technique, modified from Eigenvector-based partitioning, to preserve the correlation between filters as much as possible while keeping the consistent number of filters distributed to each device. Throughout extensive experiments with the CIFAR-100 (Canadian Institute For Advanced Research-100) dataset, it has been observed that the proposed approach maintains high inference accuracy (over 70%, 1.53× improvement over the state-of-the-art approach), on average, even when a half of eight devices in a distributed NN fail to deliver their partial inference results.

Sign in / Sign up

Export Citation Format

Share Document