scholarly journals G-SD: Achieving Fast Reverse Lookup using Scalable Declustering Layout in Large-Scale File Systems

2018 ◽  
Vol 6 (4) ◽  
pp. 1017-1030
Author(s):  
Jun Wang ◽  
Dezhi Han ◽  
Junyao Zhang ◽  
Jiangling Yin
Keyword(s):  
Large Scale ◽  
File Systems

Optimizing of metadata management in large-scale file systems

2018 ◽  
Vol 21 (4) ◽  
pp. 1865-1879 ◽  
Author(s):  
Nae Young Song ◽  
Hwajung Kim ◽  
Hyuck Han ◽  
Heon Young Yeom
Keyword(s):  
Large Scale ◽  
File Systems ◽  
Metadata Management

Grid Data Handling

2013 ◽  
pp. 294-321
Author(s):  
Alexandru Costan
Keyword(s):  
Fault Tolerance ◽  
Data Storage ◽  
Large Scale ◽  
File Systems ◽  
Future Research ◽  
Distributed Data ◽  
Data Handling ◽  
Grid Data ◽  
Distributed Data Storage ◽  
Grid Environments

To accommodate the needs of large-scale distributed systems, scalable data storage and management strategies are required, allowing applications to efficiently cope with continuously growing, highly distributed data. This chapter addresses the key issues of data handling in grid environments focusing on storing, accessing, managing and processing data. We start by providing the background for the data storage issue in grid environments. We outline the main challenges addressed by distributed storage systems: high availability which translates into high resilience and consistency, corruption handling regarding arbitrary faults, fault tolerance, asynchrony, fairness, access control and transparency. The core part of the chapter presents how existing solutions cope with these high requirements. The most important research results are organized along several themes: grid data storage, distributed file systems, data transfer and retrieval and data management. Important characteristics such as performance, efficient use of resources, fault tolerance, security, and others are strongly determined by the adopted system architectures and the technologies behind them. For each topic, we shortly present previous work, describe the most recent achievements, highlight their advantages and limitations, and indicate future research trends in distributed data storage and management.

Dynamic Load Rebalancing Algorithm for Private Cloud

2014 ◽  
Vol 573 ◽  
pp. 556-559
Author(s):  
A. Shenbaga Bharatha Priya ◽  
J. Ganesh ◽  
Mareeswari M. Devi
Keyword(s):  
Dynamic Load ◽  
Large Scale ◽  
File System ◽  
Single Point ◽  
File Systems ◽  
Distributed File System ◽  
Private Cloud ◽  
Global Knowledge ◽  
Load Imbalance ◽  
And Storage

Infrastructure-As-A-Service (IAAS) provides an environmental setup under any type of cloud. In Distributed file system (DFS), nodes are simultaneously serve computing and storage functions; that is parallel Data Processing and storage in cloud. Here, file is considered as a data or load. That file is partitioned into a number of File chunks (FC) allocated in distinct nodes so that Map Reduce tasks can be performed in parallel over the nodes. Files and Nodes can be dynamically created, deleted, and added. This results in load imbalance in a distributed file system; that is, the file chunks are not distributed as uniformly as possible among the Chunk Servers (CS). Emerging distributed file systems in production systems strongly depend on a central node for chunk reallocation or Distributed node to maintain global knowledge of all chunks. This dependence is clearly inadequate in a large-scale, failure-prone environment because the central load balancer is put under considerable workload that is linearly scaled with the system size, it may thus become the performance bottleneck and the single point of failure and memory wastage in distributed nodes. So, we have to enhance the Client side module with server side module to create, delete and update the file chunks in Client Module. And manage the overall private cloud and apply dynamic load balancing algorithm to perform auto scaling options in private cloud. In this project, a fully distributed load rebalancing algorithm is presented to cope with the load imbalance problem.

scholarly journals Scalable and Adaptive Metadata Management in Ultra Large-Scale File Systems

2008 ◽  
Author(s):  
Yu Hua ◽  
Yifeng Zhu ◽  
Hong Jiang ◽  
Dan Feng ◽  
Lei Tian
Keyword(s):  
Large Scale ◽  
File Systems ◽  
Metadata Management

Metadata Distribution and Consistency Techniques for Large-Scale Cluster File Systems

2011 ◽  
Vol 22 (5) ◽  
pp. 803-816 ◽  
Author(s):  
Jin Xiong ◽  
Yiming Hu ◽  
Guojie Li ◽  
Rongfeng Tang ◽  
Zhihua Fan
Keyword(s):  
Large Scale ◽  
File Systems ◽  
Cluster File

Toward a more biologically accurate model of diffusion: adhesion and collisions

SIMULATION ◽  
2017 ◽  
Vol 93 (12) ◽  
pp. 1037-1044
Author(s):  
Ahmed M Fouad ◽  
John A Noel
Keyword(s):  
Large Scale ◽  
Particle Density ◽  
File Systems ◽  
Tracer Diffusion ◽  
Transport Processes ◽  
Average Particle ◽  
Diffusion Behavior ◽  
Single File ◽  
Tracer Particles ◽  
The One

In single-file dynamics, Brownian particles (referred to as tracer or tagged particles) diffuse and collide with each other in one-dimensional domains. If the average particle density is kept fixed during the diffusion, the collisions between the tracer particles result in their famous anomalous sub-diffusion behavior with time to the one half dependence. Many systems in nature are found to obey single-file dynamics, such as ion transport processes, and inter-particle adhesion plays a crucial role, either structurally or functionally, in the diffusion of such systems; however, the exact effect of adhesion on the diffusion has not been studied so far. We have examined the effect of adhesion on the collective diffusion of single-file systems. Here, we extend previous work where we perform large-scale numerical simulations that utilize Monte Carlo techniques and high-performance computing resources to examine the effect of adhesion on the diffusion of the tracer particles in systems that obey single-file dynamics. We show that if all the tracer particles experience the same adhesion coefficient, adhesion only slows down the diffusion by reducing the magnitude of the tracer diffusion coefficient; however, both the anomalous sub-diffusion behavior and time to the one half dependence of the tracer particles remain almost intact, independent of the adhesion.

scholarly journals Parallel network file systems using authenticated key exchange protocols

2017 ◽  
Vol 2 (3) ◽  
pp. 161
Author(s):  
S. Sathya ◽  
M. Ranjith Kumar ◽  
K. Madheswaran
Keyword(s):  
Large Scale ◽  
File Systems ◽  
Key Exchange ◽  
Distributed File Systems ◽  
Authenticated Key Exchange ◽  
Forward Secrecy ◽  
Storage Devices ◽  
Key Exchange Protocols ◽  
Parallel Network ◽  
Metadata Server

The keyestablishment for secure many-to-many communications is very important nowadays. The problem is inspired by the proliferation of large-scale distributed file systems supporting parallel access to multiple storage devices. In this, a variety of authenticated key exchange protocols that are designed to address the issues. This shows that these protocols are capable of reducing the workload of the metadata server and concurrently supporting forward secrecy and escrow-freeness. All this requires only a small fraction of increased computation overhead at the client. This proposed three authenticated key exchange protocols for parallel network file system (pNFS). The protocols offer three appealing advantages over the existing Kerberos-based protocol. First, the metadata server executing these protocols has much lower workload than that of the Kerberos-based approach. Second, two of these protocols provide forward secrecy: one is partially forward secure (with respect to multiple sessions within a time period), while the other is fully forward secure (with respect to a session). Third, designed a protocol which not only provides forward secrecy, but is also escrow-free.

Large-scale file systems with the flexibility of databases

1996 ◽  
Vol 28 (4es) ◽  
pp. 207 ◽  
Author(s):  
Alok Choudhary ◽  
David Kotz
Keyword(s):  
Large Scale ◽  
File Systems
Sign in / Sign up

Export Citation Format

Share Document