NVMTFS: A Non-Volatile Memory Adaptive File System for Tiered Storage System

2018 ◽  
Author(s):  
Shiyong Liu ◽  
Zhichao Cao ◽  
Zhongwen Guo ◽  
Guohua Wang ◽  
Xupeng Wang ◽  
...  
Keyword(s):  
File System ◽  
Storage System ◽  
Non Volatile Memory ◽  
Volatile Memory ◽  
Adaptive File System

Octopus + : An RDMA-Enabled Distributed Persistent Memory File System

2021 ◽  
Vol 17 (3) ◽  
pp. 1-25
Author(s):  
Bohong Zhu ◽  
Youmin Chen ◽  
Qing Wang ◽  
Youyou Lu ◽  
Jiwu Shu
Keyword(s):  
High Speed ◽  
High Performance ◽  
File System ◽  
Direct Memory Access ◽  
File Systems ◽  
Distributed File Systems ◽  
Persistent Memory ◽  
Memory Modules ◽  
Non Volatile Memory ◽  
Volatile Memory

Non-volatile memory and remote direct memory access (RDMA) provide extremely high performance in storage and network hardware. However, existing distributed file systems strictly isolate file system and network layers, and the heavy layered software designs leave high-speed hardware under-exploited. In this article, we propose an RDMA-enabled distributed persistent memory file system, Octopus + , to redesign file system internal mechanisms by closely coupling non-volatile memory and RDMA features. For data operations, Octopus + directly accesses a shared persistent memory pool to reduce memory copying overhead, and actively fetches and pushes data all in clients to rebalance the load between the server and network. For metadata operations, Octopus + introduces self-identified remote procedure calls for immediate notification between file systems and networking, and an efficient distributed transaction mechanism for consistency. Octopus + is enabled with replication feature to provide better availability. Evaluations on Intel Optane DC Persistent Memory Modules show that Octopus + achieves nearly the raw bandwidth for large I/Os and orders of magnitude better performance than existing distributed file systems.

UMFS: An efficient user-space file system for non-volatile memory

2018 ◽  
Vol 89 ◽  
pp. 18-29 ◽  
Author(s):  
Xianzhang Chen ◽  
Edwin H.-M. Sha ◽  
Qingfeng Zhuge ◽  
Ting Wu ◽  
Weiwen Jiang ◽  
...  
Keyword(s):  
File System ◽  
Non Volatile Memory ◽  
User Space ◽  
Volatile Memory

A novel non-volatile memory storage system for I/O-intensive applications

2018 ◽  
Vol 19 (10) ◽  
pp. 1291-1302 ◽  
Author(s):  
Wen-Bing Han ◽  
Xiao-Gang Chen ◽  
Shun-Fen Li ◽  
Ge-Zi Li ◽  
Zhi-Tang Song ◽  
...  
Keyword(s):  
Storage System ◽  
Memory Storage ◽  
Non Volatile Memory ◽  
Volatile Memory

scholarly journals Mapping Datasets to Object Storage System

2020 ◽  
Vol 245 ◽  
pp. 04037
Author(s):  
Xiaowei Aaron Chu ◽  
Jeff LeFevre ◽  
Aldrin Montana ◽  
Dana Robinson ◽  
Quincey Koziol ◽  
...  
Keyword(s):  
Storage Systems ◽  
Distributed Storage ◽  
Storage System ◽  
Random Access ◽  
Data Partitioning ◽  
Data Layout ◽  
Storage Devices ◽  
Storage Servers ◽  
Non Volatile Memory ◽  
Volatile Memory

Access libraries such as ROOT[1] and HDF5[2] allow users to interact with datasets using high level abstractions, like coordinate systems and associated slicing operations. Unfortunately, the implementations of access libraries are based on outdated assumptions about storage systems interfaces and are generally unable to fully benefit from modern fast storage devices. For example, access libraries often implement buffering and data layout that assume that large, single-threaded sequential access patterns are causing less overall latency than small parallel random access: while this is true for spinning media, it is not true for flash media. The situation is getting worse with rapidly evolving storage devices such as non-volatile memory and ever larger datasets. This project explores distributed dataset mapping infrastructures that can integrate and scale out existing access libraries using Ceph’s extensible object model, avoiding re-implementation or even modifications of these access libraries as much as possible. These programmable storage extensions coupled with our distributed dataset mapping techniques enable: 1) access library operations to be offloaded to storage system servers, 2) the independent evolution of access libraries and storage systems and 3) fully leveraging of the existing load balancing, elasticity, and failure management of distributed storage systems like Ceph. They also create more opportunities to conduct storage server-local optimizations specific to storage servers. For example, storage servers might include local key/value stores combined with chunk stores that require different optimizations than a local file system. As storage servers evolve to support new storage devices like non-volatile memory, these server-local optimizations can be implemented while minimizing disruptions to applications. We will report progress on the means by which distributed dataset mapping can be abstracted over particular access libraries, including access libraries for ROOT data, and how we address some of the challenges revolving around data partitioning and composability of access operations.

Sign in / Sign up

Export Citation Format

Share Document