scholarly journals InK: In-Kernel Key-Value Storage with Persistent Memory

Electronics ◽  
2020 ◽  
Vol 9 (11) ◽  
pp. 1913
Author(s):  
Minjong Ha ◽  
Sang-Hoon Kim
Keyword(s):  
High Performance ◽  
File Systems ◽  
Cloud Service ◽  
Main Memory ◽  
Storage Device ◽  
Storage Devices ◽  
Persistent Memory ◽  
Non Volatile Memory ◽  
Long Time ◽  
Block Based

Block-based storage devices exhibit different characteristics from main memory, and applications and systems have been optimized for a long time considering the characteristics in mind. However, emerging non-volatile memory technologies are about to change the situation. Persistent Memory (PM) provides a huge, persistent, and byte-addressable address space to the system, thereby enabling new opportunities for systems software. However, existing applications are usually apt to indirectly utilize PM as a storage device on top of file systems. This makes applications and file systems perform unnecessary operations and amplify I/O traffic, thereby under-utilizing the high performance of PM. In this paper, we make the case for an in-Kernel key-value storage service optimized for PM, called InK. While providing the persistence of data at a high performance, InK considers the characteristics of PM to guarantee the crash consistency. To this end, InK indexes key-value pairs with B+ tree, which is more efficient on PM. We implemented InK based on the Linux kernel and evaluated its performance with Yahoo Cloud Service Benchmark (YCSB) and RocksDB. Evaluation results confirms that InK has advantages over LSM-tree-based key-value store systems in terms of throughput and tail latency.

scholarly journals Implications of NVM Based Storage on Memory Subsystem Management

2020 ◽  
Vol 10 (3) ◽  
pp. 999
Author(s):  
Hyokyung Bahn ◽  
Kyungwoon Cho
Keyword(s):  
Random Access ◽  
Disk Drive ◽  
Main Memory ◽  
Memory Storage ◽  
Storage Device ◽  
Storage Devices ◽  
Large Memory ◽  
Memory Subsystems ◽  
Non Volatile Memory ◽  
Management Techniques

Recently, non-volatile memory (NVM) has advanced as a fast storage medium, and legacy memory subsystems optimized for DRAM (dynamic random access memory) and HDD (hard disk drive) hierarchies need to be revisited. In this article, we explore the memory subsystems that use NVM as an underlying storage device and discuss the challenges and implications of such systems. As storage performance becomes close to DRAM performance, existing memory configurations and I/O (input/output) mechanisms should be reassessed. This article explores the performance of systems with NVM based storage emulated by the RAMDisk under various configurations. Through our measurement study, we make the following findings. (1) We can decrease the main memory size without performance penalties when NVM storage is adopted instead of HDD. (2) For buffer caching to be effective, judicious management techniques like admission control are necessary. (3) Prefetching is not effective in NVM storage. (4) The effect of synchronous I/O and direct I/O in NVM storage is less significant than that in HDD storage. (5) Performance degradation due to the contention of multi-threads is less severe in NVM based storage than in HDD. Based on these observations, we discuss a new PC configuration consisting of small memory and fast storage in comparison with a traditional PC consisting of large memory and slow storage. We show that this new memory-storage configuration can be an alternative solution for ever-growing memory demands and the limited density of DRAM memory. We anticipate that our results will provide directions in system software development in the presence of ever-faster storage devices.

Octopus + : An RDMA-Enabled Distributed Persistent Memory File System

2021 ◽  
Vol 17 (3) ◽  
pp. 1-25
Author(s):  
Bohong Zhu ◽  
Youmin Chen ◽  
Qing Wang ◽  
Youyou Lu ◽  
Jiwu Shu
Keyword(s):  
High Speed ◽  
High Performance ◽  
File System ◽  
Direct Memory Access ◽  
File Systems ◽  
Distributed File Systems ◽  
Persistent Memory ◽  
Memory Modules ◽  
Non Volatile Memory ◽  
Volatile Memory

Non-volatile memory and remote direct memory access (RDMA) provide extremely high performance in storage and network hardware. However, existing distributed file systems strictly isolate file system and network layers, and the heavy layered software designs leave high-speed hardware under-exploited. In this article, we propose an RDMA-enabled distributed persistent memory file system, Octopus + , to redesign file system internal mechanisms by closely coupling non-volatile memory and RDMA features. For data operations, Octopus + directly accesses a shared persistent memory pool to reduce memory copying overhead, and actively fetches and pushes data all in clients to rebalance the load between the server and network. For metadata operations, Octopus + introduces self-identified remote procedure calls for immediate notification between file systems and networking, and an efficient distributed transaction mechanism for consistency. Octopus + is enabled with replication feature to provide better availability. Evaluations on Intel Optane DC Persistent Memory Modules show that Octopus + achieves nearly the raw bandwidth for large I/Os and orders of magnitude better performance than existing distributed file systems.

NVMM-Oriented Hierarchical Persistent Client Caching for Lustre

2021 ◽  
Vol 17 (1) ◽  
pp. 1-22
Author(s):  
Wen Cheng ◽  
Chunyan Li ◽  
Lingfang Zeng ◽  
Yingjin Qian ◽  
Xi Li ◽  
...  
Keyword(s):  
High Performance ◽  
File System ◽  
File Systems ◽  
Main Memory ◽  
Full Potential ◽  
Storage Devices ◽  
Hierarchical Storage ◽  
Client Cache ◽  
Client Side ◽  
Performance Computing

In high-performance computing (HPC), data and metadata are stored on special server nodes and client applications access the servers’ data and metadata through a network, which induces network latencies and resource contention. These server nodes are typically equipped with (slow) magnetic disks, while the client nodes store temporary data on fast SSDs or even on non-volatile main memory (NVMM). Therefore, the full potential of parallel file systems can only be reached if fast client side storage devices are included into the overall storage architecture. In this article, we propose an NVMM-based hierarchical persistent client cache for the Lustre file system (NVMM-LPCC for short). NVMM-LPCC implements two caching modes: a read and write mode (RW-NVMM-LPCC for short) and a read only mode (RO-NVMM-LPCC for short). NVMM-LPCC integrates with the Lustre Hierarchical Storage Management (HSM) solution and the Lustre layout lock mechanism to provide consistent persistent caching services for I/O applications running on client nodes, meanwhile maintaining a global unified namespace of the entire Lustre file system. The evaluation results presented in this article show that NVMM-LPCC can increase the average read throughput by up to 35.80 times and the average write throughput by up to 9.83 times compared with the native Lustre system, while providing excellent scalability.

scholarly journals Layered double hydroxide membrane with high hydroxide conductivity and ion selectivity for energy storage device

2021 ◽  
Vol 12 (1) ◽  
Author(s):  
Jing Hu ◽  
Xiaomin Tang ◽  
Qing Dai ◽  
Zhiqiang Liu ◽  
Huamin Zhang ◽  
...  
Keyword(s):  
Energy Storage ◽  
High Performance ◽  
Ion Selectivity ◽  
Storage Device ◽  
Hydroxyl Groups ◽  
Flow Battery ◽  
Energy Storage Devices ◽  
Storage Devices ◽  
Ions Transport ◽  
Double Hydroxides

AbstractMembranes with fast and selective ions transport are highly demanded for energy storage devices. Layered double hydroxides (LDHs), bearing uniform interlayer galleries and abundant hydroxyl groups covalently bonded within two-dimensional (2D) host layers, make them superb candidates for high-performance membranes. However, related research on LDHs for ions separation is quite rare, especially the deep-going study on ions transport behavior in LDHs. Here, we report a LDHs-based composite membrane with fast and selective ions transport for flow battery application. The hydroxide ions transport through LDHs via vehicular (standard diffusion) & Grotthuss (proton hopping) mechanisms is uncovered. The LDHs-based membrane enables an alkaline zinc-based flow battery to operate at 200 mA cm−2, along with an energy efficiency of 82.36% for 400 cycles. This study offers an in-depth understanding of ions transport in LDHs and further inspires their applications in other energy-related devices.

scholarly journals Layered double hydroxide membrane with high hydroxide conductivity and ion selectivity for energy storage device

2020 ◽  
Author(s):  
Jing Hu ◽  
Xiaomin Tang ◽  
Qing Dai ◽  
Zhiqiang Liu ◽  
Huamin Zhang ◽  
...  
Keyword(s):  
Energy Storage ◽  
High Performance ◽  
Ion Selectivity ◽  
Storage Device ◽  
Hydroxyl Groups ◽  
Flow Battery ◽  
Energy Storage Devices ◽  
Storage Devices ◽  
Ions Transport ◽  
Double Hydroxides

Abstract Membranes with fast and selective ions transport are highly demanded for energy storage devices. Layered double hydroxides (LDHs), bearing uniform interlayer galleries and abundant hydroxyl groups covalently bonded within two-dimensional (2D) host layers, make them superb candidates for high-performance membranes. However, related research on LDHs for ions separation is quite rare, especially the deep-going study on ions transport behavior in LDHs. Here, we report a LDHs-based composite membrane with fast and selective ions transport for flow battery application. The hydroxide ions transport through LDHs via vehicular (standard diffusion) & Grotthuss (proton hopping) mechanisms is uncovered. The LDHs-based membrane enables an alkaline zinc-based flow battery to operate at 200 mA cm− 2, along with an energy efficiency of 82.36% for 400 cycles, which is among the highest efficiencies for zinc-based flow batteries. This study offers an in-depth understanding of ions transport in LDHs and further inspires their applications in other energy-related devices.

scholarly journals Synergistic approach of high-performance N-NiCo/PC environment benign electrode material for energy storage device

2021 ◽  
Author(s):  
Muhammad Irfan ◽  
Xianhua Liu ◽  
Suraya Mushtaq ◽  
Jonnathan Cabrera ◽  
Pingping Zhang
Keyword(s):  
Energy Storage ◽  
High Performance ◽  
Electrode Materials ◽  
Composite Catalyst ◽  
Storage Device ◽  
Energy Storage Devices ◽  
Biomass Waste ◽  
Supercapacitor Application ◽  
Suitable Material ◽  
Storage Devices

Abstract Development of sustainable electrochemical energy storage devices faces great challenge in exploring highly efficient and low cost electrode materials. Biomass waste derived carbonaceous materials can be used as an alternative to expensive metals in supercapacitor. However, their application limited by low performance. In this study, the combination use of persimmon waste derived carbon and transition metal nitride demonstrated strong potential for supercapacitor application. Persimmon based carbonaceous gel decorated with bimetallic-nitride (N-NiCo/PC) was firstly synthesized through a green hydrothermal method. Electrochemical properties of N-NiCo/PC as electrode in 6 M KOH electrolyte solution were evaluated by using cyclic voltammetry (CV) and charge-discharge measurements. The N-NiCo/PC exhibited 895.5 F/g specific capacitance at 1 A/g current density and maintained 91.5% capacitance retention after 900 cycles. Hence, the bimetallic nitride-based-composite catalyst is a potentially suitable material for high-performance energy storage devices. In addition, this work demonstrated a promising pathway for transforming environmental waste into sustainable energy conversion materials.

Optimizing synchronization mechanism for block-based file systems using persistent memory

2020 ◽  
Vol 111 ◽  
pp. 288-299
Author(s):  
Chaoshu Yang ◽  
Qingfeng Zhuge ◽  
Xianzhang Chen ◽  
Edwin H.-M. Sha ◽  
Duo Liu ◽  
...  
Keyword(s):  
File Systems ◽  
Synchronization Mechanism ◽  
Persistent Memory ◽  
Block Based

scholarly journals An Empirical Evaluation of NVM-Aware File Systems on Intel Optane DC Persistent Memory Modules

Electronics ◽  
2021 ◽  
Vol 10 (16) ◽  
pp. 1977
Author(s):  
Guangyu Zhu ◽  
Jaehyun Han ◽  
Sangjin Lee ◽  
Yongseok Son
Keyword(s):  
Data Management ◽  
Empirical Evaluation ◽  
File Systems ◽  
Data Management System ◽  
Nand Flash ◽  
Storage Devices ◽  
Persistent Memory ◽  
Memory Modules ◽  
And Performance ◽  
And Storage

The emergence of non-volatile memories (NVM) brings new opportunities and challenges to data management system design. As an important part of the data management systems, several new file systems are developed to take advantage of the characteristics of NVM. However, these NVM-aware file systems are usually designed and evaluated based on simulations or emulations. In order to explore the performance and characteristics of these file systems on real hardware, in this article, we provide an empirical evaluation of NVM-aware file systems on the first commercially available byte-addressable NVM (i.e., the Intel Optane DC Persistent Memory Module (DCPMM)). First, to compare the performance difference between traditional file systems and NVM-aware file systems, we evaluate the performance of Ext4, XFS, F2FS, Ext4-DAX, XFS-DAX, and NOVA file systems on DCPMMs. To compare DCPMMs with other secondary storage devices, we also conduct the same evaluations on Optane SSDs and NAND-flash SSDs. Second, we observe how remote NUMA node access and device mapper striping affect the performance of DCPMMs. Finally, we evaluate the performance of the database (i.e., MySQL) on DCPMMs with Ext4 and Ext4-DAX file systems. We summarize several observations from the evaluation results and performance analysis. We anticipate that these observations will provide implications for various memory and storage systems.

Exploring Efficient Architectures on Remote In-Memory NVM over RDMA

2021 ◽  
Vol 20 (5s) ◽  
pp. 1-20
Author(s):  
Qingfeng Zhuge ◽  
Hao Zhang ◽  
Edwin Hsing-Mean Sha ◽  
Rui Xu ◽  
Jun Liu ◽  
...  
Keyword(s):  
High Performance ◽  
File System ◽  
File Systems ◽  
Data Access ◽  
Main Memory ◽  
Memory Modules ◽  
Significant Performance ◽  
Architectural Structures ◽  
Memory Accesses ◽  
Careful Design

Efficiently accessing remote file data remains a challenging problem for data processing systems. Development of technologies in non-volatile dual in-line memory modules (NVDIMMs), in-memory file systems, and RDMA networks provide new opportunities towards solving the problem of remote data access. A general understanding about NVDIMMs, such as Intel Optane DC Persistent Memory (DCPM), is that they expand main memory capacity with a cost of multiple times lower performance than DRAM. With an in-depth exploration presented in this paper, however, we show an interesting finding that the potential of NVDIMMs for high-performance, remote in-memory accesses can be revealed through careful design. We explore multiple architectural structures for accessing remote NVDIMMs in a real system using Optane DCPM, and compare the performance of various structures. Experiments are conducted to show significant performance gaps among different ways of using NVDIMMs as memory address space accessible through RDMA interface. Furthermore, we design and implement a prototype of user-level, in-memory file system, RIMFS, in the device DAX mode on Optane DCPM. By comparing against the DAX-supported Linux file system, Ext4-DAX, we show that the performance of remote reads on RIMFS over RDMA is 11.44 higher than that on a remote Ext4-DAX on average. The experimental results also show that the performance of remote accesses on RIMFS is maintained on a heavily loaded data server with CPU utilization as high as 90%, while the performance of remote reads on Ext4-DAX is significantly reduced by 49.3%, and the performance of local reads on Ext4-DAX is even more significantly reduced by 90.1%. The performance comparisons of writes exhibit the same trends.

scholarly journals Exploring the Future of Out-of-Core Computing with Compute-Local Non-Volatile Memory

2014 ◽  
Vol 22 (2) ◽  
pp. 125-139 ◽  
Author(s):  
Myoungsoo Jung ◽  
Ellis H. Wilson ◽  
Wonil Choi ◽  
John Shalf ◽  
Hasan Metin Aktulga ◽  
...  
Keyword(s):  
High Performance ◽  
File Systems ◽  
Network Capacity ◽  
General Purpose ◽  
Graphical Processing Units ◽  
Non Volatile Memory ◽  
Order Of Magnitude ◽  
Volatile Memory ◽  
Graphical Processing ◽  
Point To Point

Drawing parallels to the rise of general purpose graphical processing units (GPGPUs) as accelerators for specific high-performance computing (HPC) workloads, there is a rise in the use of non-volatile memory (NVM) as accelerators for I/O-intensive scientific applications. However, existing works have explored use of NVM within dedicated I/O nodes, which are distant from the compute nodes that actually need such acceleration. As NVM bandwidth begins to out-pace point-to-point network capacity, we argue for the need to break from the archetype of completely separated storage. Therefore, in this work we investigate co-location of NVM and compute by varying I/O interfaces, file systems, types of NVM, and both current and future SSD architectures, uncovering numerous bottlenecks implicit in these various levels in the I/O stack. We present novel hardware and software solutions, including the new Unified File System (UFS), to enable fuller utilization of the new compute-local NVM storage. Our experimental evaluation, which employs a real-world Out-of-Core (OoC) HPC application, demonstrates throughput increases in excess of an order of magnitude over current approaches.

Sign in / Sign up

Export Citation Format

Share Document