Metadata Caching Subsystem for Cloud Storage

2012 ◽  
Vol 214 ◽  
pp. 584-590 ◽  
Author(s):  
De Jiao Niu ◽  
Tao Cai ◽  
Yong Zhao Zhan ◽  
Shi Guang Ju
Keyword(s):  
Cloud Storage ◽  
File System ◽  
Experimental Results ◽  
Distributed File System ◽  
Metadata Management ◽  
Metadata Cache

Cloud storage is a hot topic in current research. Different from previous work, we emphasize the importance of metadata cache in the study of cloud storage. Because the efficiency of distributed file system has much effect on cloud storage The metadata operation accounts for more than 50% of the total file operation. So the strategy of efficient metadata management is important. There are three parts in this paper. We start with a brief introduction of cloud storage. Then a metadata caching algorithm for cloud storage is proposed. An additional discussion of its performance is also provided. The prototype which incorporates the proposed metadata caching algorithm is realized on Luster to evaluate its performance. Comparing experimental results from this study conclude that the metadata caching subsystem can improve the performance of cloud storage.

Data Storage Technology and its Development Based on Cloud Computing

2013 ◽  
Vol 756-759 ◽  
pp. 1275-1279
Author(s):  
Lin Na Huang ◽  
Feng Hua Liu
Keyword(s):  
Cloud Computing ◽  
Data Storage ◽  
Cloud Storage ◽  
High Performance ◽  
File System ◽  
Storage System ◽  
Distributed File System ◽  
Cloud Data ◽  
Storage Technology ◽  
Cloud Data Storage

Cloud storage of high performance is the basic condition for cloud computing. This article introduces the concept and advantage of cloud storage, discusses the infrastructure of cloud storage system as well as the architecture of cloud data storage, researches the details about the design of Distributed File System within cloud data storage, at the same time, puts forward different developing strategies for the enterprises according to the different roles that the enterprises are acting as during the developing process of cloud computing.

scholarly journals Application of Filters to Multiway Joins in MapReduce

2014 ◽  
Vol 2014 ◽  
pp. 1-11 ◽  
Author(s):  
Taewhi Lee ◽  
Dong-Hyuk Im ◽  
Hangkyu Kim ◽  
Hyoung-Joo Kim
Keyword(s):  
File System ◽  
Experimental Results ◽  
Distributed File System ◽  
Multiple Datasets ◽  
Processing Order

Joining multiple datasets in MapReduce may amplify the disk and network overheads because intermediate join results have to be written to the underlying distributed file system, or map output records have to be replicated multiple times. This paper proposes a method for applying filters based on the processing order of input datasets, which is appropriate for the two types of multiway joins: common attribute joins and distinct attribute joins. The number of redundant records filtered depends on the processing order. In common attribute joins, the input records do not need to be replicated, so a set of filters is created, which are applied in turn. In distinct attribute joins, the input records have to be replicated, so multiple sets of filters need to be created, which depend on the number of join attributes. The experimental results showed that our approach outperformed a cascade of two-way joins and basic multiway joins in cases where small portions of input datasets were joined.

Metadata Indexing Sub-System for Distributed File System

2011 ◽  
Vol 143-144 ◽  
pp. 864-868
Author(s):  
De Jiao Niu ◽  
Tao Cai ◽  
Yong Zhao Zhan ◽  
Shi Guang Ju
Keyword(s):  
Open Source ◽  
Hash Function ◽  
File System ◽  
Distributed File System ◽  
Metadata Management ◽  
Time And Space ◽  
Management Algorithm ◽  
Simulation Results ◽  
System Time ◽  
Management Algorithms

The efficiency of metadata indexing is important to the performance of distributed file system. Time and space spending of current metadata management algorithms are unstable. In this paper, we use B-tree to index the metadata of distributed file system. Lustre is an open source distributed file system in which Hash function is used to manage the metadata. We implement the prototype of metadata indexing sub-system on Lustre and use Iozone to test the I/O performance of Lustre with and without the metadata indexing sub-system respectively. The simulation results show that Lustre with the metadata indexing sub-system has higher adaptability than Lustre with Hash-based metadata management algorithm.

Sign in / Sign up

Export Citation Format

Share Document