scholarly journals REGENERATING CODETECHNIQUE IN DISTRIBUTED STORAGE

2017 ◽  
Vol 5 (1) ◽  
pp. 60
Author(s):  
Agus Maman Abadi ◽  
Karyati Karyati ◽  
Musthofa Musthofa ◽  
Emut Emut
Keyword(s):  
Data Storage ◽  
Algebraic Structure ◽  
Distributed Storage ◽  
Storage System ◽  
Distributed Data ◽  
Distributed Data Storage ◽  
Code Technique ◽  
Code Module ◽  
Data Storage System ◽  
Regenerating Code

Abstract The Increasing need of storing large amounts of data presents a new challenge. One way to address this challenge is to use distributed data storage system. One of the strategies implemented in the distributed data storage system is using the technique of regenerating code. The code used in this technique is based on the algebraic structure of fields. Some studies have also been carried out to create code that is based on the other algebraic structure namely module. In this study, we attempted to assess the implementation of the code module at regenerating technique code. The study showed there is a potential properties code based on module that can be used in regenerating code technique. Keywords: Distributed storage, regenerating code technique, module code

scholarly journals REQUEST BALANCING METHOD FOR INCREASING THEIR PROCESSING EFFICIENCY WITH INFORMATION REPLICATION IN A DISTRIBUTED DATA STORAGE SYSTEM

2021 ◽  
pp. 75-82
Author(s):  
Igor Boyarshin ◽  
Anna Doroshenko ◽  
Pavlo Rehida
Keyword(s):  
Data Storage ◽  
Storage Systems ◽  
Storage System ◽  
New Method ◽  
Distributed Data ◽  
Processing Efficiency ◽  
Distributed Data Storage ◽  
Shared Data ◽  
Multiple Data ◽  
Data Storage System

The article describes a new method of improving efficiency of the systems that deal with storage and providing access of shared data of many users by utilizing replication. Existing methods of load balancing in data storage systems are described, namely RR and WRR. A new method of request balancing among multiple data storage nodes is proposed, that is able to adjust to input request stream intensity in real time and utilize disk space efficiently while doing so.

On construction of a distributed data storage system in cloud

Computing ◽  
2014 ◽  
Vol 98 (1-2) ◽  
pp. 93-118 ◽  
Author(s):  
Chao-Tung Yang ◽  
Wen-Chung Shih ◽  
Chih-Lin Huang ◽  
Fuu-Cheng Jiang ◽  
William Cheng-Chung Chu
Keyword(s):  
Data Storage ◽  
Storage System ◽  
Distributed Data ◽  
Distributed Data Storage ◽  
Data Storage System

scholarly journals PRELIMINARY STUDY ON APPLICATION OF MAX PLUS ALGEBRA IN DISTRIBUTED STORAGE SYSTEM THROUGH NETWORK CODING

2016 ◽  
Vol 4 (1) ◽  
Author(s):  
Agus Maman Abadi ◽  
Musthofa Musthofa ◽  
Emut Emut
Keyword(s):  
Network Coding ◽  
Data Storage ◽  
Algebraic Structure ◽  
Storage Systems ◽  
Distributed Storage ◽  
Storage System ◽  
Distributed Data Storage ◽  
Erasure Code ◽  
Distributed Storage Systems ◽  
Set Up

The increasing need in techniques of storing big data presents a new challenge. One way to address this challenge is the use of distributed storage systems. One strategy that implemented in distributed data storage systems is the use of Erasure Code which applied to network coding. The code used in this technique is based on the algebraic structure which is called as vector space. Some studies have also been carried out to create code that is based on other algebraic structures such as module.  In this study, we are going to try to set up a code based on the algebraic structure which is a generalization of the module that is semimodule by utilizing the max operations and sum operations at max plus algebra. The results of this study indicate that the max operation and the addition operation on max plus algebra cannot be used to establish a semimodule code, but by modifying the operation "+" as "min", we get a code based on semimodule. Keywords:   code, distributed storage systems, network coding, semimodule, max plus algebra

scholarly journals A Novel Trade Transaction Agreement Algorithm Using Blockchain Consensus Mechanism

2021 ◽  
Vol 2021 ◽  
pp. 1-9
Author(s):  
Pan Yi
Keyword(s):  
Data Storage ◽  
Simulation Analysis ◽  
Storage System ◽  
Experimental Tests ◽  
Distributed Data ◽  
Distributed Data Storage ◽  
Data Owner ◽  
Data Storage System ◽  
Time And Energy ◽  
Further Development

Blockchain, the underlying technology of Bitcoin, has been deeply studied in various fields after its development in recent years. As a typical decentralized distributed data storage system, consensus reached among all participants in a blockchain system requires a consensus mechanism to be realized. In order to make blockchain applicable to different application scenarios, different consensus mechanisms have been proposed. With the further development of blockchain applications, more and more studies have been conducted on the consensus mechanism. However, some existing consensus mechanisms still have some problems in various aspects. Therefore, this paper proposes a trade deal algorithm based on the blockchain mechanism of consensus. First of all, according to PBFT, the lack of a dynamic problem in the VPBFT voting mechanism was introduced. The node system is divided into four types with different responsibilities and gives the number of relations between nodes. When the number of nodes is changed, it can be calculated according to the quantity relation, ensuring dynamic. Second, a data anonymous transaction and authentication protocol is designed. In the protocol, when the seller sells data, the mapping relationship between the real identity and the false identity of the data owner is blinded and sent to the buyer. When the buyer wants to verify their identity, the seller’s identity can only be verified with the authentication of the blockchain. The proposed algorithm is superior to the current consensus in terms of time and energy consumption, throughput, and fault tolerance methods, which is proven through experimental tests and simulation analysis.

scholarly journals PetaShare: A Reliable, Efficient and Transparent Distributed Storage Management System

2011 ◽  
Vol 19 (1) ◽  
pp. 27-43
Author(s):  
Tevfik Kosar ◽  
Ismail Akturk ◽  
Mehmet Balman ◽  
Xinqi Wang
Keyword(s):  
Data Management ◽  
Data Storage ◽  
Management System ◽  
Distributed Storage ◽  
Storage System ◽  
Data Management System ◽  
Distributed Data ◽  
Distributed Data Storage ◽  
Data Movement ◽  
Reliability And Availability

Modern collaborative science has placed increasing burden on data management infrastructure to handle the increasingly large data archives generated. Beside functionality, reliability and availability are also key factors in delivering a data management system that can efficiently and effectively meet the challenges posed and compounded by the unbounded increase in the size of data generated by scientific applications. We have developed a reliable and efficient distributed data storage system, PetaShare, which spans multiple institutions across the state of Louisiana. At the back-end, PetaShare provides a unified name space and efficient data movement across geographically distributed storage sites. At the front-end, it provides light-weight clients the enable easy, transparent and scalable access. In PetaShare, we have designed and implemented an asynchronously replicated multi-master metadata system for enhanced reliability and availability, and an advanced buffering system for improved data transfer performance. In this paper, we present the details of our design and implementation, show performance results, and describe our experience in developing a reliable and efficient distributed data management system for data-intensive science.

scholarly journals Analysis of data integrity and storage quality of a distributed storage system

2021 ◽  
Vol 251 ◽  
pp. 02035
Author(s):  
Adrian Eduard Negru ◽  
Latchezar Betev ◽  
Mihai Carabaș ◽  
Costin Grigoraș ◽  
Nicolae Țăpuş ◽  
...  
Keyword(s):  
Data Storage ◽  
Distributed Storage ◽  
Storage System ◽  
Essential Element ◽  
Data Access ◽  
Distributed Data ◽  
Distributed Data Storage ◽  
Data Lifetime ◽  
Operational Issues ◽  
And Storage

CERN uses the world’s largest scientific computing grid, WLCG, for distributed data storage and processing. Monitoring of the CPU and storage resources is an important and essential element to detect operational issues in its systems, for example in the storage elements, and to ensure their proper and efficient function. The processing of experiment data depends strongly on the data access quality, as well as its integrity and both of these key parameters must be assured for the data lifetime. Given the substantial amount of data, O(200 PB), already collected by ALICE and kept at various storage elements around the globe, scanning every single data chunk would be a very expensive process, both in terms of computing resources usage and in terms of execution time. In this paper, we describe a distributed file crawler that addresses these natural limits by periodically extracting and analyzing statistically significant samples of files from storage elements, evaluates the results and is integrated with the existing monitoring solution, MonALISA.

Sign in / Sign up

Export Citation Format

Share Document