A Distributed Storage Algorithm Based on Cauchy RS Code

2013 ◽  
Vol 336-338 ◽  
pp. 2188-2194
Author(s):  
Xing Gao ◽  
Min Li ◽  
Juan Juan Huang ◽  
Bing Chang Liu
Keyword(s):  
Fault Tolerance ◽  
Data Center ◽  
Distributed Storage ◽  
Erasure Codes ◽  
Data Encoding ◽  
Multiplication Operation ◽  
Rs Code ◽  
Rs Coding ◽  
Encoding Efficiency ◽  
Better Than

Data fault tolerance is a key technology in the field of distributed storage. In this paper, an algorithm to encode massive amounts of data and then distribute storage these data on each node in the data center is proposed, aiming at coping with the serious challenges in the protection of data fault tolerance. The method converts multiplication operation in Cauchy RS coding into a binary multiplication through the transition on bit operation, so that the entire operation on RS encoding is converted to an operation containing only simple XOR operator. The experiment proves that the method is better than the copy and the original RS coding in the data encoding efficiency. Furthermore, it saves the storage space and promotes the application of erasure codes strongly in distributed storage field.

scholarly journals A Novel Repairable Fountain Codes Scheme Based on Partial Duplication Technique

2021 ◽  
Author(s):  
Anan Zhou ◽  
Benshun Yi ◽  
Mian Xiang ◽  
Laigan Luo
Keyword(s):  
Fault Tolerance ◽  
Distributed Storage ◽  
Low Cost ◽  
Storage System ◽  
Erasure Codes ◽  
Fountain Codes ◽  
Partial Duplication ◽  
Storage Network ◽  
Source Data ◽  
And Storage

Abstract Distributed storage system (DSS) is an emerging paradigm which provides reliable storage services for various source data. As the fault-tolerance scheme for DSS, erasure codes are required to provide redundancy service with high fault-tolerance and low cost. However, the existing coding scheme cannot provide these requirements well. Thus, it becomes an important yet challenging issue to find a code for storing various source data with high fault-tolerance and low cost. In this paper, a novel construction of repairable fountain codes with unequal locality is proposed by combining with partial duplication tech- nique, namely the PD-ULRFC scheme. We construct a multi-tier heterogeneous storage network, where data core, processing units and storage nodes collaboratively store and transmit data. Moreover, the proposed PD-ULRFC scheme can reduce the repair and download cost by sacrificing a little extra storage occupation. Furthermore, the expressions of the repair cost and download cost are derived to analyze the performance of PD-ULRFC scheme. The simulation results demonstrate that the PD-ULRFC scheme significantly outperforms other redundant schemes in communication cost saving.

Distributed storage evaluation on a three-wide inter-data center deployment

2013 ◽  
Author(s):  
Yih-Farn Chen ◽  
Scott Daniels ◽  
Marios Hadjieleftheriou ◽  
Pingkai Liu ◽  
Chao Tian ◽  
...  
Keyword(s):  
Data Center ◽  
Distributed Storage

An Improved Method for CT Image Coding Using PSNR Prediction Model Based on ELM

2015 ◽  
Vol 738-739 ◽  
pp. 598-601
Author(s):  
Han Yang Zhu ◽  
Xin Yu Jin ◽  
Jian Feng Shen
Keyword(s):  
Computational Complexity ◽  
Prediction Model ◽  
Image Coding ◽  
Region Of Interest ◽  
Image Encoding ◽  
Large Size ◽  
Learning Machine ◽  
Intractable Problem ◽  
Encoding Efficiency ◽  
Better Than

In telemedicine, medical images are always considered very important telemedicine diagnostic evidences. High transmission delay in a bandwidth limited network becomes an intractable problem because of its large size. It’s important to achieve a quality balance between Region of Interest (ROI) and Background Region (BR) when ROI-based image encoding is being used. In this paper, a research made on balancing method of LS-SVM based ROI/BR PSNR prediction model to optimize the ROI encoding shows it’s much better than conventional methods but with very high computational complexity. We propose a new method using extreme learning machine (ELM) with lower computational complexity to improve encoding efficiency compared to LS-SVM based model. Besides, it also achieves the same effect of balancing ROI and BR.

AN EXPLORATION OF NON-ASYMPTOTIC LOW-DENSITY, PARITY CHECK ERASURE CODES FOR WIDE-AREA STORAGE APPLICATIONS

2007 ◽  
Vol 17 (01) ◽  
pp. 103-123 ◽  
Author(s):  
JAMES S. PLANK ◽  
MICHAEL G. THOMASON
Keyword(s):  
Storage Systems ◽  
Distributed Storage ◽  
Ldpc Codes ◽  
Peer To Peer ◽  
Erasure Codes ◽  
Low Density ◽  
Parity Check ◽  
Finite Systems ◽  
Low Density Parity Check ◽  
Distributed Storage Systems

As peer-to-peer and widely distributed storage systems proliferate, the need to perform efficient erasure coding, instead of replication, is crucial to performance and efficiency. Low-Density Parity-Check (LDPC) codes have arisen as alternatives to standard erasure codes, such as Reed-Solomon codes, trading off vastly improved decoding performance for inefficiencies in the amount of data that must be acquired to perform decoding. The scores of papers written on LDPC codes typically analyze their collective and asymptotic behavior. Unfortunately, their practical application requires the generation and analysis of individual codes for finite systems. This paper attempts to illuminate the practical considerations of LDPC codes for peer-to-peer and distributed storage systems. The three main types of LDPC codes are detailed, and a huge variety of codes are generated, then analyzed using simulation. This analysis focuses on the performance of individual codes for finite systems, and addresses several important heretofore unanswered questions about employing LDPC codes in real-world systems.

A Fault-Tolerant Data Center Network Structure

2013 ◽  
Vol 734-737 ◽  
pp. 3048-3052
Author(s):  
Peng Wang ◽  
Yan Lv ◽  
Yu Tan
Keyword(s):  
Fault Tolerance ◽  
Network Structure ◽  
Data Center ◽  
Fault Tolerant ◽  
Data Center Network ◽  
Advantages And Disadvantages

According to advantages and disadvantages of the traditional data center network structure ,this paper propose a new data center network structure base on BCube and DCell. The new structure is mainly improved based on the scalability, fault tolerance, the throughput

A Novel Hash Chain-Based Data Availability Monitoring Method for Off-site Disaster Recovery Architecture

2021 ◽  
pp. 2150294
Author(s):  
Neng Huang ◽  
Junxing Zhu ◽  
Chaonian Guo ◽  
Shuhan Cheng ◽  
Xiaoyong Li
Keyword(s):  
Information Systems ◽  
Real Time ◽  
Data Center ◽  
Disaster Recovery ◽  
Distributed Storage ◽  
Data Availability ◽  
Data Consistency ◽  
Real Time Monitoring ◽  
The Real ◽  
Hash Chain

With the rapid development of mobile Internet, there is a higher demand for the real-time, reliability and availability of information systems and to prevent the possible systemic risks of information systems, various business consistency standards and regulatory guidelines have been published, such as Recovery Time Object (RTO) and Recovery Point Object (RPO). Some of the current related researches focus on the standards, methods, management tools and technical frameworks of business consistency, while others study the data consistency algorithms in the cases of large data, cloud computing and distributed storage. However, few researchers have studied on how to monitor the data consistency and RPO of production-disaster recovery, and what architecture and technology should be applied in the monitoring. Moreover, in some information systems, due to the complex structures and distributions of data, it is difficult for traditional methods to quickly detect and accurately locate the first error data. Besides, due to the separation of production data center (PDC) and disaster recovery data center (DRDC), it is difficult to calculate the data difference and RPO between the two centers. This paper first discusses the architecture of remote distributed DRDCs. The architecture can make the disaster recovery (DR) system always online and the data always readable, and support the real-time monitoring of data availability, consistency as well as other related indicators, in this way to make DRDC out-of-the-box in disasters. Second, inspired by blockchain, this paper proposes a method to realize real-time monitoring of data consistency and RTO by building hash chains for PDC and DRDC. Third, this paper evaluates the hash chain operations from the algorithm time complexity, the data consistency, and the validity of RPO monitoring algorithms and since DR system is actually a kind of distributed system, the proposed approach can also be applied to the data consistency detection and data difference monitoring in other distributed systems.

Sign in / Sign up

Export Citation Format

Share Document