Fault Tolerance Performance Evaluation of Large-Scale Distributed Storage Systems HDFS and Ceph Case Study

2018 ◽  
Author(s):  
Yehia Arafa ◽  
Atanu Barai ◽  
Mai Zheng ◽  
Abdel-Hameed A. Badawy
Keyword(s):  
Performance Evaluation ◽  
Fault Tolerance ◽  
Large Scale ◽  
Storage Systems ◽  
Distributed Storage ◽  
Distributed Storage Systems

Sharing of Distributed Geospatial Data through Grid Technology

2010 ◽  
pp. 222-228 ◽  
Author(s):  
Yaxing Wei ◽  
Liping Di ◽  
Guangxuan Liao ◽  
Baohua Zhao ◽  
Aijun Chen ◽  
...  
Keyword(s):  
Large Scale ◽  
Storage Systems ◽  
Distributed Storage ◽  
Geospatial Data ◽  
Grid Technology ◽  
Grid System ◽  
Multiple User ◽  
User Communities ◽  
Distributed Storage Systems ◽  
Critical Requirement

With the rapid accumulation of geospatial data and the advancement of geoscience, there is a critical requirement for an infrastructure that can integrate large-scale, heterogeneous, and distributed storage systems for the sharing of geospatial data within multiple user communities. This article probes into the feasibility to share distributed geospatial data through Grid computing technology by introducing several major issues (including system heterogeneity, uniform mechanism to publish and discover geospatial data, performance, and security) to be faced by geospatial data sharing and how Grid technology can help to solve these issues. Some recent research efforts, such as ESG and the Data Grid system in GMU CSISS, have proven that Grid technology provides a large-scale infrastructure which can seamlessly integrate dispersed geospatial data together and provide uniform and efficient ways to access the data.

scholarly journals Testing of complex, large-scale distributed storage systems: a CERN disk storage case study

2019 ◽  
Vol 214 ◽  
pp. 05008 ◽  
Author(s):  
Jozsef Makai ◽  
Andreas Joachim Peters ◽  
Georgios Bitzes ◽  
Elvin Alin Sindrilaru ◽  
Michal Kamil Simon ◽  
...  
Keyword(s):  
Large Scale ◽  
Distributed Storage ◽  
Storage System ◽  
Critical System ◽  
Disk Storage ◽  
Testing Framework ◽  
The People ◽  
Working Together ◽  
Distributed Storage Systems

Complex, large-scale distributed systems are frequently used to solve extraordinary computing, storage and other problems. However, the development of these systems usually requires working with several software components, maintaining and improving a large codebase and also providing a collaborative environment for many developers working together. The central role that such complex systems play in mission critical tasks and also in the daily activity of the users means that any software bug affecting the availability of the service has far reaching effects. Providing an easily extensible testing framework is a pre-requisite for building both confidence in the system but also among developers who contribute to the code. The testing framework can address concrete bugs found in the odebase thus avoiding any future regressions and also provides a high degree of confidence for the people contributing new code. Easily incorporating other people's work into the project greatly helps scaling out manpower so that having more developers contributing to the project can actually result in more work being done rather then more bugs added. In this paper we go through the case study of EOS, the CERN disk storage system and introduce the methods and mechanisms of how to achieve all-automatic regression and robustness testing along with continuous integration for such a large-scale, complex and critical system using a container-based environment.

Sign in / Sign up

Export Citation Format

Share Document