Performance Measurement on Scale-Up and Scale-Out Hadoop with Remote and Local File Systems

2016 ◽  
Author(s):  
Zhuozhao Li ◽  
Haiying Shen
Keyword(s):  
Performance Measurement ◽  
Scale Up ◽  
File Systems ◽  
Local File

scholarly journals Performance Evaluations of Distributed File Systems for Scientific Big Data in FUSE Environment

Electronics ◽  
2021 ◽  
Vol 10 (12) ◽  
pp. 1471
Author(s):  
Jun-Yeong Lee ◽  
Moon-Hyun Kim ◽  
Syed Asif Raza Raza Shah ◽  
Sang-Un Ahn ◽  
Heejun Yoon ◽  
...  
Keyword(s):  
Data Storage ◽  
Scale Up ◽  
File Systems ◽  
Performance Evaluations ◽  
Distributed File Systems ◽  
Data Intensive Computing ◽  
Data Intensive ◽  
Tremendous Amount ◽  
Computing Environments ◽  
And Performance

Data are important and ever growing in data-intensive scientific environments. Such research data growth requires data storage systems that play pivotal roles in data management and analysis for scientific discoveries. Redundant Array of Independent Disks (RAID), a well-known storage technology combining multiple disks into a single large logical volume, has been widely used for the purpose of data redundancy and performance improvement. However, this requires RAID-capable hardware or software to build up a RAID-enabled disk array. In addition, it is difficult to scale up the RAID-based storage. In order to mitigate such a problem, many distributed file systems have been developed and are being actively used in various environments, especially in data-intensive computing facilities, where a tremendous amount of data have to be handled. In this study, we investigated and benchmarked various distributed file systems, such as Ceph, GlusterFS, Lustre and EOS for data-intensive environments. In our experiment, we configured the distributed file systems under a Reliable Array of Independent Nodes (RAIN) structure and a Filesystem in Userspace (FUSE) environment. Our results identify the characteristics of each file system that affect the read and write performance depending on the features of data, which have to be considered in data-intensive computing environments.

Fragmentation Degree Research Based on File’s Layout

2011 ◽  
Vol 80-81 ◽  
pp. 1217-1225
Author(s):  
Ya Dong Zhu ◽  
Jing Wang Zhang ◽  
Chao Wang ◽  
Zhen Jun Liu
Keyword(s):  
Performance Measurement ◽  
File Systems ◽  
Experimental Analyses ◽  
Fragmentation Degree

Numerous resource allocating and releasing will cause fragmentation which will seriously affect the I/O performance of file systems. One method to solve this problem is file defragmentation. This paper focuses on fragmentation detection, the key part of the file defragmentation. We estimate file fragmentation situation through file’s layout, and then analyze the detailed effects of the situation on performance measurement. Then we quantize this by the file’s “fragmentation degree” through theoretical and experimental analyses. Finally we use it to estimate the file’s fragmentation and decide whether the file needs to be defragmented.

scholarly journals How Can Technology Support Quality Improvement? Lessons Learned from the adoption of an analytics tool for advanced performance measurement in a hospital unit.

2020 ◽  
Author(s):  
Sara Tolf ◽  
Johan Mesterton ◽  
Daniel Söderberg ◽  
Isis Amer-Wåhlin ◽  
Pamela Mazzocato
Keyword(s):  
Quality Improvement ◽  
Focus Group ◽  
Performance Measurement ◽  
Scale Up ◽  
Lessons Learned ◽  
Focus Group Interviews ◽  
Organizational Conditions ◽  
Local Setting ◽  
Timely Access ◽  
Group Interviews

Abstract Background Technology for timely feedback of data has the potential to support quality improvement (QI) in health care. However, such technology may pose difficulties stemming from the complex interaction with the setting in which it is implemented. To enable professionals to use data in QI there is a need to better understand of how to handle this complexity. This study aims to explore factors that influence the adoption of a technology-supported QI programme in an obstetric unit through a complexity informed framework . Methods This qualitative study, based on focus group interviews, was conducted at a Swedish university hospital’s obstetric unit, which used an analytics tool for advanced performance measurement that gave timely and case mix adjusted feedback of performance data to support QI. Data was collected through three focus group interviews conducted with 16 managers and staff. The Nonadoption, Abandonment, Scale-up, Spread, and Sustainability (NASSS) framework guided the data collection and analysis. Results Staff and managers deemed the technology to effectively support ongoing QI efforts by providing timely access to reliable data. The value of the technology was associated with a clear need to make better use of existing data in QI. The data and the methodology in the analytics tool reflected the complexity of the clinical conditions treated but was presented through an interface that was easy to access and user friendly. However, prior understanding of statistics was helpful to be able to fully grasp the presented data. The tool was adapted to the needs and the organizational conditions of the local setting through a collaborative approach between the technology supplier and the adopters. Conclusions Technology has the potential to enable systematic QI through motivating professionals by providing timely and adequate feedback of performance. The adoption of such technology is complex and requires openness for gradual learning and improvement.

scholarly journals Building and using containers at HPC centres for the ATLAS experiment

2019 ◽  
Vol 214 ◽  
pp. 07005
Author(s):  
Douglas Benjamin ◽  
Taylor Childers ◽  
David Lesny ◽  
Danila Oleynik ◽  
Sergey Panitkin ◽  
...  
Keyword(s):  
Time Scale ◽  
Scale Up ◽  
File Systems ◽  
Production Operation ◽  
Atlas Experiment ◽  
Shared File ◽  
Software Distribution ◽  
Computing Grid ◽  
Vital Component ◽  
Atlas Software

The HPC environment presents several challenges to the ATLAS experiment in running their automated computational workflows smoothly and efficiently, in particular regarding issues such as software distribution and I/O load. A vital component of the LHC Computing Grid, CVMFS, is not always available in HPC environments. ATLAS computing has experimented with all-inclusive containers, and later developed an environment to produce such containers for both Shifter and Singularity. The all-inclusive containers include most of the recent ATLAS software releases, database releases, and other tools extracted from CVMFS. This helped ATLAS to distribute software automatically to HPC centres with an environment identical to those in CVMFS. It also significantly reduced the metadata I/O load to HPC shared file systems. The production operation at NERSC has proved that by using this type of containers, we can transparently fit into the previously developed ATLAS operation methods, and at the same time scale up to run many more jobs.

NiAl precipitation in Fe-12w/o Cr-5w/o Ni-4w/o Al

1983 ◽  
Vol 41 ◽  
pp. 202-203
Author(s):  
L.E. Murr ◽  
J.S. Dunning ◽  
S. Shankar
Keyword(s):  
Solid Solution ◽  
Stainless Steel ◽  
Stainless Steels ◽  
Alloy Composition ◽  
Chromium Content ◽  
Scale Up ◽  
Optical Metallography ◽  
Property Evaluation ◽  
310 Stainless Steel ◽  
Microstructural Studies

Aluminum additions to conventional 18Cr-8Ni austenitic stainless steel compositions impart excellent resistance to high sulfur environments. However, problems are typically encountered with aluminum additions above about 1% due to embrittlement caused by aluminum in solid solution and the precipitation of NiAl. Consequently, little use has been made of aluminum alloy additions to stainless steels for use in sulfur or H2S environments in the chemical industry, energy conversion or generation, and mineral processing, for example.A research program at the Albany Research Center has concentrated on the development of a wrought alloy composition with as low a chromium content as possible, with the idea of developing a low-chromium substitute for 310 stainless steel (25Cr-20Ni) which is often used in high-sulfur environments. On the basis of workability and microstructural studies involving optical metallography on 100g button ingots soaked at 700°C and air-cooled, a low-alloy composition Fe-12Cr-5Ni-4Al (in wt %) was selected for scale up and property evaluation.

Sign in / Sign up

Export Citation Format

Share Document