Optimizing the Hadoop MapReduce Framework with high-performance storage devices

2015 ◽  
Vol 71 (9) ◽  
pp. 3525-3548 ◽  
Author(s):  
Sangwhan Moon ◽  
Jaehwan Lee ◽  
Xiling Sun ◽  
Yang-suk Kee
Keyword(s):  
High Performance ◽  
Mapreduce Framework ◽  
Storage Devices ◽  
Hadoop Mapreduce

scholarly journals Computational storage: an efficient and scalable platform for big data and HPC applications

2019 ◽  
Vol 6 (1) ◽  
Author(s):  
Mahdi Torabzadehkashi ◽  
Siavash Rezaei ◽  
Ali HeydariGorji ◽  
Hosein Bobarshad ◽  
Vladimir Alves ◽  
...  
Keyword(s):  
Big Data ◽  
High Performance ◽  
Distributed Processing ◽  
Data Access ◽  
Distributed Applications ◽  
Process Data ◽  
Storage Devices ◽  
Hadoop Mapreduce ◽  
Big Data Applications ◽  
Application Processor

AbstractIn the era of big data applications, the demand for more sophisticated data centers and high-performance data processing mechanisms is increasing drastically. Data are originally stored in storage systems. To process data, application servers need to fetch them from storage devices, which imposes the cost of moving data to the system. This cost has a direct relation with the distance of processing engines from the data. This is the key motivation for the emergence of distributed processing platforms such as Hadoop, which move process closer to data. Computational storage devices (CSDs) push the “move process to data” paradigm to its ultimate boundaries by deploying embedded processing engines inside storage devices to process data. In this paper, we introduce Catalina, an efficient and flexible computational storage platform, that provides a seamless environment to process data in-place. Catalina is the first CSD equipped with a dedicated application processor running a full-fledged operating system that provides filesystem-level data access for the applications. Thus, a vast spectrum of applications can be ported for running on Catalina CSDs. Due to these unique features, to the best of our knowledge, Catalina CSD is the only in-storage processing platform that can be seamlessly deployed in clusters to run distributed applications such as Hadoop MapReduce and HPC applications in-place without any modifications on the underlying distributed processing framework. For the proof of concept, we build a fully functional Catalina prototype and a CSD-equipped platform using 16 Catalina CSDs to run Intel HiBench Hadoop and HPC benchmarks to investigate the benefits of deploying Catalina CSDs in the distributed processing environments. The experimental results show up to 2.2× improvement in performance and 4.3× reduction in energy consumption, respectively, for running Hadoop MapReduce benchmarks. Additionally, thanks to the Neon SIMD engines, the performance and energy efficiency of DFT algorithms are improved up to 5.4× and 8.9×, respectively.

scholarly journals Referential DNA Data Compression using Hadoop Map Reduce Framework

2019 ◽  
Vol 17 (2) ◽  
pp. 207-214
Author(s):  
Raju Bhukya ◽  
Sumit Deshmuk
Keyword(s):  
Data Compression ◽  
Dna Sequences ◽  
High Performance ◽  
Deoxyribonucleic Acid ◽  
Genetic Data ◽  
Disk Arrays ◽  
Distributed Environment ◽  
Mapreduce Framework ◽  
Hadoop Mapreduce ◽  
Time Required

The indispensable knowledge of Deoxyribonucleic Acid (DNA) sequences and sharply reducing cost of the DNA sequencing techniques has attracted numerous researchers in the field of Genetics. These sequences are getting available at an exponential rate leading to the bulging size of molecular biology databases making large disk arrays and compute clusters inevitable for analysis.In this paper, we proposed referential DNA data compression using hadoop MapReduce Framework to process humongous amount of genetic data in distributed environment on high performance compute clusters. Our method has successfully achieved a better balance between compression ratio and the amount of time required for DNA data compression as compared to other Referential DNA Data Compression methods.

Mixed ternary metal MFeCo (M = Al, Mg, Cu, Zn, or Ni) oxide electrodes for high-performance energy storage devices

Ionics ◽  
2021 ◽  
Author(s):  
Morteza Saghafi Yazdi ◽  
Seied Ali Hosseini ◽  
Zeynodin Karami ◽  
Ali Olamaee ◽  
Mohammad Abedini ◽  
...  
Keyword(s):  
Energy Storage ◽  
High Performance ◽  
Energy Storage Devices ◽  
Storage Devices ◽  
Oxide Electrodes ◽  
Ternary Metal ◽  
Ni Oxide

scholarly journals Oxygen-Deficient Stannic Oxide/Graphene for Ultrahigh-Performance Supercapacitors and Gas Sensors

Nanomaterials ◽  
2021 ◽  
Vol 11 (2) ◽  
pp. 372
Author(s):  
Liyang Lin ◽  
Susu Chen ◽  
Tao Deng ◽  
Wen Zeng
Keyword(s):  
Energy Storage ◽  
Gas Sensors ◽  
High Performance ◽  
Gas Sensing ◽  
Synthesis Method ◽  
Rate Capability ◽  
Stannic Oxide ◽  
Energy Storage Devices ◽  
Graphene Nanocomposites ◽  
Storage Devices

The metal oxides/graphene nanocomposites have great application prospects in the fields of electrochemical energy storage and gas sensing detection. However, rational synthesis of such materials with good conductivity and electrochemical activity is the topical challenge for high-performance devices. Here, SnO2/graphene nanocomposite is taken as a typical example and develops a universal synthesis method that overcome these challenges and prepares the oxygen-deficient SnO2 hollow nanospheres/graphene (r-SnO2/GN) nanocomposite with excellent performance for supercapacitors and gas sensors. The electrode r-SnO2/GN exhibits specific capacitance of 947.4 F g−1 at a current density of 2 mA cm−2 and of 640.0 F g−1 even at 20 mA cm−2, showing remarkable rate capability. For gas-sensing application, the sensor r-SnO2/GN showed good sensitivity (~13.8 under 500 ppm) and short response/recovering time toward methane gas. These performance features make r-SnO2/GN nanocomposite a promising candidate for high-performance energy storage devices and gas sensors.

scholarly journals Layered double hydroxide membrane with high hydroxide conductivity and ion selectivity for energy storage device

2021 ◽  
Vol 12 (1) ◽  
Author(s):  
Jing Hu ◽  
Xiaomin Tang ◽  
Qing Dai ◽  
Zhiqiang Liu ◽  
Huamin Zhang ◽  
...  
Keyword(s):  
Energy Storage ◽  
High Performance ◽  
Ion Selectivity ◽  
Storage Device ◽  
Hydroxyl Groups ◽  
Flow Battery ◽  
Energy Storage Devices ◽  
Storage Devices ◽  
Ions Transport ◽  
Double Hydroxides

AbstractMembranes with fast and selective ions transport are highly demanded for energy storage devices. Layered double hydroxides (LDHs), bearing uniform interlayer galleries and abundant hydroxyl groups covalently bonded within two-dimensional (2D) host layers, make them superb candidates for high-performance membranes. However, related research on LDHs for ions separation is quite rare, especially the deep-going study on ions transport behavior in LDHs. Here, we report a LDHs-based composite membrane with fast and selective ions transport for flow battery application. The hydroxide ions transport through LDHs via vehicular (standard diffusion) & Grotthuss (proton hopping) mechanisms is uncovered. The LDHs-based membrane enables an alkaline zinc-based flow battery to operate at 200 mA cm−2, along with an energy efficiency of 82.36% for 400 cycles. This study offers an in-depth understanding of ions transport in LDHs and further inspires their applications in other energy-related devices.

Catalytic Separator with Co–N–C Nanoreactor for High-Performance Lithium-Sulfur Batteries

2021 ◽  
Author(s):  
Longtao Ren ◽  
Qian Wang ◽  
Yajie Li ◽  
Cejun Hu ◽  
Yajun Zhao ◽  
...  
Keyword(s):  
Energy Storage ◽  
Energy Density ◽  
High Performance ◽  
Next Generation ◽  
Energy Storage Devices ◽  
Storage Devices ◽  
Lithium Sulfur Batteries ◽  
Lithium Sulfur

Rechargeable lithium-sulfur (Li–S) batteries are considered one of the most promising next-generation energy storage devices because of their high theoretical energy density. However, the dissolution of lithium polysulfides (LiPSs) in...

scholarly journals A germanium and zinc chalcogenide as an anode for a high-capacity and long cycle life lithium battery

RSC Advances ◽  
2019 ◽  
Vol 9 (60) ◽  
pp. 35045-35049
Author(s):  
Xu Chen ◽  
Jian Zhou ◽  
Jiarui Li ◽  
Haiyan Luo ◽  
Lin Mei ◽  
...  
Keyword(s):  
Energy Storage ◽  
High Performance ◽  
Lithium Battery ◽  
Large Scale ◽  
High Capacity ◽  
Lithium Ion ◽  
Energy Storage Devices ◽  
Storage Devices ◽  
Long Cycle Life ◽  
Zinc Chalcogenide

High-performance lithium ion batteries are ideal energy storage devices for both grid-scale and large-scale applications.

Sign in / Sign up

Export Citation Format

Share Document