Performance Analysis of ECG Big Data using Apache Hive and Apache Pig

2019 ◽  
Author(s):  
Mudassar Ahmad ◽  
Safina Kanwal ◽  
Maryam Cheema ◽  
Muhammad Asif Habib
Keyword(s):  
Big Data ◽  
Performance Analysis ◽  
Apache Pig ◽  
Apache Hive

scholarly journals An Overview of Apache Pig and Apache Hive

2019 ◽  
pp. 432-436 ◽  
Author(s):  
Saiyam Arora ◽  
Abinesh Verma ◽  
Richa Vasuja ◽  
Richa Vasuja
Keyword(s):  
Big Data ◽  
Distributed Storage ◽  
Data Sets ◽  
Great Work ◽  
Apache Hadoop ◽  
The Social ◽  
Tremendous Amount ◽  
Hadoop Ecosystem ◽  
Apache Pig ◽  
Apache Hive

Ever since the enhancement of technology has taken place, the data is growing at an alarming rate. The most prominent factor of data growth is the “Social Media”, leads to the origination of a tremendous amount of data called Big Data. Big Data is a term used for data sets that are extremely large in size as well as complicated to store and process using traditional database processing applications. A saviour to deal with Big Data is “Hadoop” and two major components of Hadoop which are HDFS (Distributed Storage) and Map Reduce(Parallel Processing). Apache Pig and Hive is an essential part of the Hadoop Ecosystem. This paper covers an overview of both Apache Pig and Hive with their architecture. As Hadoop, no doubt is doing tremendously great work by storing and processing the huge volume of data but there are more frameworks now a days to increase the efficiency of Hadoop framework which are basically seen as the layers of Hadoop or a part of Apache Hadoop project. And that is why this paper includes the two most important layers namely Apache Pig and Apache Hive.

scholarly journals Hadoop Performance Analysis Model with Deep Data Locality

Information ◽  
2019 ◽  
Vol 10 (7) ◽  
pp. 222 ◽  
Author(s):  
Sungchul Lee ◽  
Ju-Yeon Jo ◽  
Yoohwan Kim
Keyword(s):  
Big Data ◽  
Performance Analysis ◽  
Data Locality ◽  
Performance Model ◽  
Data System ◽  
Analysis Model ◽  
Physical Test ◽  
Data Movement ◽  
Hadoop Distributed File System ◽  
Hadoop System

Background: Hadoop has become the base framework on the big data system via the simple concept that moving computation is cheaper than moving data. Hadoop increases a data locality in the Hadoop Distributed File System (HDFS) to improve the performance of the system. The network traffic among nodes in the big data system is reduced by increasing a data-local on the machine. Traditional research increased the data-local on one of the MapReduce stages to increase the Hadoop performance. However, there is currently no mathematical performance model for the data locality on the Hadoop. Methods: This study made the Hadoop performance analysis model with data locality for analyzing the entire process of MapReduce. In this paper, the data locality concept on the map stage and shuffle stage was explained. Also, this research showed how to apply the Hadoop performance analysis model to increase the performance of the Hadoop system by making the deep data locality. Results: This research proved the deep data locality for increasing performance of Hadoop via three tests, such as, a simulation base test, a cloud test and a physical test. According to the test, the authors improved the Hadoop system by over 34% by using the deep data locality. Conclusions: The deep data locality improved the Hadoop performance by reducing the data movement in HDFS.

Performance analysis of clustering algorithm under two kinds of big data architecture

2017 ◽  
Vol 23 (1) ◽  
pp. 49-57 ◽  
Author(s):  
Beibei Li ◽  
Bo Liu ◽  
Weiwei Lin ◽  
Ying Zhang
Keyword(s):  
Big Data ◽  
Performance Analysis ◽  
Clustering Algorithm ◽  
Data Architecture
Sign in / Sign up

Export Citation Format

Share Document