Benchmarking Apache Spark and Hadoop MapReduce on Big Data Classification

2021 ◽  
Author(s):  
Taha Tekdogan ◽  
Ali Cakmak
Keyword(s):  
Big Data ◽  
Data Classification ◽  
Apache Spark ◽  
Hadoop Mapreduce ◽  
Big Data Classification

scholarly journals The Deep Learning and Apache Spark Enabled Architecture for Improving the Performance of Big Data Classification

2019 ◽  
Vol 8 (11) ◽  
pp. 2908-2914
Keyword(s):  
Big Data ◽  
Deep Learning ◽  
Review Paper ◽  
Data Classification ◽  
Apache Spark ◽  
The Novel ◽  
Second Stage ◽  
Big Data Applications ◽  
Big Data Classification ◽  
Two Stages

At present the Big Data applications, for example, informal communication, therapeutic human services, horticulture, banking, financial exchange, instruction, Facebook and so forth are producing the information with extremely rapid. Volume and Velocity of the Big information assumes a significant job in the presentation of Big information applications. Execution of the Big information application can be influenced by different parameters. Expediently search, proficiency and precision are the a portion of the overwhelming parameters which influence the general execution of any Big information applications. Due the immediate and aberrant inclusion of the qualities of 7Vs of Big information, each Big Data administrations anticipate the elite. Elite is the greatest test in the present evolving situation. In this paper we propose the Big Data characterization way to deal with speedup the Big Data applications. This paper is the review paper, we allude different Big information advancements and the related work in the field of Big Data Classification. In the wake of learning and understanding the writing we discover the holes in existing work and techniques. Finally we propose the novel methodology of Big Data characterization. Our methodology relies on the Deep Learning and Apache Spark engineering. In the proposed work two stages are appeared; first stage is include choice and second stage is Big Data Classification. Apache Spark is the most reasonable and predominant innovation to execute this proposed work. Apache Spark is having two hubs; introductory hubs and last hubs. The element choice will be occur in introductory hubs and Big Data Classification will happen in definite hubs of Apache Spark

scholarly journals An Efficient Procedure for Classification of Big Data using the Deep Learning Enabled Spark Architecture and Machine Learning

2019 ◽  
Vol 8 (2S11) ◽  
pp. 2662-2668
Keyword(s):  
Big Data ◽  
Deep Learning ◽  
Data Classification ◽  
Apache Spark ◽  
The Novel ◽  
Efficient Procedure ◽  
Big Data Applications ◽  
Big Data Classification ◽  
Two Stages

At present the Big Data applications, for example, informal communication, therapeutic human services, horticulture, banking, financial exchange, instruction, Facebook and so forth are producing the information with extremely rapid. Volume and Velocity of the Big information assumes a significant job in the presentation of Big information applications. Execution of the Big information application can be influenced by different parameters. Expediently search, proficiency and precision are the a portion of the overwhelming parameters which influence the general execution of any Big information applications. Due the immediate and aberrant inclusion of the qualities of 7Vs of Big information, each Big Data administrations anticipate the elite. Elite is the greatest test in the present evolving situation. In this paper we propose the Big Data characterization way to deal with speedup the Big Data applications. This paper is the review paper, we allude different Big information advancements and the related work in the field of Big Data Classification. In the wake of learning and understanding the writing we discover the holes in existing work and techniques. Finally we propose the novel methodology of Big Data characterization. Our methodology relies on the Deep Learning and Apache Spark engineering. In the proposed work two stages are appeared; first stage is include choice and second stage is Big Data Classification. Apache Spark is the most reasonable and predominant innovation to execute this proposed work. Apache Spark is having two hubs; introductory hubs and last hubs. The element choice will be occur in introductory hubs and Big Data Classification will happen in definite hubs of Apache Spark.

Research on complex attribute big data classification based on iterative fuzzy clustering algorithm

2021 ◽  
pp. 1-12
Author(s):  
Li Qian
Keyword(s):  
Big Data ◽  
Fuzzy Clustering ◽  
Classification Accuracy ◽  
Clustering Algorithm ◽  
Principal Component ◽  
Data Classification ◽  
Fisher Discriminant Analysis ◽  
Fuzzy Clustering Algorithm ◽  
Local Fisher Discriminant Analysis ◽  
Big Data Classification

In order to overcome the low classification accuracy of traditional methods, this paper proposes a new classification method of complex attribute big data based on iterative fuzzy clustering algorithm. Firstly, principal component analysis and kernel local Fisher discriminant analysis were used to reduce dimensionality of complex attribute big data. Then, the Bloom Filter data structure is introduced to eliminate the redundancy of the complex attribute big data after dimensionality reduction. Secondly, the redundant complex attribute big data is classified in parallel by iterative fuzzy clustering algorithm, so as to complete the complex attribute big data classification. Finally, the simulation results show that the accuracy, the normalized mutual information index and the Richter’s index of the proposed method are close to 1, the classification accuracy is high, and the RDV value is low, which indicates that the proposed method has high classification effectiveness and fast convergence speed.

Sign in / Sign up

Export Citation Format

Share Document