A Survey on Implementation of Word-Count with Map Reduce Programming Oriented Model using Hadoop Framework

2019 ◽  
Author(s):  
Santosh Yadav ◽  
Jay Prakash
Keyword(s):  
Map Reduce ◽  
Word Count ◽  
Hadoop Framework

Implementation of Word Count- Hadoop Framework with Map Reduce Algorithm

2017 ◽  
Vol 49 (3) ◽  
pp. 179-182 ◽  
Author(s):  
Keerthi Bangari ◽  
◽  
Sujitha Meduri ◽  
CY Rao
Keyword(s):  
Map Reduce ◽  
Word Count ◽  
Hadoop Framework

scholarly journals Distributed Framework for Processing High-Resolution Remote Sensing Images

2019 ◽  
Vol 9 (1) ◽  
pp. 4287-4292
Keyword(s):  
Remote Sensing ◽  
High Resolution ◽  
Map Reduce ◽  
Distributed Model ◽  
Remote Sensing Images ◽  
Satellite Technology ◽  
Distributed Framework ◽  
Block Based ◽  
Hadoop Framework

Now-a-days, sensing of remote satellite data processing is a very challenging task. The current development of satellite technology has led to explosive growth in quantity as well as the quality of the High-Resolution Remote Sensing (HRRS) images. These images can sometimes be in Gigabytes and Terabytes, which is heavy to load into the memory and also takes more time for processing. To address the challenges of processing HRRS images, a distributed map Reduce framework is proposed in this paper. This paper reflects Map-reduce as a distributed model using the Hadoop framework for processing large amounts of images. To process large amounts of images, block-based and size-based methods are introduced for effective processing. From the experiments, the proposed framework has proven to be effective in performance and speed.

scholarly journals Responsive Job Scheduling for Map-Reduce in Hadoop Framework

IJARCCE ◽  
2017 ◽  
Vol 6 (3) ◽  
pp. 745-747
Author(s):  
Poonam Mahajan ◽  
Manish Patel ◽  
Amol Agarwal ◽  
Nikhil Raut ◽  
Devendra Gadekar
Keyword(s):  
Job Scheduling ◽  
Map Reduce ◽  
Hadoop Framework

scholarly journals Implementation of Big-Data Applications Using Map Reduce Framework

2020 ◽  
Vol 9 (08) ◽  
pp. 25125-25131
Author(s):  
kapil Sahu ◽  
Kaveri Bhatt ◽  
Prof. Amit Saxena ◽  
Kaptan Singh
Keyword(s):  
Rapid Development ◽  
Efficiency Analysis ◽  
Map Reduce ◽  
Apriori Algorithm ◽  
Word Count ◽  
Computing Environment ◽  
Hadoop Mapreduce ◽  
Cloud Cluster ◽  
Big Data Applications ◽  
Hands On

Clustering As a result of the rapid development in cloud computing, it & fundamental to investigate the performance of extraordinary Hadoop MapReduce purposes and to realize the performance bottleneck in a cloud cluster that contributes to higher or diminish performance. It is usually primary to research the underlying hardware in cloud cluster servers to permit the optimization of program and hardware to achieve the highest performance feasible. Hadoop is founded on MapReduce, which is among the most popular programming items for huge knowledge analysis in a parallel computing environment. In this paper, we reward a particular efficiency analysis, characterization, and evaluation of Hadoop MapReduce Word Count utility. The main aim of this paper is to give implements of Hadoop map-reduce programming by giving a hands-on experience in developing Hadoop based Word-Count and Apriori application. Word count problem using Hadoop Map Reduce framework. The Apriori Algorithm has been used for finding frequent item set using Map Reduce framework.

Research of Distributed Search Engine Based on Hadoop (DSEH)

2013 ◽  
Vol 427-429 ◽  
pp. 2126-2129
Author(s):  
An Sheng Lu ◽  
Zi Hui Li ◽  
Hui Xiu Jin ◽  
Jia Yi Zhang ◽  
Hang Wei
Keyword(s):  
Web Service ◽  
Search Engine ◽  
System Structure ◽  
Map Reduce ◽  
Distributed Search ◽  
Key Technologies ◽  
Hadoop Framework ◽  
Forward System

The study of distributed search engine based on Hadoop (referred to as DSEH) has put forward system structure of distributed web service search engine based on Map/Reduce, and made introduction to related modules. Built the whole system on Hadoop framework by Map/Reduce and analyzed the key technologies of distributed search engine.

scholarly journals The File System Recommendations to Reduce the Space and Time Parameters in Hadoop File Storage and Map Reduce Processing of Big Data Applications

2020 ◽  
Vol 9 (10) ◽  
pp. 353-356
Keyword(s):  
Big Data ◽  
Data Processing ◽  
Data Storage ◽  
File System ◽  
Distributed File System ◽  
Map Reduce ◽  
Space And Time ◽  
File Storage ◽  
Hadoop Distributed File System ◽  
Hadoop Framework

The study of Hadoop Distributed File System (HDFS) and Map Reduce (MR) are the key aspects of the Hadoop framework. The big data scenarios like Face Book (FB) data processing or the twitter analytics such as storing the tweets and processing the tweets is other scenario of big data which can depends on Hadoop framework to perform the storage and processing through which further analytics can be done. The point here is the usage of space and time in the processing of the above-mentioned huge amounts of the data definitely leads to higher amounts of space and time consumption of the Hadoop framework. The problem here is usage of huge amounts of the space and at the same time the processing time is also high which need to be reduced so as to get the fastest response from the framework. The attempt is important as all the other eco system tools also depends on HDFS and MR so as to perform the data storage and processing of the data and alternative architecture so as to improve the usage of the space and effective utilization of the resources so as to reduce the time requirements of the framework. The outcome of the work is faster data processing and less space utilization of the framework in the processing of MR along with other eco system tools like Hive, Flume, Sqoop and Pig Latin. The work is proposing an alternative framework of the HDFS and MR and the name we are assigning is Unified Space Allocation and Data Processing with Metadata based Distributed File System (USAMDFS).

scholarly journals Sqoop usage in Hadoop Distributed File System and Observations to Handle Common Errors

2020 ◽  
Vol 9 (4) ◽  
pp. 452-454
Keyword(s):  
File System ◽  
Distributed File System ◽  
Current Work ◽  
Map Reduce ◽  
The Social ◽  
Source Data ◽  
Hadoop Distributed File System ◽  
Import And Export ◽  
The Common ◽  
Hadoop Framework

The Hadoop framework provides a way of storing and processing the huge amounts of the data. The social media like Facebook, twitter and amazon uses Hadoop eco system tools so as to store the data in Hadoop distributed file system and to process the data Map Reduce (MR). The current work describes the usage of Sqoop in the process of import and export with HDFS. The work involves various possible import/export commands supported by the tool Sqoop in the eco system of Hadoop. The importance of the work is to highlight the common errors while installing Sqoop and working with Sqoop. Many developers and researchers were using Sqoop so as to perform the import/export process and to handle the source data in the relational format. In the current work the connectivity between mysql and sqoop were presented and various commands usage along with the results were presented. The outcome of the work is for each command the possible errors encountered and the corresponding solution is mentioned. The common configuration settings we have to follow so as to handle the Sqoop without any errors is also mentioned

Sign in / Sign up

Export Citation Format

Share Document