scholarly journals Image Processing in Hadoop Distributed Environment

10.29007/hxcv ◽  
2018 ◽  
Author(s):  
Mosab Shaheen ◽  
Madhukar B Potdar
Keyword(s):  
Image Processing ◽  
Big Data ◽  
Data Processing ◽  
Satellite Images ◽  
Unsupervised Classification ◽  
Distributed Environment ◽  
Big Data Processing ◽  
Multispectral Satellite Images ◽  
Hadoop Framework

Satellite images are a growing resource of information and have many applications. In this research, the multispectral satellite images have been subjected to unsupervised classification based on K-Means clustering using Hadoop Framework, which is designed for big data processing, along with Hadoop Image Processing Interface (HIPI). We developed support for the GeoTIFF format, which is usually used for satellite images, and we will show that our methodology enhances the performance.

scholarly journals The Experimental Study of Performance Impairment of Big Data Processing in Dynamic and Opportunistic Environments

2020 ◽  
pp. 776-789
Author(s):  
Wei Li ◽  
◽  
William W. Guo
Keyword(s):  
Big Data ◽  
Impact Strength ◽  
Data Processing ◽  
Impact Factors ◽  
Distributed Environments ◽  
Distributed Environment ◽  
Big Data Processing ◽  
Performance Impairment ◽  
Result Analysis ◽  
The Impact

In contrast to HPC clusters, when big data is processing in a distributed, particularly dynamic and opportunistic environment, the overall performance must be impaired and even bottlenecked by the dynamics of overlay and the opportunism of computing nodes. The dynamics and opportunism are caused by churn and unreliability of a generic distributed environment, and they cannot be ignored or avoided. Understanding impact factors, their impact strength and the relevance between these impacts is the foundation of potential optimization. This paper derives the research background, methodology and results by reasoning the necessity of distributed environments for big data processing, scrutinizing the dynamics and opportunism of distributed environments, classifying impact factors, proposing evaluation metrics and carrying out a series of intensive experiments. The result analysis of this paper provides important insights to the impact strength of the factors and the relevance of impact across the factors. The production of the results aims at paving a way to future optimization or avoidance of potential bottlenecks for big data processing in distributed environments.

scholarly journals A COMPARATIVE ANALYSIS OF CONVENTIONAL HADOOP WITH PROPOSED CLOUD ENABLED HADOOP FRAMEWORK FOR SPATIAL BIG DATA PROCESSING

2018 ◽  
Vol IV-5 ◽  
pp. 425-430 ◽  
Author(s):  
A. K. Tripathi ◽  
S. Agrawal ◽  
R. D. Gupta
Keyword(s):  
Big Data ◽  
Data Processing ◽  
High Performance ◽  
Computing Environment ◽  
Big Data Processing ◽  
Huge Data ◽  
Spatial Big Data ◽  
New Research ◽  
Performance Computing ◽  
Hadoop Framework

<p><strong>Abstract.</strong> The emergence of new tools and technologies to gather the information generate the problem of processing spatial big data. The solution of this problem requires new research, techniques, innovation and development. Spatial big data is categorized by the five V’s: volume, velocity, veracity, variety and value. Hadoop is a most widely used framework which address these problems. But it requires high performance computing resources to store and process such huge data. The emergence of cloud computing has provided, on demand, elastic, scalable and payment based computing resources to users to develop their own computing environment. The main objective of this paper is to develop a cloud enabled hadoop framework which combines cloud technology and high computing resources with the conventional hadoop framework to support the spatial big data solutions. The paper also compares the conventional hadoop framework and proposed cloud enabled hadoop framework. It is observed that the propose cloud enabled hadoop framework is much efficient to spatial big data processing than the current available solutions.</p>

Load Balancing in Big Data Processing Systems

2019 ◽  
Vol 12 (1) ◽  
pp. 42 ◽  
Author(s):  
Andrey I. Vlasov ◽  
Konstantin A. Muraviev ◽  
Alexandra A. Prudius ◽  
Demid A. Uzenkov
Keyword(s):  
Big Data ◽  
Load Balancing ◽  
Data Processing ◽  
Big Data Processing
Sign in / Sign up

Export Citation Format

Share Document