scholarly journals Review of Big Data and Processing Frameworks for Disaster Response Applications

2019 ◽  
Vol 8 (9) ◽  
pp. 387 ◽  
Author(s):  
Silvino Pedro Cumbane ◽  
Gyozo Gidófalvi
Keyword(s):  
Machine Learning ◽  
Big Data ◽  
Data Processing ◽  
Disaster Management ◽  
Disaster Response ◽  
Human Life ◽  
Streaming Data ◽  
Data Sources ◽  
Big Data Processing ◽  
Processing Framework

Natural hazards result in devastating losses in human life, environmental assets and personal, and regional and national economies. The availability of different big data such as satellite imageries, Global Positioning System (GPS) traces, mobile Call Detail Records (CDRs), social media posts, etc., in conjunction with advances in data analytic techniques (e.g., data mining and big data processing, machine learning and deep learning) can facilitate the extraction of geospatial information that is critical for rapid and effective disaster response. However, disaster response systems development usually requires the integration of data from different sources (streaming data sources and data sources at rest) with different characteristics and types, which consequently have different processing needs. Deciding which processing framework to use for a specific big data to perform a given task is usually a challenge for researchers from the disaster management field. Therefore, this paper contributes in four aspects. Firstly, potential big data sources are described and characterized. Secondly, the big data processing frameworks are characterized and grouped based on the sources of data they handle. Then, a short description of each big data processing framework is provided and a comparison of processing frameworks in each group is carried out considering the main aspects such as computing cluster architecture, data flow, data processing model, fault-tolerance, scalability, latency, back-pressure mechanism, programming languages, and support for machine learning libraries, which are related to specific processing needs. Finally, a link between big data and processing frameworks is established, based on the processing provisioning for essential tasks in the response phase of disaster management.

Machine Learning Based Big Data Processing Framework for Cancer Diagnosis Using Hidden Markov Model and GM Clustering

2017 ◽  
Vol 102 (3) ◽  
pp. 2099-2116 ◽  
Author(s):  
Gunasekaran Manogaran ◽  
V. Vijayakumar ◽  
R. Varatharajan ◽  
Priyan Malarvizhi Kumar ◽  
Revathi Sundarasekar ◽  
...  
Keyword(s):  
Machine Learning ◽  
Big Data ◽  
Data Processing ◽  
Markov Model ◽  
Hidden Markov Model ◽  
Cancer Diagnosis ◽  
Hidden Markov ◽  
Big Data Processing ◽  
Processing Framework

Astronomical big data processing using machine learning: A comprehensive review

2022 ◽  
Author(s):  
Snigdha Sen ◽  
Sonali Agarwal ◽  
Pavan Chakraborty ◽  
Krishna Pratap Singh
Keyword(s):  
Machine Learning ◽  
Big Data ◽  
Data Processing ◽  
Big Data Processing ◽  
Comprehensive Review

scholarly journals Big Data Usage Patterns in the Health Care Domain: A Use Case Driven Approach Applied to the Assessment of Vaccination Benefits and Risks

2014 ◽  
Vol 23 (01) ◽  
pp. 27-35 ◽  
Author(s):  
S. de Lusignan ◽  
S-T. Liaw ◽  
C. Kuziemsky ◽  
F. Mold ◽  
P. Krause ◽  
...  
Keyword(s):  
Big Data ◽  
Data Processing ◽  
Real Time ◽  
Predictive Analytics ◽  
Regulatory Compliance ◽  
Use Cases ◽  
Data Sources ◽  
Use Case ◽  
Crowd Sourcing ◽  
Big Data Processing

Summary Background: Generally benefits and risks of vaccines can be determined from studies carried out as part of regulatory compliance, followed by surveillance of routine data; however there are some rarer and more long term events that require new methods. Big data generated by increasingly affordable personalised computing, and from pervasive computing devices is rapidly growing and low cost, high volume, cloud computing makes the processing of these data inexpensive. Objective: To describe how big data and related analytical methods might be applied to assess the benefits and risks of vaccines. Method: We reviewed the literature on the use of big data to improve health, applied to generic vaccine use cases, that illustrate benefits and risks of vaccination. We defined a use case as the interaction between a user and an information system to achieve a goal. We used flu vaccination and pre-school childhood immunisation as exemplars. Results: We reviewed three big data use cases relevant to assessing vaccine benefits and risks: (i) Big data processing using crowd-sourcing, distributed big data processing, and predictive analytics, (ii) Data integration from heterogeneous big data sources, e.g. the increasing range of devices in the “internet of things”, and (iii) Real-time monitoring for the direct monitoring of epidemics as well as vaccine effects via social media and other data sources. Conclusions: Big data raises new ethical dilemmas, though its analysis methods can bring complementary real-time capabilities for monitoring epidemics and assessing vaccine benefit-risk balance.

A Study of Big Data Processing for Sentiments Analysis

2022 ◽  
pp. 1162-1191
Author(s):  
Dinesh Chander ◽  
Hari Singh ◽  
Abhinav Kirti Gupta
Keyword(s):  
Big Data ◽  
Data Processing ◽  
Data Analytics ◽  
Important Application ◽  
Streaming Data ◽  
Test Case ◽  
Big Data Processing ◽  
Important Field ◽  
Batch Data ◽  
Different Sources

Data processing has become an important field in today's big data-dominated world. The data has been generating at a tremendous pace from different sources. There has been a change in the nature of data from batch-data to streaming-data, and consequently, data processing methodologies have also changed. Traditional SQL is no longer capable of dealing with this big data. This chapter describes the nature of data and various tools, techniques, and technologies to handle this big data. The chapter also describes the need of shifting big data on to cloud and the challenges in big data processing in the cloud, the migration from data processing to data analytics, tools used in data analytics, and the issues and challenges in data processing and analytics. Then the chapter touches an important application area of streaming data, sentiment analysis, and tries to explore it through some test case demonstrations and results.

scholarly journals Big data processing framework for manufacturing

Procedia CIRP ◽  
2019 ◽  
Vol 83 ◽  
pp. 661-664 ◽  
Author(s):  
Yinghao Ye ◽  
Meilin Wang ◽  
Shuhong Yao ◽  
Jarvis N. Jiang ◽  
Qing Liu
Keyword(s):  
Big Data ◽  
Data Processing ◽  
Big Data Processing ◽  
Processing Framework

scholarly journals Attributes Reduction in Big Data

2020 ◽  
Vol 10 (14) ◽  
pp. 4901
Author(s):  
Waleed Albattah ◽  
Rehan Ullah Khan ◽  
Khalil Khan
Keyword(s):  
Machine Learning ◽  
Big Data ◽  
Data Processing ◽  
Data Analytics ◽  
Detection Efficiency ◽  
Large Data ◽  
Model Learning ◽  
Big Data Processing ◽  
Performance Loss ◽  
Points Of View

Processing big data requires serious computing resources. Because of this challenge, big data processing is an issue not only for algorithms but also for computing resources. This article analyzes a large amount of data from different points of view. One perspective is the processing of reduced collections of big data with less computing resources. Therefore, the study analyzed 40 GB data to test various strategies to reduce data processing. Thus, the goal is to reduce this data, but not to compromise on the detection and model learning in machine learning. Several alternatives were analyzed, and it is found that in many cases and types of settings, data can be reduced to some extent without compromising detection efficiency. Tests of 200 attributes showed that with a performance loss of only 4%, more than 80% of the data could be ignored. The results found in the study, thus provide useful insights into large data analytics.

Sign in / Sign up

Export Citation Format

Share Document