scholarly journals A new Internet of Things architecture for real-time prediction of various diseases using machine learning on big data environment

2019 ◽  
Vol 6 (1) ◽  
Author(s):  
Abderrahmane Ed-daoudy ◽  
Khalil Maalmi
Keyword(s):  
Machine Learning ◽  
Big Data ◽  
Health Status ◽  
Real Time ◽  
Tracking System ◽  
Health Data ◽  
Streaming Data ◽  
Data Generation ◽  
Care Providers ◽  
Alert Message

AbstractA number of technologies enabled by Internet of Thing (IoT) have been used for the prevention of various chronic diseases, continuous and real-time tracking system is a particularly important one. Wearable medical devices with sensor, health cloud and mobile applications have continuously generating a huge amount of data which is often called as streaming big data. Due to the higher speed of the data generation, it is difficult to collect, process and analyze such massive data in real-time in order to perform real-time actions in case of emergencies and extracting hidden value. using traditional methods which are limited and time-consuming. Therefore, there is a significant need to real-time big data stream processing to ensure an effective and scalable solution. In order to overcome this issue, this work proposes a new architecture for real-time health status prediction and analytics system using big data technologies. The system focus on applying distributed machine learning model on streaming health data events ingested to Spark streaming through Kafka topics. Firstly, we transform the standard decision tree (DT) (C4.5) algorithm into a parallel, distributed, scalable and fast DT using Spark instead of Hadoop MapReduce which becomes limited for real-time computing. Secondly, this model is applied to streaming data coming from distributed sources of various diseases to predict health status. Based on several input attributes, the system predicts health status, send an alert message to care providers and store the details in a distributed database to perform health data analytics and stream reporting. We measure the performance of Spark DT against traditional machine learning tools including Weka. Finally, performance evaluation parameters such as throughput and execution time are calculated to show the effectiveness of the proposed architecture. The experimental results show that the proposed system is able to effectively process and predict real-time and massive amount of medical data enabled by IoT from distributed and various diseases.

Decision Based Model for Real-Time IoT Analysis Using Big Data and Machine Learning

2021 ◽  
Author(s):  
Hina Jamil ◽  
Tariq Umer ◽  
Celal Ceken ◽  
Fadi Al-Turjman
Keyword(s):  
Machine Learning ◽  
Big Data ◽  
Real Time

Cloud-Based Big Data Analysis Tools and Techniques Towards Sustainable Smart City Services

2021 ◽  
pp. 63-90
Author(s):  
Suresh P. ◽  
Keerthika P. ◽  
Sathiyamoorthi V. ◽  
Logeswaran K. ◽  
Manjula Devi R. ◽  
...  
Keyword(s):  
Big Data ◽  
Real Time ◽  
Smart City ◽  
Traffic Management ◽  
Smart Cities ◽  
Big Data Analytics ◽  
Streaming Data ◽  
City Development ◽  
Time Data ◽  
Public And Private

Cloud computing and big data analytics are the key parts of smart city development that can create reliable, secure, healthier, more informed communities while producing tremendous data to the public and private sectors. Since the various sectors of smart cities generate enormous amounts of streaming data from sensors and other devices, storing and analyzing this huge real-time data typically entail significant computing capacity. Most smart city solutions use a combination of core technologies such as computing, storage, databases, data warehouses, and advanced technologies such as analytics on big data, real-time streaming data, artificial intelligence, machine learning, and the internet of things (IoT). This chapter presents a theoretical and experimental perspective on the smart city services such as smart healthcare, water management, education, transportation and traffic management, and smart grid that are offered using big data management and cloud-based analytics services.

scholarly journals Data-driven decision support under concept drift in streamed big data

2019 ◽  
Vol 6 (1) ◽  
pp. 157-163 ◽  
Author(s):  
Jie Lu ◽  
Anjin Liu ◽  
Yiliao Song ◽  
Guangquan Zhang
Keyword(s):  
Decision Making ◽  
Big Data ◽  
Real Time ◽  
Concept Drift ◽  
High Volume ◽  
Streaming Data ◽  
Data Driven ◽  
Research Directions ◽  
Decision Outcomes ◽  
Past Data

Abstract Data-driven decision-making ($$\mathrm {D^3}$$D3M) is often confronted by the problem of uncertainty or unknown dynamics in streaming data. To provide real-time accurate decision solutions, the systems have to promptly address changes in data distribution in streaming data—a phenomenon known as concept drift. Past data patterns may not be relevant to new data when a data stream experiences significant drift, thus to continue using models based on past data will lead to poor prediction and poor decision outcomes. This position paper discusses the basic framework and prevailing techniques in streaming type big data and concept drift for $$\mathrm {D^3}$$D3M. The study first establishes a technical framework for real-time $$\mathrm {D^3}$$D3M under concept drift and details the characteristics of high-volume streaming data. The main methodologies and approaches for detecting concept drift and supporting $$\mathrm {D^3}$$D3M are highlighted and presented. Lastly, further research directions, related methods and procedures for using streaming data to support decision-making in concept drift environments are identified. We hope the observations in this paper could support researchers and professionals to better understand the fundamentals and research directions of $$\mathrm {D^3}$$D3M in streamed big data environments.

scholarly journals Streaming feature selection algorithms for big data: A survey

2020 ◽  
Vol ahead-of-print (ahead-of-print) ◽  
Author(s):  
Noura AlNuaimi ◽  
Mohammad Mehedy Masud ◽  
Mohamed Adel Serhani ◽  
Nazar Zaki
Keyword(s):  
Machine Learning ◽  
Feature Selection ◽  
Big Data ◽  
Real Time ◽  
Relevant Literature ◽  
Heterogeneous Data ◽  
Proper Solution ◽  
Exact Figure ◽  
Selection Algorithms ◽  
Over Time

Organizations in many domains generate a considerable amount of heterogeneous data every day. Such data can be processed to enhance these organizations’ decisions in real time. However, storing and processing large and varied datasets (known as big data) is challenging to do in real time. In machine learning, streaming feature selection has always been considered a superior technique for selecting the relevant subset features from highly dimensional data and thus reducing learning complexity. In the relevant literature, streaming feature selection refers to the features that arrive consecutively over time; despite a lack of exact figure on the number of features, numbers of instances are well-established. Many scholars in the field have proposed streaming-feature-selection algorithms in attempts to find the proper solution to this problem. This paper presents an exhaustive and methodological introduction of these techniques. This study provides a review of the traditional feature-selection algorithms and then scrutinizes the current algorithms that use streaming feature selection to determine their strengths and weaknesses. The survey also sheds light on the ongoing challenges in big-data research.

Sign in / Sign up

Export Citation Format

Share Document