scholarly journals Understanding the End-Users and Technical Requirements for Real-Time Streaming Data Analytics and Visualisation

2021 ◽  
Author(s):  
Fateme Dinmohammadi ◽  
Duncan Wilson
Keyword(s):  
Real Time ◽  
Data Analytics ◽  
End Users ◽  
Streaming Data ◽  
Technical Requirements

scholarly journals Fossel: Efficient Latency Reduction in Approximating Streaming Sensor Data

2020 ◽  
Vol 12 (23) ◽  
pp. 10175
Author(s):  
Fatima Abdullah ◽  
Limei Peng ◽  
Byungchul Tak
Keyword(s):  
Real Time ◽  
Data Analytics ◽  
Streaming Data ◽  
Sensor Data ◽  
Simple Type ◽  
Data Sampling ◽  
Time Data ◽  
Latency Reduction ◽  
Network Delays ◽  
Real Time Data

The volume of streaming sensor data from various environmental sensors continues to increase rapidly due to wider deployments of IoT devices at much greater scales than ever before. This, in turn, causes massive increase in the fog, cloud network traffic which leads to heavily delayed network operations. In streaming data analytics, the ability to obtain real time data insight is crucial for computational sustainability for many IoT enabled applications such as environmental monitors, pollution and climate surveillance, traffic control or even E-commerce applications. However, such network delays prevent us from achieving high quality real-time data analytics of environmental information. In order to address this challenge, we propose the Fog Sampling Node Selector (Fossel) technique that can significantly reduce the IoT network and processing delays by algorithmically selecting an optimal subset of fog nodes to perform the sensor data sampling. In addition, our technique performs a simple type of query executions within the fog nodes in order to further reduce the network delays by processing the data near the data producing devices. Our extensive evaluations show that Fossel technique outperforms the state-of-the-art in terms of latency reduction as well as in bandwidth consumption, network usage and energy consumption.

scholarly journals A Real-Time Patient Monitoring Framework for Fall Detection

2019 ◽  
Vol 2019 ◽  
pp. 1-13 ◽  
Author(s):  
Dharmitha Ajerla ◽  
Sazia Mahfuz ◽  
Farhana Zulkernine
Keyword(s):  
Real Time ◽  
Data Analytics ◽  
Short Term Memory ◽  
Detection System ◽  
Fall Detection ◽  
Edge Computing ◽  
Streaming Data ◽  
Real Time Analysis ◽  
Long Short Term Memory ◽  
Lstm Network

Fall detection is a major problem in the healthcare department. Elderly people are more prone to fall than others. There are more than 50% of injury-related hospitalizations in people aged over 65. Commercial fall detection devices are expensive and charge a monthly fee for their services. A more affordable and adaptable system is necessary for retirement homes and clinics to build a smart city powered by IoT and artificial intelligence. An effective fall detection system would detect a fall and send an alarm to the appropriate authorities. We propose a framework that uses edge computing where instead of sending data to the cloud, wearable devices send data to a nearby edge device like a laptop or mobile device for real-time analysis. We use cheap wearable sensor devices from MbientLab, an open source streaming engine called Apache Flink for streaming data analytics, and a long short-term memory (LSTM) network model for fall classification. The model is trained using a published dataset called “MobiAct.” Using the trained model, we analyse optimal sampling rates, sensor placement, and multistream data correction. Our edge computing framework can perform real-time streaming data analytics to detect falls with an accuracy of 95.8%.

HCache: A Hash-based Hybrid Caching Model for Real-Time Streaming Data Analytics

2018 ◽  
pp. 1-1
Author(s):  
Feng Zhao ◽  
Shao Feng Li ◽  
Bing B. Zhou ◽  
Hai Jin ◽  
Laurence T. Yang
Keyword(s):  
Real Time ◽  
Data Analytics ◽  
Streaming Data

Fog-Cloud Collaboration for Real-Time Streaming Applications

2019 ◽  
pp. 128-147
Author(s):  
Biji Nair ◽  
S. Mary Saira Bhanu
Keyword(s):  
Real Time ◽  
Data Analytics ◽  
High Speed ◽  
Fog Computing ◽  
Data Representation ◽  
Streaming Data ◽  
Cloud Data ◽  
Streaming Applications ◽  
Stream Management

Real-time streaming applications (RTSAs) generate huge volumes of temporally ordered, infinite, continuous, high speed data streams demanding both real-time and long-term data analytics. Fog computing is a reliable solution for processing and analyzing real-time streaming data as it offers low latency, location-aware, geographically distributed service at fog node and provides long-term services at the cloud data center (DC). This chapter addresses the challenge of coordinating the fog nodes and cloud for efficient processing of real-time streaming data in motion and at rest. The fog-cloud collaboration framework proposed in this chapter employs data stream management system (DSMS) schema at the fog node for real-time stream data processing and response generation. The data representation in micro-clusters at fog node and macro-clusters at DC facilitates accurate data analytics. The coordination between fog node and DC is through local ontology and global ontology respectively.

scholarly journals Designing Framework for Real Time Twitter Data Analytics using Apache Flume and Pig

2020 ◽  
Vol 8 (6) ◽  
pp. 4474-4477
Keyword(s):  
Real Time ◽  
Data Analytics ◽  
Social Issues ◽  
Analytical Framework ◽  
Streaming Data ◽  
Data Streaming ◽  
Text Data ◽  
Breaking News ◽  
Unstructured Text ◽  
Twitter Data

In the world of technology, people prefer social media to express themselves. Record says Twitter has more than 321 million active users with 100 million users posting approximately 340 million tweets a day. Twitter is the largest source of breaking news on social issues specially election-related where people can express their views also suggest their opinion. Twitter is generating unlimited unstructured text data. Hadoop is one of the finest tools accessible for analyzing twitter data because it supports processing of distributed big data, streaming data, time stamped data, text data etc. Whereas Apache Flume is used to extract real time twitter data into HDFS. This study attempts to establish an analytical framework to derive and interpret structured as well as unstructured Twitter data. The proposed framework comprises of real time twitter data insertion, its processing, and data visualization utilizing Apache Flume and pig. In this project we fetch positive and negative tweets on election data from twitter and analyzing the party status and the probability to win the election.

Leveraging Business Intelligence and Data Analytics in an Integrated Digital Production Platform to Unlock Optimization Potentials

2021 ◽  
Author(s):  
Ayesha Ahmed Abdulla Salem Alsaeedi ◽  
Manar Maher Mohamed Elabrashy ◽  
Mohamed Ali Alzeyoudi ◽  
Mohamed Mubarak Albadi ◽  
Sandeep Soni ◽  
...  
Keyword(s):  
Real Time ◽  
Business Intelligence ◽  
Data Analytics ◽  
System Optimization ◽  
End Users ◽  
Production Optimization ◽  
Gas Condensate ◽  
Production Platform ◽  
Digital Production ◽  
Multiple Variables

Abstract This paper discusses business intelligence algorithms and data analytics capabilities of an integrated digital production platform implemented in a giant gas condensate field. The advanced workflow focuses on helping the user navigate through the bulk of data to identify patterns and make predictions utilizing exception-based intelligence alarming. This helps derive insightful findings and provides recommendations for users to make efficient business decisions for achieving field potential optimization objectives. An Integrated digital production platform within a giant gas condensate field is implemented with numerous production optimization workflows encompassing daily well and facility performance monitoring and surveillance. The data integration within the systems is enhanced by integration with powerful Business Intelligence (BI) tools, enabling users to create customized dashboards, KPI screens, and exception-based alarm screens. An additional integration to the production platform is carried out with data from real-time sources like PI Asset Framework and corporate databases, improving the integrated production system's daily well and facility surveillance capabilities. The advanced integration of BI tools provided users with various opportunities to identify bottlenecks, production improvement chances, and troubleshooting areas by capitalizing insights from various dashboards and business KPI screens. Further, integrating these dashboards with several corporate data sources and a real-time asset data framework enabled users to harness maximized information embedded in the bulk of data. This also enabled end-users to harness maximized system potential, with all information available under a single collaborative platform. The integration powered by various inbuilt complex algorithms extended scripting capabilities, and enhanced visualization assisted the asset in realizing business KPIs requirements. Business intelligence algorithms in user interface established a drill-down approach to utilize information associated with multiple variables on top of one another. This allowed for the quick identification of trends and patterns in data. The customization approach helped the user to draw maximum information out of data as per their engineering requirements and current practices. This advanced integration facilitated users to minimize their efforts in traditional data analysis such as gathering, mapping, filtering, and plotting. With the help of these powerful features embedded in an integrated platform, the user was able to drive more focus on optimization and minimize time and effort on system configuration. This unique integration was one of its kind. An online integrated digital production platform comprising of wells, networks, and various workflows was integrated with business intelligence tools, thereby providing end-users tremendous opportunities related to system optimization.

Opinion Mining with Real Time Ontology Streaming Data

2019 ◽  
Vol 23 (1) ◽  
pp. 346-357
Author(s):  
Vithya G ◽  
Naren J ◽  
Varun V
Keyword(s):  
Real Time ◽  
Opinion Mining ◽  
Streaming Data

scholarly journals Stream Data Load Prediction for Resource Scaling Using Online Support Vector Regression

Algorithms ◽  
2019 ◽  
Vol 12 (2) ◽  
pp. 37 ◽  
Author(s):  
Zhigang Hu ◽  
Hui Kang ◽  
Meiguang Zheng
Keyword(s):  
Support Vector Regression ◽  
Real Time ◽  
Virtual Machines ◽  
Time Window ◽  
Performance Model ◽  
Streaming Data ◽  
Support Vector ◽  
Load Prediction ◽  
Stream Data ◽  
Online Support Vector Regression

A distributed data stream processing system handles real-time, changeable and sudden streaming data load. Its elastic resource allocation has become a fundamental and challenging problem with a fixed strategy that will result in waste of resources or a reduction in QoS (quality of service). Spark Streaming as an emerging system has been developed to process real time stream data analytics by using micro-batch approach. In this paper, first, we propose an improved SVR (support vector regression) based stream data load prediction scheme. Then, we design a spark-based maximum sustainable throughput of time window (MSTW) performance model to find the optimized number of virtual machines. Finally, we present a resource scaling algorithm TWRES (time window resource elasticity scaling algorithm) with MSTW constraint and streaming data load prediction. The evaluation results show that TWRES could improve resource utilization and mitigate SLA (service level agreement) violation.

Sign in / Sign up

Export Citation Format

Share Document