HCache: A Hash-based Hybrid Caching Model for Real-Time Streaming Data Analytics

Author(s):  
Feng Zhao ◽  
Shao Feng Li ◽  
Bing B. Zhou ◽  
Hai Jin ◽  
Laurence T. Yang
2020 ◽  
Vol 12 (23) ◽  
pp. 10175
Author(s):  
Fatima Abdullah ◽  
Limei Peng ◽  
Byungchul Tak

The volume of streaming sensor data from various environmental sensors continues to increase rapidly due to wider deployments of IoT devices at much greater scales than ever before. This, in turn, causes massive increase in the fog, cloud network traffic which leads to heavily delayed network operations. In streaming data analytics, the ability to obtain real time data insight is crucial for computational sustainability for many IoT enabled applications such as environmental monitors, pollution and climate surveillance, traffic control or even E-commerce applications. However, such network delays prevent us from achieving high quality real-time data analytics of environmental information. In order to address this challenge, we propose the Fog Sampling Node Selector (Fossel) technique that can significantly reduce the IoT network and processing delays by algorithmically selecting an optimal subset of fog nodes to perform the sensor data sampling. In addition, our technique performs a simple type of query executions within the fog nodes in order to further reduce the network delays by processing the data near the data producing devices. Our extensive evaluations show that Fossel technique outperforms the state-of-the-art in terms of latency reduction as well as in bandwidth consumption, network usage and energy consumption.


2019 ◽  
Vol 2019 ◽  
pp. 1-13 ◽  
Author(s):  
Dharmitha Ajerla ◽  
Sazia Mahfuz ◽  
Farhana Zulkernine

Fall detection is a major problem in the healthcare department. Elderly people are more prone to fall than others. There are more than 50% of injury-related hospitalizations in people aged over 65. Commercial fall detection devices are expensive and charge a monthly fee for their services. A more affordable and adaptable system is necessary for retirement homes and clinics to build a smart city powered by IoT and artificial intelligence. An effective fall detection system would detect a fall and send an alarm to the appropriate authorities. We propose a framework that uses edge computing where instead of sending data to the cloud, wearable devices send data to a nearby edge device like a laptop or mobile device for real-time analysis. We use cheap wearable sensor devices from MbientLab, an open source streaming engine called Apache Flink for streaming data analytics, and a long short-term memory (LSTM) network model for fall classification. The model is trained using a published dataset called “MobiAct.” Using the trained model, we analyse optimal sampling rates, sensor placement, and multistream data correction. Our edge computing framework can perform real-time streaming data analytics to detect falls with an accuracy of 95.8%.


Author(s):  
Biji Nair ◽  
S. Mary Saira Bhanu

Real-time streaming applications (RTSAs) generate huge volumes of temporally ordered, infinite, continuous, high speed data streams demanding both real-time and long-term data analytics. Fog computing is a reliable solution for processing and analyzing real-time streaming data as it offers low latency, location-aware, geographically distributed service at fog node and provides long-term services at the cloud data center (DC). This chapter addresses the challenge of coordinating the fog nodes and cloud for efficient processing of real-time streaming data in motion and at rest. The fog-cloud collaboration framework proposed in this chapter employs data stream management system (DSMS) schema at the fog node for real-time stream data processing and response generation. The data representation in micro-clusters at fog node and macro-clusters at DC facilitates accurate data analytics. The coordination between fog node and DC is through local ontology and global ontology respectively.


2020 ◽  
Vol 8 (6) ◽  
pp. 4474-4477

In the world of technology, people prefer social media to express themselves. Record says Twitter has more than 321 million active users with 100 million users posting approximately 340 million tweets a day. Twitter is the largest source of breaking news on social issues specially election-related where people can express their views also suggest their opinion. Twitter is generating unlimited unstructured text data. Hadoop is one of the finest tools accessible for analyzing twitter data because it supports processing of distributed big data, streaming data, time stamped data, text data etc. Whereas Apache Flume is used to extract real time twitter data into HDFS. This study attempts to establish an analytical framework to derive and interpret structured as well as unstructured Twitter data. The proposed framework comprises of real time twitter data insertion, its processing, and data visualization utilizing Apache Flume and pig. In this project we fetch positive and negative tweets on election data from twitter and analyzing the party status and the probability to win the election.


2019 ◽  
Vol 23 (1) ◽  
pp. 346-357
Author(s):  
Vithya G ◽  
Naren J ◽  
Varun V

Algorithms ◽  
2019 ◽  
Vol 12 (2) ◽  
pp. 37 ◽  
Author(s):  
Zhigang Hu ◽  
Hui Kang ◽  
Meiguang Zheng

A distributed data stream processing system handles real-time, changeable and sudden streaming data load. Its elastic resource allocation has become a fundamental and challenging problem with a fixed strategy that will result in waste of resources or a reduction in QoS (quality of service). Spark Streaming as an emerging system has been developed to process real time stream data analytics by using micro-batch approach. In this paper, first, we propose an improved SVR (support vector regression) based stream data load prediction scheme. Then, we design a spark-based maximum sustainable throughput of time window (MSTW) performance model to find the optimized number of virtual machines. Finally, we present a resource scaling algorithm TWRES (time window resource elasticity scaling algorithm) with MSTW constraint and streaming data load prediction. The evaluation results show that TWRES could improve resource utilization and mitigate SLA (service level agreement) violation.


Author(s):  
Huijun Wu ◽  
Xiaoyao Qian ◽  
Aleks Shulman ◽  
Kanishk Karanawat ◽  
Tushar Singh ◽  
...  

Sign in / Sign up

Export Citation Format

Share Document