scholarly journals Analysis of Data Stream Processing At Edge Layer for Internet of Things

2020 ◽  
Vol 2 (1) ◽  
pp. 26-37
Author(s):  
Dr. Pasumponpandian
Keyword(s):  
Internet Of Things ◽  
Data Streams ◽  
Data Stream ◽  
Smart Cities ◽  
Stream Processing ◽  
Middle Layer ◽  
Cloud Services ◽  
Decentralized Systems ◽  
Data Stream Processing ◽  
Edge Layer

The progress of internet of things at a rapid pace and simultaneous development of the technologies and the processing capabilities has paved way for the development of decentralized systems that are relying on cloud services. Though the decentralized systems are founded on cloud complexities still prevail in transferring all the information’s that are been sensed through the IOT devices to the cloud. This because of the huge streams of information’s gathered by certain applications and the expectation to have a timely response, incurring minimized delay, computing energy and enhanced reliability. So this kind of decentralization has led to the development of middle layer between the cloud and the IOT, and was termed as the Edge layer, meaning bringing down the service of the cloud to the user edge. The paper puts forth the analysis of the data stream processing in the edge layer taking in the complexities involved in the computing the data streams of IOT in an edge layer and puts forth the real time analytics in the edge layer to examine the data streams of the internet of things offering a data- driven insight for parking system in the smart cities.

scholarly journals QoS-Aware Approximate Query Processing for Smart Cities Spatial Data Streams

Sensors ◽  
2021 ◽  
Vol 21 (12) ◽  
pp. 4160
Author(s):  
Isam Mashhour Al Jawarneh ◽  
Paolo Bellavista ◽  
Antonio Corradi ◽  
Luca Foschini ◽  
Rebecca Montanari
Keyword(s):  
Data Streams ◽  
Spatial Data ◽  
Data Stream ◽  
Smart Cities ◽  
Stream Processing ◽  
Processing System ◽  
Strategic Decision ◽  
Approximate Query Processing ◽  
Data Stream Processing ◽  
Mobility Data

Large amounts of georeferenced data streams arrive daily to stream processing systems. This is attributable to the overabundance of affordable IoT devices. In addition, interested practitioners desire to exploit Internet of Things (IoT) data streams for strategic decision-making purposes. However, mobility data are highly skewed and their arrival rates fluctuate. This nature poses an extra challenge on data stream processing systems, which are required in order to achieve pre-specified latency and accuracy goals. In this paper, we propose ApproxSSPS, which is a system for approximate processing of geo-referenced mobility data, at scale with quality of service guarantees. We focus on stateful aggregations (e.g., means, counts) and top-N queries. ApproxSSPS features a controller that interactively learns the latency statistics and calculates proper sampling rates to meet latency or/and accuracy targets. An overarching trait of ApproxSSPS is its ability to strike a plausible balance between latency and accuracy targets. We evaluate ApproxSSPS on Apache Spark Structured Streaming with real mobility data. We also compared ApproxSSPS against a state-of-the-art online adaptive processing system. Our extensive experiments prove that ApproxSSPS can fulfill latency and accuracy targets with varying sets of parameter configurations and load intensities (i.e., transient peaks in data loads versus slow arriving streams). Moreover, our results show that ApproxSSPS outperforms the baseline counterpart by significant magnitudes. In short, ApproxSSPS is a novel spatial data stream processing system that can deliver real accurate results in a timely manner, by dynamically specifying the limits on data samples.

scholarly journals A Survey of Challenges Facing Streaming Data

2020 ◽  
Vol 8 (4) ◽  
pp. 63-73
Author(s):  
Sikha Bagui ◽  
Katie Jin
Keyword(s):  
Data Reduction ◽  
Data Streams ◽  
Data Stream ◽  
Stream Processing ◽  
Streaming Data ◽  
Data Detection ◽  
Data Stream Processing ◽  
The Face ◽  
Concept Drifts

This survey performs a thorough enumeration and analysis of existing methods for data stream processing. It is a survey of the challenges facing streaming data. The challenges addressed are preprocessing of streaming data, detection and dealing with concept drifts in streaming data, data reduction in the face of data streams, approximate queries and blocking operations in streaming data.

A context-driven platform using Internet of things and data stream processing for heating, ventilation and air conditioning systems control

2019 ◽  
Vol 233 (7) ◽  
pp. 877-888
Author(s):  
Fadwa Lachhab ◽  
Mohamed Bakhouya ◽  
Radouane Ouladsine ◽  
Mohammed Essaaidi
Keyword(s):  
Feedback Control ◽  
Internet Of Things ◽  
Air Conditioning ◽  
Data Stream ◽  
State Feedback ◽  
Stream Processing ◽  
State Feedback Control ◽  
Data Stream Processing ◽  
Air Conditioning System ◽  
Air Conditioning Systems

Control approaches of heating, ventilation and air conditioning systems in buildings have been proposed in the past years for minimizing energy consumption and maintaining occupants’ comfort. However, recent studies have shown that context-driven control approaches using Internet of things and data stream processing technologies could further improve energy saving in heating, ventilation and air conditioning systems. In this article, an intelligent control approach using a state feedback technique is introduced to regulate the heating, ventilation and air conditioning system according to the actual context. The proposed thermal state feedback control was then implemented and deployed in our EEBLab to study its effectiveness in a real-setting scenario. The performance of the proposed control was evaluated in a real test-site by deploying a control card that links the controller with the heating, ventilation and air conditioning system. A smart mobile application for real feedback control was also developed and deployed to dynamically adapt the controller to context’s changes. The mobile application and the heating, ventilation and air conditioning system communicate and exchange data under a data acquisition and visualization platform. In this article, a holistic platform that combines Internet of things and data stream processing technologies was developed and deployed in a real-setting scenario. Experiments have been performed, and results are reported to demonstrate the effectiveness and usefulness of the proposed approach in terms of energy saving while maintaining a comfortable room temperature. The proposed state feedback control outperforms the proportional–integral–derivative and ON/OFF approaches in terms of energy consumption while providing acceptable thermal comfort by allowing a neutral thermal sensation with ± 0.30 of predictive mean vote and less than 7% of predicted percentage of dissatisfaction.

Design and Evaluation of an Autonomous Load Balancing System for Mobile Data Stream Processing Based On a Data Centric Publish Subscribe Approach

2014 ◽  
Vol 5 (3) ◽  
pp. 1-19 ◽  
Author(s):  
Rafael Oliveira Vasconcelos ◽  
Markus Endler ◽  
Berto de Tácio Pereira Gomes ◽  
Francisco José da Silva e Silva
Keyword(s):  
Load Balancing ◽  
Data Streams ◽  
Intelligent Transportation Systems ◽  
Data Stream ◽  
Stream Processing ◽  
Industrial Process ◽  
Transportation Systems ◽  
Mobile Nodes ◽  
Data Stream Processing ◽  
Mobile Data

Several new applications of mobile computing environments, such as Intelligent Transportation Systems, Fleet Management and Logistics, and integrated Industrial Process Automation share the requirement of remote monitoring and high performance processing of huge data streams produced by large sets of mobile nodes. Two key requirements for the deployment and operation of such mobile infrastructures are the handling of large and variable numbers of wireless connections to the monitored mobile nodes regardless of their current use or locations, and to automatically adapt to variations in the volume of the mobile data streams. This article describes the design, implementation, and evaluation of an autonomic mechanism for load balancing of mobile data streams. The autonomic capability has been incorporated into a scalable middleware system based on a Data Centric Publish Subscribe approach using the OMG Data Distribution Service (DDS) standard and aimed at real-time and adaptive handling of mobile connectivity and data stream processing for great sets of mobile nodes. A significant amount of evaluation experiments of the proposed infrastructure is presented, reinforcing its viability and the benefits arising from the use of an autonomic approach to handle the requirements of high variability and scalability.

scholarly journals GEOSPATIAL DATA STREAM PROCESSING IN PYTHON USING FOSS4G COMPONENTS

2016 ◽  
Vol XLI-B7 ◽  
pp. 931-937
Author(s):  
G. McFerren ◽  
T. van Zyl
Keyword(s):  
Data Streams ◽  
Data Model ◽  
Data Stream ◽  
Stream Processing ◽  
Geospatial Data ◽  
Common Data Model ◽  
Spatial Relationships ◽  
Software Components ◽  
Data Stream Processing ◽  
Software Libraries

One viewpoint of current and future IT systems holds that there is an increase in the scale and velocity at which data are acquired and analysed from heterogeneous, dynamic sources. In the earth observation and geoinformatics domains, this process is driven by the increase in number and types of devices that report location and the proliferation of assorted sensors, from satellite constellations to oceanic buoy arrays. Much of these data will be encountered as self-contained messages on data streams - continuous, infinite flows of data. Spatial analytics over data streams concerns the search for spatial and spatio-temporal relationships within and amongst data “on the move”. In spatial databases, queries can assess a store of data to unpack spatial relationships; this is not the case on streams, where spatial relationships need to be established with the incomplete data available. Methods for spatially-based indexing, filtering, joining and transforming of streaming data need to be established and implemented in software components. This article describes the usage patterns and performance metrics of a number of well known FOSS4G Python software libraries within the data stream processing paradigm. In particular, we consider the RTree library for spatial indexing, the Shapely library for geometric processing and transformation and the PyProj library for projection and geodesic calculations over streams of geospatial data. We introduce a message oriented Python-based geospatial data streaming framework called Swordfish, which provides data stream processing primitives, functions, transports and a common data model for describing messages, based on the Open Geospatial Consortium Observations and Measurements (O&M) and Unidata Common Data Model (CDM) standards. We illustrate how the geospatial software components are integrated with the Swordfish framework. Furthermore, we describe the tight temporal constraints under which geospatial functionality can be invoked when processing high velocity, potentially infinite geospatial data streams. The article discusses the performance of these libraries under simulated streaming loads (size, complexity and volume of messages) and how they can be deployed and utilised with Swordfish under real load scenarios, illustrated by a set of Vessel Automatic Identification System (AIS) use cases. We conclude that the described software libraries are able to perform adequately under geospatial data stream processing scenarios - many real application use cases will be handled sufficiently by the software.

RFID data stream processing technology based on CEP

2009 ◽  
Vol 29 (10) ◽  
pp. 2786-2790 ◽  
Author(s):  
Xiao-jia YIN ◽  
Shi-guang JU ◽  
Ying-jie WANG
Keyword(s):  
Data Stream ◽  
Stream Processing ◽  
Processing Technology ◽  
Data Stream Processing ◽  
Rfid Data
Sign in / Sign up

Export Citation Format

Share Document