Adaptive Segmentation of Streaming Sensor Data on Edge Devices

Sensor data streams often represent signals/trajectories which are twice differentiable (e.g., to give a continuous velocity and acceleration), and this property must be reflected in their segmentation. An adaptive streaming algorithm for this problem is presented. It is based on the greedy look-ahead strategy and is built on the concept of a cubic splinelet. A characteristic feature of the proposed algorithm is the real-time simultaneous segmentation, smoothing, and compression of data streams. The segmentation quality is measured in terms of the signal approximation accuracy and the corresponding compression ratio. The numerical results show the relatively high compression ratios (from 135 to 208, i.e., compressed stream sizes up to 208 times smaller) combined with the approximation errors comparable to those obtained from the state-of-the-art global reference algorithm. The proposed algorithm can be applied to various domains, including online compression and/or smoothing of data streams coming from sensors, real-time IoT analytics, and embedded time-series databases.

Download Full-text

Real-time spatial interpolation of continuous phenomena using mobile sensor data streams

Proceedings of the 20th International Conference on Advances in Geographic Information Systems - SIGSPATIAL '12 ◽

10.1145/2424321.2424407 ◽

2012 ◽

Cited By ~ 8

Author(s):

Silvia Nittel ◽

J. C. Whittier ◽

Qinghan Liang

Keyword(s):

Real Time ◽

Data Streams ◽

Spatial Interpolation ◽

Sensor Data ◽

Mobile Sensor

Download Full-text

A Tour of Lattice-Based Skyline Algorithms

Handbook of Research on Investigations in Artificial Life Research and Development - Advances in Computational Intelligence and Robotics ◽

10.4018/978-1-5225-5396-0.ch006 ◽

2018 ◽

pp. 96-122

Author(s):

Markus Endres ◽

Lena Rudenko

Keyword(s):

Real Time ◽

Data Streams ◽

High Performance ◽

State Of The Art ◽

Experimental Results ◽

Lattice Structures ◽

Skyline Query ◽

Basic Concepts ◽

Generic Index

A skyline query retrieves all objects in a dataset that are not dominated by other objects according to some given criteria. There exist many skyline algorithms which can be classified into generic, index-based, and lattice-based algorithms. This chapter takes a tour through lattice-based skyline algorithms. It summarizes the basic concepts and properties, presents high-performance parallel approaches, shows how one overcomes the low-cardinality restriction of lattice structures, and finally presents an application on data streams for real-time skyline computation. Experimental results on synthetic and real datasets show that lattice-based algorithms outperform state-of-the-art skyline techniques, and additionally have a linear runtime complexity.

Download Full-text

REAL-TIME EARTHQUAKE MONITORING WITH SPATIO-TEMPORAL FIELDS

ISPRS Annals of Photogrammetry Remote Sensing and Spatial Information Sciences ◽

10.5194/isprs-annals-iv-4-w2-215-2017 ◽

2017 ◽

Vol IV-4/W2 ◽

pp. 215-222 ◽

Cited By ~ 1

Author(s):

J. C. Whittier ◽

S. Nittel ◽

I. Subasinghe

Keyword(s):

Real Time ◽

Data Streams ◽

Data Stream ◽

Apache Spark ◽

Sensor Data ◽

Live Streaming ◽

Earthquake Activity ◽

Maximum Displacement ◽

Earthquake Event ◽

Spatio Temporal

With live streaming sensors and sensor networks, increasingly large numbers of individual sensors are deployed in physical space. Sensor data streams are a fundamentally novel mechanism to deliver observations to information systems. They enable us to represent spatio-temporal continuous phenomena such as radiation accidents, toxic plumes, or earthquakes almost as instantaneously as they happen in the real world. Sensor data streams discretely sample an earthquake, while the earthquake is continuous over space and time. Programmers attempting to integrate many streams to analyze earthquake activity and scope need to write code to integrate potentially very large sets of asynchronously sampled, concurrent streams in tedious application code. In previous work, we proposed the field stream data model (Liang et al., 2016) for data stream engines. Abstracting the stream of an individual sensor as a temporal field, the field represents the Earth’s movement at the sensor position as continuous. This simplifies analysis across many sensors significantly. In this paper, we undertake a feasibility study of using the field stream model and the open source Data Stream Engine (DSE) Apache Spark(Apache Spark, 2017) to implement a real-time earthquake event detection with a subset of the 250 GPS sensor data streams of the Southern California Integrated GPS Network (SCIGN). The field-based real-time stream queries compute maximum displacement values over the latest query window of each stream, and related spatially neighboring streams to identify earthquake events and their extent. Further, we correlated the detected events with an USGS earthquake event feed. The query results are visualized in real-time.

Download Full-text

Real-Time Monitoring of Mobile Biological Sensor Data-Streams: Architecture and Cost-Model

2008 Ninth International Conference on Mobile Data Management Workshops, MDMW ◽

10.1109/mdmw.2008.22 ◽

2008 ◽

Cited By ~ 1

Author(s):

Alfredo Goni ◽

Jimena Rodriguez ◽

Alfredo Burgos ◽

Arantza Illarramendi ◽

Lacramioara Dranca

Keyword(s):

Real Time ◽

Data Streams ◽

Cost Model ◽

Sensor Data ◽

Real Time Monitoring ◽

Biological Sensor

Download Full-text

INCREMENTAL PRINCIPAL COMPONENT ANALYSIS BASED OUTLIER DETECTION METHODS FOR SPATIOTEMPORAL DATA STREAMS

ISPRS Annals of Photogrammetry Remote Sensing and Spatial Information Sciences ◽

10.5194/isprsannals-ii-4-w2-67-2015 ◽

2015 ◽

Vol II-4/W2 ◽

pp. 67-71 ◽

Cited By ~ 2

Author(s):

A. Bhushan ◽

M. H. Sharker ◽

H. A. Karimi

Keyword(s):

Principal Component Analysis ◽

Real Time ◽

Outlier Detection ◽

Data Streams ◽

Credit Card ◽

Principal Component ◽

Component Analysis ◽

Spatiotemporal Data ◽

Sensor Data ◽

Detection Methods

In this paper, we address outliers in spatiotemporal data streams obtained from sensors placed across geographically distributed locations. Outliers may appear in such sensor data due to various reasons such as instrumental error and environmental change. Real-time detection of these outliers is essential to prevent propagation of errors in subsequent analyses and results. Incremental Principal Component Analysis (IPCA) is one possible approach for detecting outliers in such type of spatiotemporal data streams. IPCA has been widely used in many real-time applications such as credit card fraud detection, pattern recognition, and image analysis. However, the suitability of applying IPCA for outlier detection in spatiotemporal data streams is unknown and needs to be investigated. To fill this research gap, this paper contributes by presenting two new IPCA-based outlier detection methods and performing a comparative analysis with the existing IPCA-based outlier detection methods to assess their suitability for spatiotemporal sensor data streams.

Download Full-text

Generating Linked Data in Real-time from Sensor Data Streams

Materializing the Web of Linked Data ◽

10.1007/978-3-319-16074-0_5 ◽

2015 ◽

pp. 103-126

Author(s):

Nikolaos Konstantinou ◽

Dimitrios-Emmanuel Spanos

Keyword(s):

Real Time ◽

Data Streams ◽

Linked Data ◽

Sensor Data

Download Full-text

Seismic-while-drilling applications from the first DrillCAM trial with wireless geophones and instrumented top drive

The Leading Edge ◽

10.1190/tle39060422.1 ◽

2020 ◽

Vol 39 (6) ◽

pp. 422-429

Author(s):

Andrey Bakulin ◽

Ali Aldawood ◽

Ilya Silvestrov ◽

Emad Hemyari ◽

Flavio Poletto

Keyword(s):

Real Time ◽

Polycrystalline Diamond ◽

Drill String ◽

Sensor Data ◽

Continuous Data ◽

Near Surface ◽

Initial Application ◽

Look Ahead ◽

Drilling Operations ◽

Single Sensor

Advanced geophysical sensing while drilling is being driven by trends to automate and optimize drilling and the desire to better characterize complex near surface and overburden in desert environments. We introduce the DrillCAM system, which combines a set of geophysical techniques from seismic while drilling (SWD), drill-string vibration health, estimation of formation properties at the bit, and imaging ahead of and around the bit. We present data acquisition, processing, and initial application results from the first field trial on an onshore well in a desert environment. In this study, we focus on SWD applications. For the first time, wireless geophones installed around a rig were used to acquire continuous data while drilling. We demonstrate the feasibility of such a system to provide flexible acquisition geometries that are easily expandable with increasing bit depth without interference from drilling operations. Using a top-drive sensor as a pilot, we transform the drill-bit noise into meaningful and reliable seismic signals. The data were used to retrieve a check shot while drilling, make kinematic look-ahead predictions, and obtain a vertical seismic profiling corridor stack matching surface seismic. Robust near-offset check-shot signals were received from roller-cone and polycrystalline diamond compact (PDC) bits above 7200 ft after limited preprocessing of challenging single-sensor data with supergrouping. Detecting signals from deeper sections drilled with PDC bits may require more advanced processing by using an entire 2D spread of wireless geophones and downhole pilots. The real-time capabilities of the system make the data available for continuous data processing and interpretation that will facilitate drilling automation and improve real-time decision making.

Download Full-text