scholarly journals REAL-TIME EARTHQUAKE MONITORING WITH SPATIO-TEMPORAL FIELDS

2017 ◽  
Vol IV-4/W2 ◽  
pp. 215-222 ◽  
Author(s):  
J. C. Whittier ◽  
S. Nittel ◽  
I. Subasinghe
Keyword(s):  
Real Time ◽  
Data Streams ◽  
Data Stream ◽  
Apache Spark ◽  
Sensor Data ◽  
Live Streaming ◽  
Earthquake Activity ◽  
Maximum Displacement ◽  
Earthquake Event ◽  
Spatio Temporal

With live streaming sensors and sensor networks, increasingly large numbers of individual sensors are deployed in physical space. Sensor data streams are a fundamentally novel mechanism to deliver observations to information systems. They enable us to represent spatio-temporal continuous phenomena such as radiation accidents, toxic plumes, or earthquakes almost as instantaneously as they happen in the real world. Sensor data streams discretely sample an earthquake, while the earthquake is continuous over space and time. Programmers attempting to integrate many streams to analyze earthquake activity and scope need to write code to integrate potentially very large sets of asynchronously sampled, concurrent streams in tedious application code. In previous work, we proposed the field stream data model (Liang et al., 2016) for data stream engines. Abstracting the stream of an individual sensor as a temporal field, the field represents the Earth’s movement at the sensor position as continuous. This simplifies analysis across many sensors significantly. In this paper, we undertake a feasibility study of using the field stream model and the open source Data Stream Engine (DSE) Apache Spark(Apache Spark, 2017) to implement a real-time earthquake event detection with a subset of the 250 GPS sensor data streams of the Southern California Integrated GPS Network (SCIGN). The field-based real-time stream queries compute maximum displacement values over the latest query window of each stream, and related spatially neighboring streams to identify earthquake events and their extent. Further, we correlated the detected events with an USGS earthquake event feed. The query results are visualized in real-time.

Knowledge Discovery From Evolving Data Streams

2019 ◽  
pp. 19-39
Author(s):  
Prasanna Lakshmi Kompalli
Keyword(s):  
Real Time ◽  
Data Streams ◽  
Data Stream ◽  
Concept Drift ◽  
Data Stream Mining ◽  
Time Data ◽  
Stream Mining ◽  
New Challenges ◽  
Mining Data Streams ◽  
Different Sources

Data coming from different sources is referred to as data streams. Data stream mining is an online learning technique where each data point must be processed as the data arrives and discarded as the processing is completed. Progress of technologies has resulted in the monitoring these data streams in real time. Data streams has created many new challenges to the researchers in real time. The main features of this type of data are they are fast flowing, large amounts of data which are continuous and growing in nature, and characteristics of data might change in course of time which is termed as concept drift. This chapter addresses the problems in mining data streams with concept drift. Due to which, isolating the correct literature would be a grueling task for researchers and practitioners. This chapter tries to provide a solution as it would be an amalgamation of all techniques used for data stream mining with concept drift.

Research on Sequence Query Processing Techniques over Data Streams

2013 ◽  
Vol 284-287 ◽  
pp. 3507-3511 ◽  
Author(s):  
Edgar Chia Han Lin
Keyword(s):  
Query Processing ◽  
Data Streams ◽  
Management System ◽  
Data Stream ◽  
Sensor Data ◽  
Data Management System ◽  
Content Filtering ◽  
Data Stream Management ◽  
Stream Management ◽  
Great Progress

Due to the great progress of computer technology and mature development of network, more and more data are generated and distributed through the network, which is called data streams. During the last couple of years, a number of researchers have paid their attention to data stream management, which is different from the conventional database management. At present, the new type of data management system, called data stream management system (DSMS), has become one of the most popular research areas in data engineering field. Lots of research projects have made great progress in this area. Since the current DSMS does not support queries on sequence data, this project will study the issues related to two types of data. First, we will focus on the content filtering on single-attribute streams, such as sensor data. Second, we will focus on multi-attribute streams, such as video films. We will discuss the related issues such as how to build an efficient index for all queries of different streams and the corresponding query processing mechanisms.

PLACE1A Query Processor for Handling Real-time Spatio-temporal Data Streams

2004 ◽  
pp. 1377-1380 ◽  
Author(s):  
M MOKBEL ◽  
X XIONG ◽  
W AREF ◽  
S HAMBRUSCH ◽  
S PRABHAKAR ◽  
...  
Keyword(s):  
Real Time ◽  
Data Streams ◽  
Temporal Data ◽  
Spatio Temporal

scholarly journals MCDAStream: a real-time data stream clustering based on micro-cluster density and attraction

2018 ◽  
Vol 7 (2) ◽  
pp. 270 ◽  
Author(s):  
Shyam Sunder Reddy K ◽  
Shoba Bindu C
Keyword(s):  
Real Time ◽  
Data Streams ◽  
Data Stream ◽  
Clustering Algorithm ◽  
Positional Information ◽  
Time Data ◽  
Stream Clustering ◽  
Real Time Data ◽  
Cluster Density ◽  
Data Stream Clustering

Real-time data stream clustering has been widely used in many fields, and it can extract useful information from massive sets of data. Most of the existing density-based algorithms cluster the data streams based on the density within the micro-clusters. These algorithms completely omit the data density in the area between the micro-clusters and recluster the micro-clusters based on erroneous assumptions about the distribution of the data within and between the micro-clusters that lead to poor clustering results. This paper describes a novel density-based clustering algorithm for evolving data streams called MCDAStream, which clusters the data stream based on micro-cluster density and attraction between the micro-clusters. The attraction of micro-clusters characterizes the positional information of the data points in each micro-cluster. We generate better clustering results by considering both micro-cluster density and attraction of micro-clusters. The quality of the proposed algorithm is evaluated on various synthetic and real-time datasets with distinct characteristics and quality metrics.

PROBABILISTIC QUERYING OVER UNCERTAIN DATA STREAMS

2012 ◽  
Vol 20 (05) ◽  
pp. 701-728 ◽  
Author(s):  
MOHAMMAD G. DEZFULI ◽  
MOSTAFA S. HAGHJOO
Keyword(s):  
Data Streams ◽  
Data Stream ◽  
Memory Management ◽  
Query Language ◽  
Uncertain Data ◽  
Sensor Data ◽  
Data Stream Management ◽  
Probabilistic Data ◽  
Stream Management ◽  
Probabilistic Data Streams

Inherent imprecision of data in many applications motivates us to support uncertainty as a first-class concept. Data stream and probabilistic data have been recently considered noticeably in isolation. However, there are many applications including sensor data management systems and object monitoring systems which need both issues in tandem. Our main contribution is designing a probabilistic data stream management system, called Sarcheshmeh, for continuous querying over probabilistic data streams. Sarcheshmeh supports uncertainty from input data to final query results. In this paper, after reviewing requirements and applications of probabilistic data streams, we present our new data model for probabilistic data streams and define our main logical operators formally. Then, we present query language and physical operators. In addition, we introduce the architecture of Sarcheshmeh and also describe some major challenges like memory management and our floating precision mechanism toward designing a more robust system. Finally, we report evaluation of our system and the effect of floating precision on the tradeoff between accuracy and efficiency.

scholarly journals CHANGE SEMANTIC CONSTRAINED ONLINE DATA CLEANING METHOD FOR REAL-TIME OBSERVATIONAL DATA STREAM

2016 ◽  
Vol XLI-B2 ◽  
pp. 177-183
Author(s):  
Yulin Ding ◽  
Hui Lin ◽  
Rongrong Li
Keyword(s):  
Real Time ◽  
Observational Data ◽  
Data Streams ◽  
Data Stream ◽  
Data Cleaning ◽  
Data Distribution ◽  
Online Data ◽  
Filter Parameter ◽  
Changing Patterns ◽  
Cleaning Methods

Recent breakthroughs in sensor networks have made it possible to collect and assemble increasing amounts of real-time observational data by observing dynamic phenomena at previously impossible time and space scales. Real-time observational data streams present potentially profound opportunities for real-time applications in disaster mitigation and emergency response, by providing accurate and timeliness estimates of environment’s status. However, the data are always subject to inevitable anomalies (including errors and anomalous changes/events) caused by various effects produced by the environment they are monitoring. The “big but dirty” real-time observational data streams can rarely achieve their full potential in the following real-time models or applications due to the low data quality. Therefore, timely and meaningful online data cleaning is a necessary pre-requisite step to ensure the quality, reliability, and timeliness of the real-time observational data. <br><br> In general, a straightforward streaming data cleaning approach, is to define various types of models/classifiers representing normal behavior of sensor data streams and then declare any deviation from this model as normal or erroneous data. The effectiveness of these models is affected by dynamic changes of deployed environments. Due to the changing nature of the complicated process being observed, real-time observational data is characterized by diversity and dynamic, showing a typical Big (Geo) Data characters. Dynamics and diversity is not only reflected in the data values, but also reflected in the complicated changing patterns of the data distributions. This means the pattern of the real-time observational data distribution is not <i>stationary or static</i> but <i>changing and dynamic</i>. After the data pattern changed, it is necessary to adapt the model over time to cope with the changing patterns of real-time data streams. Otherwise, the model will not fit the following observational data streams, which may led to large estimation error. In order to achieve the best generalization error, it is an important challenge for the data cleaning methodology to be able to characterize the behavior of data stream distributions and adaptively update a model to include new information and remove old information. However, the complicated data changing property invalidates traditional data cleaning methods, which rely on the assumption of a stationary data distribution, and drives the need for more dynamic and adaptive online data cleaning methods. <br><br> To overcome these shortcomings, this paper presents a change semantics constrained online filtering method for real-time observational data. Based on the principle that the filter parameter should vary in accordance to the data change patterns, this paper embeds semantic description, which quantitatively depicts the change patterns in the data distribution to self-adapt the filter parameter automatically. Real-time observational water level data streams of different precipitation scenarios are selected for testing. Experimental results prove that by means of this method, more accurate and reliable water level information can be available, which is prior to scientific and prompt flood assessment and decision-making.

scholarly journals INCREMENTAL PRINCIPAL COMPONENT ANALYSIS BASED OUTLIER DETECTION METHODS FOR SPATIOTEMPORAL DATA STREAMS

2015 ◽  
Vol II-4/W2 ◽  
pp. 67-71 ◽  
Author(s):  
A. Bhushan ◽  
M. H. Sharker ◽  
H. A. Karimi
Keyword(s):  
Principal Component Analysis ◽  
Real Time ◽  
Outlier Detection ◽  
Data Streams ◽  
Credit Card ◽  
Principal Component ◽  
Component Analysis ◽  
Spatiotemporal Data ◽  
Sensor Data ◽  
Detection Methods

In this paper, we address outliers in spatiotemporal data streams obtained from sensors placed across geographically distributed locations. Outliers may appear in such sensor data due to various reasons such as instrumental error and environmental change. Real-time detection of these outliers is essential to prevent propagation of errors in subsequent analyses and results. Incremental Principal Component Analysis (IPCA) is one possible approach for detecting outliers in such type of spatiotemporal data streams. IPCA has been widely used in many real-time applications such as credit card fraud detection, pattern recognition, and image analysis. However, the suitability of applying IPCA for outlier detection in spatiotemporal data streams is unknown and needs to be investigated. To fill this research gap, this paper contributes by presenting two new IPCA-based outlier detection methods and performing a comparative analysis with the existing IPCA-based outlier detection methods to assess their suitability for spatiotemporal sensor data streams.

Sign in / Sign up

Export Citation Format

Share Document