Combining data streams of doubtful provenance

2022 ◽  
pp. 285-305
Author(s):  
Kavya Jagan ◽  
Alistair B. Forbes
Keyword(s):  
Data Streams ◽  
Combining Data

scholarly journals Consistent assimilation of multiple data streams in a carbon cycle data assimilation system

2016 ◽  
Author(s):  
Natasha MacBean ◽  
Philippe Peylin ◽  
Frédéric Chevallier ◽  
Marko Scholze ◽  
Gregor Schürmann
Keyword(s):  
Data Assimilation ◽  
Carbon Cycle ◽  
Data Streams ◽  
Land Surface ◽  
Inversion Algorithm ◽  
Spatial And Temporal Scales ◽  
Multiple Data ◽  
Combining Data ◽  
Multiple Data Streams ◽  
The Impact

Abstract. Data assimilation methods provide a rigorous statistical framework for constraining the parametric uncertainty of land surface models (LSMs), with the aim of improving our predictive capability as well as identifying areas in which the models need improvement. The increase in the number of available datasets in recent years allows us to address different aspects of the model at a variety of spatial and temporal scales. However, combining data streams in a DA system is not a trivial task. In this study we highlight some of the challenges surrounding multiple data stream assimilation, with a particular focus on the carbon cycle component of LSMs. We examine the impact of biases and inconsistencies between the observations and the model (resulting in non Gaussian error distributions) and the impact of non-linearity in model dynamics. In addition we explore the differences between performing a simultaneous assimilation (in which all data streams are included in one optimisation) and a step-wise approach (in which each data steam is assimilated sequentially), given the assumptions inherent to the inversion algorithm chosen for this study. We demonstrate some of these issues by assimilating synthetic observations into two simple models: the first a simplified version of the carbon cycle processes represented in many LSMs, and the second a non-linear toy model. We further discuss these experimental results in the context of recent studies in the carbon cycle data assimilation literature, and finally we provide some perspectives and advice to other land surface modellers wishing to use multiple data streams to constrain their models.

Statistical analysis of data from acoustic tags: Methods for combining data streams and modeling animal behavior

2013 ◽  
Vol 134 (5) ◽  
pp. 4007-4007
Author(s):  
Stacy L. DeRuiter ◽  
Catriona Harris ◽  
Dina Sadykova ◽  
Len Thomas
Keyword(s):  
Statistical Analysis ◽  
Animal Behavior ◽  
Data Streams ◽  
Combining Data ◽  
Acoustic Tags

Improved Macro-clusters generation using Top-k shared Micro-clusters in Data Streams

2017 ◽  
Vol 7 (10) ◽  
pp. 52
Author(s):  
LAKSHMI PRANEETHA
Keyword(s):  
Real Time ◽  
Data Streams ◽  
Bloom Filter ◽  
Scientific Applications ◽  
Pruning Algorithm ◽  
Density Data ◽  
Data Points ◽  
Short Time ◽  
Information Streams

Now-a-days data streams or information streams are gigantic and quick changing. The usage of information streams can fluctuate from basic logical, scientific applications to vital business and money related ones. The useful information is abstracted from the stream and represented in the form of micro-clusters in the online phase. In offline phase micro-clusters are merged to form the macro clusters. DBSTREAM technique captures the density between micro-clusters by means of a shared density graph in the online phase. The density data in this graph is then used in reclustering for improving the formation of clusters but DBSTREAM takes more time in handling the corrupted data points In this paper an early pruning algorithm is used before pre-processing of information and a bloom filter is used for recognizing the corrupted information. Our experiments on real time datasets shows that using this approach improves the efficiency of macro-clusters by 90% and increases the generation of more number of micro-clusters within in a short time.

Complex Event Processing over LiveArchived Data Streams

2012 ◽  
Vol 35 (3) ◽  
pp. 540-554 ◽  
Author(s):  
Shang-Lian PENG ◽  
Zhan-Huai LI ◽  
Qun CHEN ◽  
Qiang LI
Keyword(s):  
Data Streams ◽  
Complex Event Processing ◽  
Event Processing
Sign in / Sign up

Export Citation Format

Share Document