Combining data streams of doubtful provenance

2022 ◽  
pp. 285-305
Author(s):  
Kavya Jagan ◽  
Alistair B. Forbes
Keyword(s):  
2016 ◽  
Author(s):  
Natasha MacBean ◽  
Philippe Peylin ◽  
Frédéric Chevallier ◽  
Marko Scholze ◽  
Gregor Schürmann

Abstract. Data assimilation methods provide a rigorous statistical framework for constraining the parametric uncertainty of land surface models (LSMs), with the aim of improving our predictive capability as well as identifying areas in which the models need improvement. The increase in the number of available datasets in recent years allows us to address different aspects of the model at a variety of spatial and temporal scales. However, combining data streams in a DA system is not a trivial task. In this study we highlight some of the challenges surrounding multiple data stream assimilation, with a particular focus on the carbon cycle component of LSMs. We examine the impact of biases and inconsistencies between the observations and the model (resulting in non Gaussian error distributions) and the impact of non-linearity in model dynamics. In addition we explore the differences between performing a simultaneous assimilation (in which all data streams are included in one optimisation) and a step-wise approach (in which each data steam is assimilated sequentially), given the assumptions inherent to the inversion algorithm chosen for this study. We demonstrate some of these issues by assimilating synthetic observations into two simple models: the first a simplified version of the carbon cycle processes represented in many LSMs, and the second a non-linear toy model. We further discuss these experimental results in the context of recent studies in the carbon cycle data assimilation literature, and finally we provide some perspectives and advice to other land surface modellers wishing to use multiple data streams to constrain their models.


2013 ◽  
Vol 134 (5) ◽  
pp. 4007-4007
Author(s):  
Stacy L. DeRuiter ◽  
Catriona Harris ◽  
Dina Sadykova ◽  
Len Thomas

Author(s):  
LAKSHMI PRANEETHA

Now-a-days data streams or information streams are gigantic and quick changing. The usage of information streams can fluctuate from basic logical, scientific applications to vital business and money related ones. The useful information is abstracted from the stream and represented in the form of micro-clusters in the online phase. In offline phase micro-clusters are merged to form the macro clusters. DBSTREAM technique captures the density between micro-clusters by means of a shared density graph in the online phase. The density data in this graph is then used in reclustering for improving the formation of clusters but DBSTREAM takes more time in handling the corrupted data points In this paper an early pruning algorithm is used before pre-processing of information and a bloom filter is used for recognizing the corrupted information. Our experiments on real time datasets shows that using this approach improves the efficiency of macro-clusters by 90% and increases the generation of more number of micro-clusters within in a short time.


2012 ◽  
Vol 35 (3) ◽  
pp. 540-554 ◽  
Author(s):  
Shang-Lian PENG ◽  
Zhan-Huai LI ◽  
Qun CHEN ◽  
Qiang LI

Sign in / Sign up

Export Citation Format

Share Document