scholarly journals Ensemble Riemannian Data Assimilation over the Wasserstein Space

2021 ◽  
Author(s):  
Sagar K. Tamang ◽  
Ardeshir Ebtehaj ◽  
Peter J. van Leeuwen ◽  
Dongmian Zou ◽  
Gilad Lerman
Keyword(s):  
Data Assimilation ◽  
Evolutionary Dynamics ◽  
Probability Distributions ◽  
Random Errors ◽  
Wasserstein Metric ◽  
New Approach ◽  
Integrable Probability ◽  
Wasserstein Space ◽  
Non Gaussian ◽  
Background State

Abstract. In this paper, we present an ensemble data assimilation paradigm over a Riemannian manifold equipped with the Wasserstein metric. Unlike the Eulerian penalization of error in the Euclidean space, the Wasserstein metric can capture translation and difference between the shapes of square-integrable probability distributions of the background state and observations – enabling to formally penalize geophysical biases in state-space with non-Gaussian distributions. The new approach is applied to dissipative and chaotic evolutionary dynamics and its potential advantages and limitations are highlighted compared to the classic variational and filtering data assimilation approaches under systematic and random errors.

scholarly journals Ensemble Riemannian data assimilation over the Wasserstein space

2021 ◽  
Vol 28 (3) ◽  
pp. 295-309
Author(s):  
Sagar K. Tamang ◽  
Ardeshir Ebtehaj ◽  
Peter J. van Leeuwen ◽  
Dongmian Zou ◽  
Gilad Lerman
Keyword(s):  
Data Assimilation ◽  
Evolutionary Dynamics ◽  
Probability Distributions ◽  
Wasserstein Metric ◽  
New Approach ◽  
Ensemble Data Assimilation ◽  
Ensemble Data ◽  
Integrable Probability ◽  
Wasserstein Space ◽  
Non Gaussian

Abstract. In this paper, we present an ensemble data assimilation paradigm over a Riemannian manifold equipped with the Wasserstein metric. Unlike the Euclidean distance used in classic data assimilation methodologies, the Wasserstein metric can capture the translation and difference between the shapes of square-integrable probability distributions of the background state and observations. This enables us to formally penalize geophysical biases in state space with non-Gaussian distributions. The new approach is applied to dissipative and chaotic evolutionary dynamics, and its potential advantages and limitations are highlighted compared to the classic ensemble data assimilation approaches under systematic errors.

scholarly journals A Localized Particle Filter for High-Dimensional Nonlinear Systems

2015 ◽  
Vol 144 (1) ◽  
pp. 59-76 ◽  
Author(s):  
Jonathan Poterjoy
Keyword(s):  
Data Assimilation ◽  
Particle Filter ◽  
Remotely Sensed Data ◽  
Ensemble Kalman Filters ◽  
New Approach ◽  
Monte Carlo Approximation ◽  
Non Gaussian ◽  
Filter Divergence ◽  
Number Of Particles ◽  
Observation Networks

Abstract This paper presents a new data assimilation approach based on the particle filter (PF) that has potential for nonlinear/non-Gaussian applications in geoscience. Particle filters provide a Monte Carlo approximation of a system’s probability density, while making no assumptions regarding the underlying error distribution. The proposed method is similar to the PF in that particles—also referred to as ensemble members—are weighted based on the likelihood of observations in order to approximate posterior probabilities of the system state. The new approach, denoted the local PF, extends the particle weights into vector quantities to reduce the influence of distant observations on the weight calculations via a localization function. While the number of particles required for standard PFs scales exponentially with the dimension of the system, the local PF provides accurate results using relatively few particles. In sensitivity experiments performed with a 40-variable dynamical system, the local PF requires only five particles to prevent filter divergence for both dense and sparse observation networks. Comparisons of the local PF and ensemble Kalman filters (EnKFs) reveal advantages of the new method in situations resembling geophysical data assimilation applications. In particular, the new filter demonstrates substantial benefits over EnKFs when observation networks consist of densely spaced measurements that relate nonlinearly to the model state—analogous to remotely sensed data used frequently in weather analyses.

A New Approach for Estimating the Observation Impact in Ensemble–Variational Data Assimilation

2018 ◽  
Vol 146 (2) ◽  
pp. 447-465 ◽  
Author(s):  
Mark Buehner ◽  
Ping Du ◽  
Joël Bédard
Keyword(s):  
Data Assimilation ◽  
Forecast Error ◽  
Forecast Model ◽  
Variational Data Assimilation ◽  
Observation Error ◽  
New Approach ◽  
Regional Prediction ◽  
Observation Impact ◽  
Adjoint Approach ◽  
The Impact

Abstract Two types of approaches are commonly used for estimating the impact of arbitrary subsets of observations on short-range forecast error. The first was developed for variational data assimilation systems and requires the adjoint of the forecast model. Comparable approaches were developed for use with the ensemble Kalman filter and rely on ensembles of forecasts. In this study, a new approach for computing observation impact is proposed for ensemble–variational data assimilation (EnVar). Like standard adjoint approaches, the adjoint of the data assimilation procedure is implemented through the iterative minimization of a modified cost function. However, like ensemble approaches, the adjoint of the forecast step is obtained by using an ensemble of forecasts. Numerical experiments were performed to compare the new approach with the standard adjoint approach in the context of operational deterministic NWP. Generally similar results are obtained with both approaches, especially when the new approach uses covariance localization that is horizontally advected between analysis and forecast times. However, large differences in estimated impacts are obtained for some surface observations. Vertical propagation of the observation impact is noticeably restricted with the new approach because of vertical covariance localization. The new approach is used to evaluate changes in observation impact as a result of the use of interchannel observation error correlations for radiance observations. The estimated observation impact in similarly configured global and regional prediction systems is also compared. Overall, the new approach should provide useful estimates of observation impact for data assimilation systems based on EnVar when an adjoint model is not available.

Non-Gaussian probability distributions of solar wind fluctuations

1994 ◽  
Vol 12 (12) ◽  
pp. 1127-1138 ◽  
Author(s):  
E. Marsch ◽  
C. Y. Tu
Keyword(s):  
Solar Wind ◽  
Probability Distributions ◽  
Time Lag ◽  
Small Scale ◽  
Time Lags ◽  
Mhd Turbulence ◽  
Theoretical Concepts ◽  
Fluid Turbulence ◽  
Long Time ◽  
Non Gaussian

Abstract. The probability distributions of field differences ∆x(τ)=x(t+τ)-x(t), where the variable x(t) may denote any solar wind scalar field or vector field component at time t, have been calculated from time series of Helios data obtained in 1976 at heliocentric distances near 0.3 AU. It is found that for comparatively long time lag τ, ranging from a few hours to 1 day, the differences are normally distributed according to a Gaussian. For shorter time lags, of less than ten minutes, significant changes in shape are observed. The distributions are often spikier and narrower than the equivalent Gaussian distribution with the same standard deviation, and they are enhanced for large, reduced for intermediate and enhanced for very small values of ∆x. This result is in accordance with fluid observations and numerical simulations. Hence statistical properties are dominated at small scale τ by large fluctuation amplitudes that are sparsely distributed, which is direct evidence for spatial intermittency of the fluctuations. This is in agreement with results from earlier analyses of the structure functions of ∆x. The non-Gaussian features are differently developed for the various types of fluctuations. The relevance of these observations to the interpretation and understanding of the nature of solar wind magnetohydrodynamic (MHD) turbulence is pointed out, and contact is made with existing theoretical concepts of intermittency in fluid turbulence.

scholarly journals The Sum and Difference of Two Lognormal Random Variables

2012 ◽  
Vol 2012 ◽  
pp. 1-13 ◽  
Author(s):  
C. F. Lo
Keyword(s):  
Stochastic Process ◽  
Probability Distributions ◽  
Operator Splitting ◽  
Splitting Method ◽  
Unified Approach ◽  
New Approach ◽  
Numerical Examples ◽  
Operator Splitting Method ◽  
Approximate Probability ◽  
Analytical Series

We have presented a new unified approach to model the dynamics of both the sum and difference of two correlated lognormal stochastic variables. By the Lie-Trotter operator splitting method, both the sum and difference are shown to follow a shifted lognormal stochastic process, and approximate probability distributions are determined in closed form. Illustrative numerical examples are presented to demonstrate the validity and accuracy of these approximate distributions. In terms of the approximate probability distributions, we have also obtained an analytical series expansion of the exact solutions, which can allow us to improve the approximation in a systematic manner. Moreover, we believe that this new approach can be extended to study both (1) the algebraic sum ofNlognormals, and (2) the sum and difference of other correlated stochastic processes, for example, two correlated CEV processes, two correlated CIR processes, and two correlated lognormal processes with mean-reversion.

scholarly journals First steps towards the assimilation of IASI ozone data into the MOCAGE-PALM system

2009 ◽  
Vol 9 (2) ◽  
pp. 6691-6737 ◽  
Author(s):  
S. Massart ◽  
C. Clerbaux ◽  
D. Cariolle ◽  
A. Piacentini ◽  
S. Turquety ◽  
...  
Keyword(s):  
Data Assimilation ◽  
Total Ozone ◽  
Transport Model ◽  
Polar Regions ◽  
Observation Error ◽  
Random Errors ◽  
Chemistry Transport Model ◽  
Ozone Data ◽  
The One ◽  
New Generation

Abstract. The Infrared Atmospheric Sounding Interferometer (IASI) is one of the five European new generation instruments carried by the polar-orbiting MetOp-A satellite. Data assimilation is a powerful tool to combine these data with a numerical model. This paper presents the first steps made towards the assimilation of the total ozone columns from the IASI measurements into a chemistry transport model. The IASI ozone data used are provided by an inversion of radiances performed at the LATMOS (Laboratoire Atmosphères, Milieux, Observations Spatiales). As a contribution to the validation of this dataset, the LATMOS-IASI data are compared to a four dimensional ozone field, with low systematic and random errors compared to ozonesondes and OMI-DOAS data. This field results from the combined assimilation of ozone profiles from the MLS instrument and of total ozone columns from the SCIAMACHY instrument. It is found that on average, the LATMOS-IASI data tends to overestimate the total ozone columns by 2% to 8%. The random observation error of the LATMOS-IASI data is estimated to about 6%, except over polar regions and deserts where it is higher. Using this information, the LATMOS-IASI data are then assimilated, combined with the MLS data. This first LATMOS-IASI data assimilation experiment shows that the resulting analysis is quite similar to the one obtained from the combined MLS and SCIAMACHY data assimilation.

scholarly journals Regularized variational data assimilation for bias treatment using the Wasserstein metric

2020 ◽  
Vol 146 (730) ◽  
pp. 2332-2346
Author(s):  
Sagar K. Tamang ◽  
Ardeshir Ebtehaj ◽  
Dongmian Zou ◽  
Gilad Lerman
Keyword(s):  
Data Assimilation ◽  
Variational Data Assimilation ◽  
Wasserstein Metric

scholarly journals Understanding and Predicting Nonlinear Turbulent Dynamical Systems with Information Theory

Atmosphere ◽  
2019 ◽  
Vol 10 (5) ◽  
pp. 248
Author(s):  
Nan Chen ◽  
Xiao Hou ◽  
Qin Li ◽  
Yingda Li
Keyword(s):  
Information Theory ◽  
Dynamical Systems ◽  
Information Criterion ◽  
Gaussian Mixture ◽  
Model Error ◽  
High Dimensions ◽  
New Approach ◽  
Fluctuation Dissipation ◽  
Spectrum Density ◽  
Non Gaussian

Complex nonlinear turbulent dynamical systems are ubiquitous in many areas. Quantifying the model error and model uncertainty plays an important role in understanding and predicting complex dynamical systems. In the first part of this article, a simple information criterion is developed to assess the model error in imperfect models. This effective information criterion takes into account the information in both the equilibrium statistics and the temporal autocorrelation function, where the latter is written in the form of the spectrum density that permits the quantification via information theory. This information criterion facilitates the study of model reduction, stochastic parameterizations, and intermittent events. In the second part of this article, a new efficient method is developed to improve the computation of the linear response via the Fluctuation Dissipation Theorem (FDT). This new approach makes use of a Gaussian Mixture (GM) to describe the unperturbed probability density function in high dimensions and avoids utilizing Gaussian approximations in computing the statistical response, as is widely used in the quasi-Gaussian (qG) FDT. Testing examples show that this GM FDT outperforms qG FDT in various strong non-Gaussian regimes.

scholarly journals Cluster Sampling Filters for Non-Gaussian Data Assimilation

Atmosphere ◽  
2018 ◽  
Vol 9 (6) ◽  
pp. 213 ◽  
Author(s):  
Ahmed Attia ◽  
Azam Moosavi ◽  
Adrian Sandu
Keyword(s):  
Data Assimilation ◽  
Cluster Sampling ◽  
Non Gaussian ◽  
Gaussian Data

scholarly journals Setting Alarm Thresholds in Measurements with Systematic and Random Errors

Stats ◽  
2019 ◽  
Vol 2 (2) ◽  
pp. 259-271 ◽  
Author(s):  
Tom Burr ◽  
Elisa Bonner ◽  
Kamil Krzysztoszek ◽  
Claude Norman
Keyword(s):  
Probability Distributions ◽  
Mean Shift ◽  
Measurement Data ◽  
Main Tool ◽  
False Alarm Probability ◽  
Data Uncertainty ◽  
Grouped Data ◽  
Random Errors ◽  
Group Data ◽  
Ordered Data

For statistical evaluations that involve within-group and between-group variance components (denoted σ W 2 and σ B 2 , respectively), there is sometimes a need to monitor for a shift in the mean of time-ordered data. Uncertainty in the estimates σ ^ W 2 and σ ^ B 2 should be accounted for when setting alarm thresholds to check for a mean shift as both σ W 2 and σ B 2 must be estimated. One-way random effects analysis of variance (ANOVA) is the main tool for analysing such grouped data. Nearly all of the ANOVA applications assume that both the within-group and between-group components are normally distributed. However, depending on the application, the within-group and/or between-group probability distributions might not be well approximated by a normal distribution. This review paper uses the same example throughout to illustrate the possible approaches to setting alarm limits in grouped data, depending on what is assumed about the within-group and between-group probability distributions. The example involves measurement data, for which systematic errors are assumed to remain constant within a group, and to change between groups. The false alarm probability depends on the assumed measurement error model and its within-group and between-group error variances, which are estimated while using historical data, usually with ample within-group data, but with a small number of groups (three to 10 typically). This paper illustrates the parametric, semi-parametric, and non-parametric options to setting alarm thresholds in such grouped data.

Sign in / Sign up

Export Citation Format

Share Document