Localization and Sampling Error Correction in Ensemble Kalman Filter Data Assimilation

2012 ◽  
Vol 140 (7) ◽  
pp. 2359-2371 ◽  
Author(s):  
Jeffrey L. Anderson
Keyword(s):  
Kalman Filter ◽  
Error Correction ◽  
Ensemble Kalman Filter ◽  
Sampling Error ◽  
Lookup Table ◽  
Correction Algorithm ◽  
Ensemble Size ◽  
State Variable ◽  
Sample Covariance ◽  
Error Correction Algorithm

Abstract Ensemble Kalman filters use the sample covariance of an observation and a model state variable to update a prior estimate of the state variable. The sample covariance can be suboptimal as a result of small ensemble size, model error, model nonlinearity, and other factors. The most common algorithms for dealing with these deficiencies are inflation and covariance localization. A statistical model of errors in ensemble Kalman filter sample covariances is described and leads to an algorithm that reduces ensemble filter root-mean-square error for some applications. This sampling error correction algorithm uses prior information about the distribution of the correlation between an observation and a state variable. Offline Monte Carlo simulation is used to build a lookup table that contains a correction factor between 0 and 1 depending on the ensemble size and the ensemble sample correlation. Correction factors are applied like a traditional localization for each pair of observations and state variables during an ensemble assimilation. The algorithm is applied to two low-order models and reduces the sensitivity of the ensemble assimilation error to the strength of traditional localization. When tested in perfect model experiments in a larger model, the dynamical core of a general circulation model, the sampling error correction algorithm produces analyses that are closer to the truth and also reduces sensitivity to traditional localization strength.

scholarly journals Parameters Estimation and Prediction of Water Movement and Solute Transport in Layered, Variably Saturated Soils Using the Ensemble Kalman Filter

Water ◽  
2019 ◽  
Vol 11 (7) ◽  
pp. 1520
Author(s):  
Zheng Jiang ◽  
Quanzhong Huang ◽  
Gendong Li ◽  
Guangyong Li
Keyword(s):  
Kalman Filter ◽  
Solute Transport ◽  
Ensemble Kalman Filter ◽  
Water Movement ◽  
Parameters Estimation ◽  
Saturated Soils ◽  
Hydrological Models ◽  
State Variables ◽  
Ensemble Size ◽  
Highly Nonlinear

The parameters of water movement and solute transport models are essential for the accurate simulation of soil moisture and salinity, particularly for layered soils in field conditions. Parameter estimation can be achieved using the inverse modeling method. However, this type of method cannot fully consider the uncertainties of measurements, boundary conditions, and parameters, resulting in inaccurate estimations of parameters and predictions of state variables. The ensemble Kalman filter (EnKF) is well-suited to data assimilation and parameter prediction in Situations with large numbers of variables and uncertainties. Thus, in this study, the EnKF was used to estimate the parameters of water movement and solute transport in layered, variably saturated soils. Our results indicate that when used in conjunction with the HYDRUS-1D software (University of California Riverside, California, CA, USA) the EnKF effectively estimates parameters and predicts state variables for layered, variably saturated soils. The assimilation of factors such as the initial perturbation and ensemble size significantly affected in the simulated results. A proposed ensemble size range of 50–100 was used when applying the EnKF to the highly nonlinear hydrological models of the present study. Although the simulation results for moisture did not exhibit substantial improvement with the assimilation, the simulation of the salinity was significantly improved through the assimilation of the salinity and relative solutetransport parameters. Reducing the uncertainties in measured data can improve the goodness-of-fit in the application of the EnKF method. Sparse field condition observation data also benefited from the accurate measurement of state variables in the case of EnKF assimilation. However, the application of the EnKF algorithm for layered, variably saturated soils with hydrological models requires further study, because it is a challenging and highly nonlinear problem.

scholarly journals Oxford Nanopore Sequencing, Hybrid Error Correction, and de novo Assembly of a Eukaryotic Genome

2015 ◽  
Author(s):  
Sara Goodwin ◽  
James Gurtowski ◽  
Scott Ethe-Sayers ◽  
Panchajanya Deshpande ◽  
Michael Schatz ◽  
...  
Keyword(s):  
Error Correction ◽  
De Novo Assembly ◽  
De Novo ◽  
Correction Algorithm ◽  
Membrane Pore ◽  
Complete Representation ◽  
Oxford Nanopore ◽  
Long Read ◽  
Error Correction Algorithm ◽  
Sequencing Instrument

Monitoring the progress of DNA molecules through a membrane pore has been postulated as a method for sequencing DNA for several decades. Recently, a nanopore-based sequencing instrument, the Oxford Nanopore MinION, has become available that we used for sequencing the S. cerevisiae genome. To make use of these data, we developed a novel open-source hybrid error correction algorithm Nanocorr (https://github.com/jgurtowski/nanocorr) specifically for Oxford Nanopore reads, as existing packages were incapable of assembling the long read lengths (5-50kbp) at such high error rate (between ~5 and 40% error). With this new method we were able to perform a hybrid error correction of the nanopore reads using complementary MiSeq data and produce a de novo assembly that is highly contiguous and accurate: the contig N50 length is more than ten-times greater than an Illumina-only assembly (678kb versus 59.9kbp), and has greater than 99.88% consensus identity when compared to the reference. Furthermore, the assembly with the long nanopore reads presents a much more complete representation of the features of the genome and correctly assembles gene cassettes, rRNAs, transposable elements, and other genomic features that were almost entirely absent in the Illumina-only assembly.

Research on error-correction algorithm of high-speed QKD system based on FPGA

2019 ◽  
Vol 17 (02) ◽  
pp. 1950013
Author(s):  
Shi-Biao Tang ◽  
Jie Cheng
Keyword(s):  
Error Correction ◽  
Quantum Key Distribution ◽  
High Speed ◽  
Information Leakage ◽  
Correction Algorithm ◽  
Performance Limit ◽  
Performance Requirement ◽  
The Future ◽  
Field Programmable ◽  
Error Correction Algorithm

In the process of quantum key distribution (QKD), error correction algorithm is used to correct the error bits of the key at both ends. The existing applied QKD system has a low key rate and is generally Kbps of magnitude. Therefore, the performance requirement of data processing such as error correction is not high. In order to cope with the development demand of high-speed QKD system in the future, this paper introduces the Winnow algorithm to realize high-speed parity and hamming error correction based on Field Programmable Gate Array (FPGA), and explores the performance limit of this algorithm. FPGA hardware implementation can achieve the scale of Mbps bandwidth, with choosing different group length of sifted key by different error rate, and can achieve higher error correction efficiency by reducing the information leakage in the process of error correction, and improves the QKD system’s secure key rate, thus helping the future high-speed QKD system.

Sampling Error Correction Evaluated Using a Convective-Scale 1000-Member Ensemble

2019 ◽  
Vol 148 (3) ◽  
pp. 1229-1249 ◽  
Author(s):  
Tobias Necker ◽  
Martin Weissmann ◽  
Yvonne Ruckstuhl ◽  
Jeffrey Anderson ◽  
Takemasa Miyoshi
Keyword(s):  
Sensitivity Analysis ◽  
Data Assimilation ◽  
Error Correction ◽  
Degrees Of Freedom ◽  
Sampling Error ◽  
Lookup Table ◽  
Ensemble Prediction ◽  
Sampling Errors ◽  
Convective Scale ◽  
The Impact

Abstract State-of-the-art ensemble prediction systems usually provide ensembles with only 20–250 members for estimating the uncertainty of the forecast and its spatial and spatiotemporal covariance. Given that the degrees of freedom of atmospheric models are several magnitudes higher, the estimates are therefore substantially affected by sampling errors. For error covariances, spurious correlations lead to random sampling errors, but also a systematic overestimation of the correlation. A common approach to mitigate the impact of sampling errors for data assimilation is to localize correlations. However, this is a challenging task given that physical correlations in the atmosphere can extend over long distances. Besides data assimilation, sampling errors pose an issue for the investigation of spatiotemporal correlations using ensemble sensitivity analysis. Our study evaluates a statistical approach for correcting sampling errors. The applied sampling error correction is a lookup table–based approach and therefore computationally very efficient. We show that this approach substantially improves both the estimates of spatial correlations for data assimilation as well as spatiotemporal correlations for ensemble sensitivity analysis. The evaluation is performed using the first convective-scale 1000-member ensemble simulation for central Europe. Correlations of the 1000-member ensemble forecast serve as truth to assess the performance of the sampling error correction for smaller subsets of the full ensemble. The sampling error correction strongly reduced both random and systematic errors for all evaluated variables, ensemble sizes, and lead times.

Sign in / Sign up

Export Citation Format

Share Document