Optimal Nonlinear Estimation of Linear Stochastic Systems

1994 ◽  
Vol 116 (3) ◽  
pp. 529-536 ◽  
Author(s):  
M. A. Hopkins ◽  
H. F. VanLandingham
Keyword(s):  
Stochastic Systems ◽  
Linear Time ◽  
Minimum Mean Square Error ◽  
Parameter Estimates ◽  
Model Parameters ◽  
Error Estimator ◽  
Nonlinear Method ◽  
True Parameter ◽  
Single Input Single Output ◽  
Persistent Excitation

This paper presents a new nonlinear method of simultaneous parameter and state estimation called pseudo-linear identification (PLID), for stochastic linear time-invariant discrete-time systems. No assumptions are required about pole or zero locations; nor about relative degree, except that the system transfer function must be strictly proper. Under standard gaussian assumptions, for completely controllable and observable systems, it is proved that PLID is the minimum mean-square-error estimator of the states and model parameters, conditioned on the input and output measurements. It is also proved, given persistent excitation, that the parameter estimates converge a.e. to the true parameter values. All results have been extended to the multiple-input, multiple-output case, but the single-input, single-output case is presented here to simplify notation.

Optimal Nonlinear Estimation of Linear Stochastic Systems: The Multivariable Extension

1996 ◽  
Vol 118 (2) ◽  
pp. 350-353 ◽  
Author(s):  
M. A. Hopkins ◽  
H. F. VanLandingham
Keyword(s):  
Stochastic Systems ◽  
Transfer Functions ◽  
Mimo Systems ◽  
Linear Time ◽  
Nonlinear Method ◽  
Single Input Single Output ◽  
Linear Time Invariant ◽  
Single Output ◽  
Time Invariant ◽  
Time Systems

This paper extends to multi-input multi-output (MIMO) systems a nonlinear method of simultaneous parameter and state estimation that appeared in the ASME JDSM&C (September, 1994), for single-input single-output (SISO) systems. The method is called pseudo-linear identification (PLID), and applies to stochastic linear time-invariant discrete-time systems. No assumptions are required about pole or zero locations; nor about relative degree, except that the system transfer functions must be strictly proper. In the earlier paper, proofs of optimality and convergence were given. Extensions of those proofs to the MIMO case are also given here.

OPTIMAL ADAPTIVE PREDICTION FOR SISO SYSTEMS

2009 ◽  
Vol 18 (05) ◽  
pp. 993-1003
Author(s):  
BEHNAM SHAHRRAVA
Keyword(s):  
Linear Time ◽  
Minimum Mean Square Error ◽  
Parameter Estimates ◽  
Single Input Single Output ◽  
Linear Time Invariant ◽  
Single Output ◽  
Noise Perturbation ◽  
Single Input ◽  
Time Invariant ◽  
Time Systems

An adaptive predictor that is optimal in the minimum mean-square error (MMSE) sense at each step is obtained for single-input, single-output (SISO) linear time-invariant discrete-time systems having general delay and white noise perturbation. The adaptive d-step-ahead predictor includes the uncertainty associated with the parameter and state estimates whereas conventional adaptive predictors, that are asymptotically optimal, ignore the uncertainty in the parameter estimates.

scholarly journals Bayesian inference of distributed time delay in transcriptional and translational regulation

2019 ◽  
Vol 36 (2) ◽  
pp. 586-593
Author(s):  
Boseung Choi ◽  
Yu-Yu Cheng ◽  
Selahattin Cinar ◽  
William Ott ◽  
Matthew R Bennett ◽  
...  
Keyword(s):  
Bayesian Inference ◽  
Stochastic Systems ◽  
Imaging Techniques ◽  
Reaction Rates ◽  
Biochemical Networks ◽  
Supplementary Information ◽  
Mcmc Methods ◽  
Parameter Estimates ◽  
Model Parameters ◽  
Cell Functions

Abstract Motivation Advances in experimental and imaging techniques have allowed for unprecedented insights into the dynamical processes within individual cells. However, many facets of intracellular dynamics remain hidden, or can be measured only indirectly. This makes it challenging to reconstruct the regulatory networks that govern the biochemical processes underlying various cell functions. Current estimation techniques for inferring reaction rates frequently rely on marginalization over unobserved processes and states. Even in simple systems this approach can be computationally challenging, and can lead to large uncertainties and lack of robustness in parameter estimates. Therefore we will require alternative approaches to efficiently uncover the interactions in complex biochemical networks. Results We propose a Bayesian inference framework based on replacing uninteresting or unobserved reactions with time delays. Although the resulting models are non-Markovian, recent results on stochastic systems with random delays allow us to rigorously obtain expressions for the likelihoods of model parameters. In turn, this allows us to extend MCMC methods to efficiently estimate reaction rates, and delay distribution parameters, from single-cell assays. We illustrate the advantages, and potential pitfalls, of the approach using a birth–death model with both synthetic and experimental data, and show that we can robustly infer model parameters using a relatively small number of measurements. We demonstrate how to do so even when only the relative molecule count within the cell is measured, as in the case of fluorescence microscopy. Availability and implementation Accompanying code in R is available at https://github.com/cbskust/DDE_BD. Supplementary information Supplementary data are available at Bioinformatics online.

scholarly journals Bayesian inference of distributed time delay in transcriptional and translational regulation

2019 ◽  
Author(s):  
Boseung Choi ◽  
Yu-Yu Cheng ◽  
Selahittin Cinar ◽  
William Ott ◽  
Matthew R. Bennett ◽  
...  
Keyword(s):  
Bayesian Inference ◽  
Regulatory Networks ◽  
Stochastic Systems ◽  
Imaging Techniques ◽  
Reaction Rates ◽  
Biochemical Networks ◽  
Mcmc Methods ◽  
Parameter Estimates ◽  
Model Parameters ◽  
Cell Functions

AbstractMotivationAdvances in experimental and imaging techniques have allowed for unprecedented insights into the dynamical processes within individual cells. However, many facets of intracellular dynamics remain hidden, or can be measured only indirectly. This makes it challenging to reconstruct the regulatory networks that govern the biochemical processes underlying various cell functions. Current estimation techniques for inferring reaction rates frequently rely on marginalization over unobserved processes and states. Even in simple systems this approach can be computationally challenging, and can lead to large uncertainties and lack of robustness in parameter estimates. Therefore we will require alternative approaches to efficiently uncover the interactions in complex biochemical networks.ResultsWe propose a Bayesian inference framework based on replacing uninteresting or unobserved reactions with time delays. Although the resulting models are non-Markovian, recent results on stochastic systems with random delays allow us to rigorously obtain expressions for the likelihoods of model parameters. In turn, this allows us to extend MCMC methods to efficiently estimate reaction rates, and delay distribution parameters, from single-cell assays. We illustrate the advantages, and potential pitfalls, of the approach using a birth-death model with both synthetic and experimental data, and show that we can robustly infer model parameters using a relatively small number of measurements. We demonstrate how to do so even when only the relative molecule count within the cell is measured, as in the case of fluorescence microscopy.

Standard Asymptotic Theory

2017 ◽  
Author(s):  
Russell Cheng
Keyword(s):  
Asymptotic Theory ◽  
Asymptotic Variance ◽  
Statistical Significance ◽  
Regularity Conditions ◽  
Parameter Estimates ◽  
Asymptotic Results ◽  
Ml Estimation ◽  
True Parameter ◽  
Log Likelihood ◽  
Ml Estimator

This book relies on maximum likelihood (ML) estimation of parameters. Asymptotic theory assumes regularity conditions hold when the ML estimator is consistent. Typically an additional third derivative condition is assumed to ensure that the ML estimator is also asymptotically normally distributed. Standard asymptotic results that then hold are summarized in this chapter; for example, the asymptotic variance of the ML estimator is then given by the Fisher information formula, and the log-likelihood ratio, the Wald and the score statistics for testing the statistical significance of parameter estimates are all asymptotically equivalent. Also, the useful profile log-likelihood then behaves exactly as a standard log-likelihood only in a parameter space of just one dimension. Further, the model can be reparametrized to make it locally orthogonal in the neighbourhood of the true parameter value. The large exponential family of models is briefly reviewed where a unified set of regular conditions can be obtained.

Complementary Sensitivity Design of PID Controllers for Arbitrary-Order Transfer Functions With Time Delay

2008 ◽  
Author(s):  
Tooran Emami ◽  
John M. Watkins
Keyword(s):  
Time Delay ◽  
Transfer Functions ◽  
Linear Time ◽  
Order System ◽  
Pid Controllers ◽  
Single Input Single Output ◽  
Linear Time Invariant ◽  
Single Output ◽  
Plant Transfer ◽  
Time Invariant

A graphical technique for finding all proportional integral derivative (PID) controllers that stabilize a given single-input-single-output (SISO) linear time-invariant (LTI) system of any order system with time delay has been solved. In this paper a method is introduced that finds all PID controllers that also satisfy an H∞ complementary sensitivity constraint. This problem can be solved by finding all PID controllers that simultaneously stabilize the closed-loop characteristic polynomial and satisfy constraints defined by a set of related complex polynomials. A key advantage of this procedure is the fact that it does not require the plant transfer function, only its frequency response.

scholarly journals Parameter stability and consistency in an alongshore-current model determined with Markov chain Monte Carlo

2008 ◽  
Vol 10 (2) ◽  
pp. 153-162 ◽  
Author(s):  
B. G. Ruessink
Keyword(s):  
Monte Carlo ◽  
Markov Chain ◽  
Markov Chain Monte Carlo ◽  
Parameter Estimates ◽  
Model Parameters ◽  
Parameter Stability ◽  
Small Uncertainty ◽  
Best Fit ◽  
Stability And Consistency

When a numerical model is to be used as a practical tool, its parameters should preferably be stable and consistent, that is, possess a small uncertainty and be time-invariant. Using data and predictions of alongshore mean currents flowing on a beach as a case study, this paper illustrates how parameter stability and consistency can be assessed using Markov chain Monte Carlo. Within a single calibration run, Markov chain Monte Carlo estimates the parameter posterior probability density function, its mode being the best-fit parameter set. Parameter stability is investigated by stepwise adding new data to a calibration run, while consistency is examined by calibrating the model on different datasets of equal length. The results for the present case study indicate that various tidal cycles with strong (say, >0.5 m/s) currents are required to obtain stable parameter estimates, and that the best-fit model parameters and the underlying posterior distribution are strongly time-varying. This inconsistent parameter behavior may reflect unresolved variability of the processes represented by the parameters, or may represent compensational behavior for temporal violations in specific model assumptions.

Multi-day flow forecasting using the Kalman filter

1991 ◽  
Vol 18 (2) ◽  
pp. 320-327 ◽  
Author(s):  
Murray A. Fitch ◽  
Edward A. McBean
Keyword(s):  
Kalman Filter ◽  
Prediction Model ◽  
Linear Prediction ◽  
Measured Data ◽  
System Model ◽  
Parameter Estimates ◽  
Model Parameters ◽  
Forecasting Performance ◽  
Optimal Forecasting ◽  
Linear Prediction Model

A model is developed for the prediction of river flows resulting from combined snowmelt and precipitation. The model employs a Kalman filter to reflect uncertainty both in the measured data and in the system model parameters. The forecasting algorithm is used to develop multi-day forecasts for the Sturgeon River, Ontario. The algorithm is shown to develop good 1-day and 2-day ahead forecasts, but the linear prediction model is found inadequate for longer-term forecasts. Good initial parameter estimates are shown to be essential for optimal forecasting performance. Key words: Kalman filter, streamflow forecast, multi-day, streamflow, Sturgeon River, MISP algorithm.

Generalised DOPs with Consideration of the Influence Function of Signal-in-Space Errors

2011 ◽  
Vol 64 (S1) ◽  
pp. S3-S18 ◽  
Author(s):  
Yuanxi Yang ◽  
Jinlong Li ◽  
Junyi Xu ◽  
Jing Tang
Keyword(s):  
Least Squares ◽  
Stochastic Models ◽  
Influence Function ◽  
A Priori ◽  
Functional Model ◽  
Global Navigation Satellite Systems ◽  
Parameter Estimates ◽  
Model Parameters ◽  
Integrated Navigation ◽  
Satellite Systems

Integrated navigation using multiple Global Navigation Satellite Systems (GNSS) is beneficial to increase the number of observable satellites, alleviate the effects of systematic errors and improve the accuracy of positioning, navigation and timing (PNT). When multiple constellations and multiple frequency measurements are employed, the functional and stochastic models as well as the estimation principle for PNT may be different. Therefore, the commonly used definition of “dilution of precision (DOP)” based on the least squares (LS) estimation and unified functional and stochastic models will be not applicable anymore. In this paper, three types of generalised DOPs are defined. The first type of generalised DOP is based on the error influence function (IF) of pseudo-ranges that reflects the geometry strength of the measurements, error magnitude and the estimation risk criteria. When the least squares estimation is used, the first type of generalised DOP is identical to the one commonly used. In order to define the first type of generalised DOP, an IF of signal–in-space (SIS) errors on the parameter estimates of PNT is derived. The second type of generalised DOP is defined based on the functional model with additional systematic parameters induced by the compatibility and interoperability problems among different GNSS systems. The third type of generalised DOP is defined based on Bayesian estimation in which the a priori information of the model parameters is taken into account. This is suitable for evaluating the precision of kinematic positioning or navigation. Different types of generalised DOPs are suitable for different PNT scenarios and an example for the calculation of these DOPs for multi-GNSS systems including GPS, GLONASS, Compass and Galileo is given. New observation equations of Compass and GLONASS that may contain additional parameters for interoperability are specifically investigated. It shows that if the interoperability of multi-GNSS is not fulfilled, the increased number of satellites will not significantly reduce the generalised DOP value. Furthermore, the outlying measurements will not change the original DOP, but will change the first type of generalised DOP which includes a robust error IF. A priori information of the model parameters will also reduce the DOP.

Sign in / Sign up

Export Citation Format

Share Document