It was emphasized in Chapter 1 that low-order, linear time-invariant models provide the foundation for much intuition about dynamic phenomena in the real world. This chapter provides a brief review of the characteristics and behavior of linear models, beginning with these simple cases and then progressing to more complex examples where this intuition no longer holds: infinite-dimensional and time-varying linear models. In continuous time, infinite-dimensional linear models arise naturally from linear partial differential equations whereas in discrete time, infinite-dimensional linear models may be used to represent a variety of “slow decay” effects. Time-varying linear models are also extremely flexible: In the continuous-time case, many of the ordinary differential equations defining special functions (e.g., the equations defining Bessel functions) may be viewed as time-varying linear models; in the discrete case, the gamma function arises naturally as the solution of a time-varying difference equation. Sec. 2.1 gives a brief discussion of low-order, time-invariant linear dynamic models, using second-order examples to illustrate both the “typical” and “less typical” behavior that is possible for these models. One of the most powerful results of linear system theory is that any time-invariant linear dynamic system may be represented as either a moving average (i.e., convolution-type) model or an autoregressive one. Sec. 2.2 presents a short review of these ideas, which will serve to establish both notation and a certain amount of useful intuition for the discussion of NARMAX models presented in Chapter 4. Sec. 2.3 then briefly considers the problem of characterizing linear models, introducing four standard input sequences that are typical of those used in linear model characterization. These standard sequences are then used in subsequent chapters to illustrate differences between nonlinear model behavior and linear model behavior. Sec. 2.4 provides a brief introduction to infinite-dimensional linear systems, including both continuous-time and discrete-time examples. Sec. 2.5 provides a similar introduction to the subject of time-varying linear systems, emphasizing the flexibility of this class. Finally, Sec. 2.6 briefly considers the nature of linearity, presenting some results that may be used to define useful classes of nonlinear models.
Bayesian Inference for Linear Dynamic Models With Dirichlet Process Mixtures
