Volterra Models

One of the main points of Chapter 4 is that nonlinear moving-average (NMAX) models are both inherently better-behaved and easier to analyze than more general NARMAX models. For example, it was shown in Sec. 4.2.2 that if ɡ(· · ·) is a continuous map from Rq+1 to R1 and if ys = ɡ (us,..., us), then uk → us implies yk → ys. Although it is not always satisfied, continuity is a relatively weak condition to impose on the map ɡ(· · ·) . For example, Hammerstein or Wiener models based on moving average models and the hard saturation nonlinearity represent discontinuous members of the class of NMAX models. This chapter considers the analytical consequences of requiring ɡ(·) to be analytic, implying the existence of a Taylor series expansion. Although this requirement is much stronger than continuity, it often holds, and when it does, it leads to an explicit representation: Volterra models. The principal objective of this chapter is to define the class of Volterra models and discuss various important special cases and qualitative results. Most of this discussion is concerned with the class V(N,M) of finite Volterra models, which includes the class of linear finite impulse response models as a special case, along with a number of practically important nonlinear moving average model classes. In particular, the finite Volterra model class includes Hammerstein models, Wiener models, and Uryson models, along with other more general model structures. In addition, one of the results established in this chapter is that most of the bilinear models discussed in Chapter 3 may be expressed as infinite-order Volterra models. This result is somewhat analogous to the equivalence between finite-dimensional linear autoregressive models and infinite-dimensional linear moving average models discussed in Chapter 2. The bilinear model result presented here is strictly weaker, however, since there exist classes of bilinear models that do not possess Volterra series representations. Specifically, it is shown in Sec. 5.6 that completely bilinear models do not exhibit Volterra series representations. Conversely, one of the results discussed at the end of this chapter is that the class of discrete-time fading memory systems may be approximated arbitrarily well by finite Volterra models (Boyd and Chua, 1985).

Linear Dynamic Models

It was emphasized in Chapter 1 that low-order, linear time-invariant models provide the foundation for much intuition about dynamic phenomena in the real world. This chapter provides a brief review of the characteristics and behavior of linear models, beginning with these simple cases and then progressing to more complex examples where this intuition no longer holds: infinite-dimensional and time-varying linear models. In continuous time, infinite-dimensional linear models arise naturally from linear partial differential equations whereas in discrete time, infinite-dimensional linear models may be used to represent a variety of “slow decay” effects. Time-varying linear models are also extremely flexible: In the continuous-time case, many of the ordinary differential equations defining special functions (e.g., the equations defining Bessel functions) may be viewed as time-varying linear models; in the discrete case, the gamma function arises naturally as the solution of a time-varying difference equation. Sec. 2.1 gives a brief discussion of low-order, time-invariant linear dynamic models, using second-order examples to illustrate both the “typical” and “less typical” behavior that is possible for these models. One of the most powerful results of linear system theory is that any time-invariant linear dynamic system may be represented as either a moving average (i.e., convolution-type) model or an autoregressive one. Sec. 2.2 presents a short review of these ideas, which will serve to establish both notation and a certain amount of useful intuition for the discussion of NARMAX models presented in Chapter 4. Sec. 2.3 then briefly considers the problem of characterizing linear models, introducing four standard input sequences that are typical of those used in linear model characterization. These standard sequences are then used in subsequent chapters to illustrate differences between nonlinear model behavior and linear model behavior. Sec. 2.4 provides a brief introduction to infinite-dimensional linear systems, including both continuous-time and discrete-time examples. Sec. 2.5 provides a similar introduction to the subject of time-varying linear systems, emphasizing the flexibility of this class. Finally, Sec. 2.6 briefly considers the nature of linearity, presenting some results that may be used to define useful classes of nonlinear models.

The Art of Model Development

The primary objective of this book has been to present a reasonably broad overview of the different classes of discrete-time dynamic models that have been proposed for empirical modeling, particularly in the process control literature. In its simplest form, the empirical modeling process consists of the following four steps: 1. Select a class C of model structures 2. Generate input/output data from the physical process P 3. Determine the model M ∊ C that best fits this dataset 4. Assess the general validity of the model M. The objective of this final chapter is to briefly examine these four modeling steps, with particular emphasis on the first since the choice of the model class C ultimately determines the utility of the empirical model, both with respect to the application (e.g., the difficulty of solving the resulting model-based control problem) and with respect to fidelity of approximation. Some of the basic issues of model structure selection are introduced in Sec. 8.1 and a more detailed treatment is given in Sec. 8.3, emphasizing connections with results presented in earlier chapters; in addition, the problem of model structure selection is an important component of the case studies presented in Secs. 8.2 and 8.5. The second step in this procedure—input sequence design—is discussed in some detail in Sec. 8.4 and is an important component of the second case study (Sec. 8.5). The literature associated with the parameter estimation problem—the third step in the empirical modeling process—is much too large to attempt to survey here, but a brief summary of some representative results is given in Sec. 8.1.1. Finally, the task of model validation often depends strongly on the details of the physical system being modelled and the ultimate application intended for the model. Consequently, detailed treatment of this topic also lies beyond the scope of this book but again, some representative results are discussed briefly in Sec. 8.1.3 and illustrated in the first case study (Sec. 8.2). Finally, Sec. 8.6 concludes both the chapter and the book with some philosophical observations on the problem of developing moderate-complexity, discrete-time dynamic models to approximate the behavior of high-complexity, continuous-time physical systems.

Relations between Model Classes

Chapter 2 has provided a brief review of some of the important characteristics of linear models, and Chapters 3 through 6 have introduced and discussed a number of specific classes of nonlinear models. The focus of this chapter is on the relationships that exist between these different model classes. Some of these relationships have already been discussed briefly in isolated places in earlier chapters, but this chapter attempts to give a much broader overview of how different model classes relate. In particular, it is useful to note that many popular model classes are included as proper subsets of other, larger classes. In more subtle cases, one class will be “almost included” in another, larger class, but a portion of the first class will fail to satisfy this inclusion. As a specific example, it was noted in Chapter 4 that the class of Hammerstein models is a proper subset of the class of additive NARMAX models. In contrast, it was also shown in Chapter 4 that Wiener models are not members of the additive NARMAX class except in the two degenerate cases where Wiener and Hammerstein models coincide: linear dynamic models and static nonlinearities. This chapter begins with a summary of these inclusion and exclusion results in Sec. 7.1, including both results assembled from previous chapters and a few new ones. Of particular interest are questions concerning the relationship between structurally-defined model classes and behaviorally-defined classes since these questions are directly related to the practical problem of initial model structure selection for empirical modeling, a topic considered further in Chapter 8. One of the principal objectives of this chapter is to illustrate the utility of category theory in characterizing relations between different classes of dynamic models. Essentially, category theory is a branch of mathematics whose aim is to elucidate relations between different classes of mathematical objects in extremely general terms. More specifically, category theory deals with mathematical objects (in fact, called objects) and transformations between objects (called morphisms), requiring only that these transformations be “well behaved” with respect to successive application (called composition of morphisms).

Linear Multimodels

This chapter briefly discusses the class of linear multimodels, which are globally nonlinear dynamic models, obtained by “piecing together” several local, linear dynamic models. The motivation behind this approach to model development is the observation that an approximate model’s complexity generally increases with the range of operation over which it must be valid. In particular, it has been noted repeatedly that linear models are often adequate approximations of process dynamics over a sufficiently narrow operating range. Thus, if the total operating range can be decomposed into small enough subsets, it is reasonable to expect that linear models will provide reasonable characterizations of process dynamics over these local regimes. The process of piecing these local models together may be approached in a number of different ways, and the details of this process are important, as subsequent examples illustrate. The principal issues that must be addressed in developing linear multimodels are illustrated with the example of a batch fermentation reactor, described in Sec. 6.1. Three possible definitions of discrete-time linear multimodels are presented in Secs. 6.2.1 through 6.2.3. The first of these definitions is based on Johansen and Foss (1993), whereas the second definition represents an apparently slight variation on the first that can lead to fundamentally different qualitative behavior. The third definition of linear multimodels is a special case of the first, described in Tong (1990) under the name open-loop threshold autoregressive models. The primary difference between Tong’s definition and that of Johansen and Foss is whether the regions of local model validity can overlap: they can in the models of Johansen and Foss, but they cannot in Tong’s model. An extremely important practical issue is the criterion by which local models are selected. The primary focus here is on Tong’s class of linear multimodels where each local model completely describes the global model dynamics over some specified operating range. If this operating range is defined entirely in terms of the input sequence {u(k)} , these models will be designated input-selected, whereas if the operating range is defined entirely in terms of the output sequence {y(k)}, they will be called output-selected; if both inputs and outputs are involved, the term generally selected will be used.

Four Views of Nonlinearity

The review of linear models presented in Chapter 2 was intended to provide a baseline, establishing notation and reviewing some important aspects of this reference class against which nonlinear models are necessarily judged. This chapter demonstrates that nonlinearity can be approached from at least two fundamentally different directions. The first of these directions is structural-— by far the most common approach—in which a class of nonlinear models is defined by specifying the basic structure of all elements of that class. The Hammerstein, Wiener, and Uryson model classes discussed in Chapter 1 illustrate this approach. The structurally defined model class considered in this chapter is the class of bilinear models discussed in Sec. 3.1, which may be viewed as “almost linear” for reasons that will become apparent. Alternatively, it is possible to adopt behavioral definitions of nonlinear model classes, although this approach is generally more difficult. There, a particular type of input/output behavior is specified and subsequent analysis seeks model structures that can exhibit this behavior. In practice, this approach tends to be difficult because it is often not clear how to constuct explicit examples that exhibit specified qualitative behavior. Of necessity, then, the primary focus of this book is structurally defined model classes like the bilinear models discussed in Sec. 3.1, although three behaviorally defined model classes are considered in some detail in Secs. 3.2 through 3.4. The first of these is the class of homogeneous models, obtained by relaxing one of the two defining conditions for linearity: homogeneous models do not obey the superposition principle of linear systems, but they are invariant under scaling of the input sequence by arbitrary real constants. Relaxing these conditions further and requiring only that this scaling hold for positive constants leads to the class of positive-homogeneous models, described in Sec. 3.3. Alternatively, requiring linearity to hold but only for constant input sequences leads to the class of static-linear models, described in Sec. 3.4. As these and subsequent discussions illustrate, some remarkably general results may be obtained concerning the structure of these three model classes.

NARMAX Models

As noted In Chapter 1, the general class of NARMAX models is extremely broad and includes almost all of the other discrete-time model classes discussed in this book, linear and nonlinear. Because of its enormity, little can be said about the qualitative behavior of the NARMAX family in general, but it is surprising how much can be said about the behavior of some important subclasses. In particular, there are sharp qualitative distinctions between nonlinear moving average models (NMAX models) and nonlinear autoregressive models (NARX models). Since these representations are equivalent for linear models (c.f. Chapter 2), this observation highlights one important difference between linear and nonlinear discrete-time dynamic models. Consequently, this chapter focuses primarily on the structure and qualitative behavior of various subclasses of the NARMAX family, particularly the NMAX and NARX classes. More specifically, Sec. 4.1 presents a brief discussion of the general NARMAX class, defining five important subclasses that are discussed in subsequent sections: the NMAX class (Sec. 4.2), the NARX class (Sec. 4.3), the class of (structurally) additive NARMAX models (Sec. 4.4), the class of polynomial NARMAX models (Sec. 4.5), and the class of rational NARMAX models (Sec. 4.6). More complex NARMAX models are then discussed briefly in Sec. 4.7 and Sec. 4.8 concludes the chapter with a brief summary of the NARMAX class. The general class of NARMAX models was defined and discussed in a series of papers by Billings and various co-authors (Billings and Voon, 1983, 1986a; Leontartis and Billings, 1985; Zhu and Billings, 1993) and is defined by the equation . . . y(k) = F(y(k – 1), . . . , y(k – p), u(k), . . . , u(k – q), e(k – 1), . . . , e(k – r)) + e(k). (4.1) . . . Because this book is primarily concerned with the qualitative input/output behavior of these models, it will be assumed that e(k) = 0 identically as in the class of linear ARMAX models discussed in Chapter 2.

Motivations and Perspectives

This book deals with the relationship between the qualitative behavior and the mathematical structure of nonlinear, discrete-time dynamic models. The motivation for this treatment is the need for such models in computerized, model-based control of complex systems like industrial manufacturing processes or internal combustion engines. Historically, linear models have provided a solid foundation for control system design, but as control requirements become more stringent and operating ranges become wider, linear models eventually become inadequate. In such cases, nonlinear models are required, and the development of these models raises a number of important new issues. One of these issues is that of model structure selection, which manifests itself in different ways, depending on the approach taken to model development (this point is examined in some detail in Sec. 1.1). This choice is critically important since it implicitly defines the range of qualitative behavior the final model can exhibit, for better or worse. The primary objective of this book is to provide insights that will be helpful in making this model structure choice wisely. One fundamental difficulty in making this choice is the notion of nonlinearity itself: the class of “nonlinear models” is defined precisely by the crucial quality they lack. Further, since much of our intuition comes from the study of linear dynamic models (heavily exploiting this crucial quality), it is not clear how to proceed in attempting to understand nonlinear dynamic phenomena. Because these phenomena are often counterintuitive, one possible approach is to follow the lead taken in mathematics books like Counterexamples in Topology (Steen and Seebach, 1978). These books present detailed discussions of counterintuitive examples, focusing on the existence and role of certain critical working assumptions that are required for the “expected results” to hold, but that are not satisfied in the example under consideration. As a specific illustration, the Central Limit Theorem in probability theory states, roughly, that “sums of N independent random variables tend toward Gaussian limits as TV grows large.” The book Counterexamples in Probability (Stoyanov, 1987) has an entire chapter (67 pages) entitled “Limit Theorems” devoted to achieving a more precise understanding of the Central Limit Theorem and closely related theorems, and to clarifying what these theorems do and do not say.