Mathematical Morphology and Fourier Analysis on Time Scales

Mathematical morphology, used extensively in image processing, tracks the support domains for the operation of convolution and deconvolution. Morphology is also important in the modelling of signals on time scales. Time scale theory provides a generalization tent under which the operations of discrete and continuous time signal and Fourier analysis rest as special cases. The time scale paradigm provides modelling under which a rich class of hybrid signals and systems can be analyzed. We begin with introductory material on mathematical morphology which is foundational to the development of time scale theory. The support of convolution is related to the operation of dilation in mathematical morphology. Mathematical morphology is most commonly associated with image processing. Applications of morphology was initially applied to binary black and white images by Matheron [966]. The field is richly developed [506, 578]. Here, we outline the fundamentals. In N dimensions, let X and H denote a set of vectors or, in the degenerate case of one dimension, a set of real numbers.

Signal Recovery

The literature on the recovery of signals and images is vast (e.g., [23, 110, 112, 257, 391, 439, 791, 795, 933, 934, 937, 945, 956, 1104, 1324, 1494, 1495, 1551]). In this Chapter, the specific problem of recovering lost signal intervals from the remaining known portion of the signal is considered. Signal recovery is also a topic of Chapter 11 on POCS. To this point, sampling has been discrete. Bandlimited signals, we will show, can also be recovered from continuous samples. Our definition of continuous sampling is best presented by illustration.Asignal, f (t), is shown in Figure 10.1a, along with some possible continuous samples. Regaining f (t) from knowledge of ge(t) = f (t)Π(t/T) in Figure 10.1b is the extrapolation problem which has applications in a number of fields. In optics, for example, extrapolation in the frequency domain is termed super resolution [2, 40, 367, 444, 500, 523, 641, 720, 864, 1016, 1099, 1117]. Reconstructing f (t) from its tails [i.e., gi(t) = f (t){1 − Π(t/T)}] is the interval interpolation problem. Prediction, shown in Figure 10.1d, is the problem of recovering a signal with knowledge of that signal only for negative time. Lastly, illustrated in Figure 10.1e, is periodic continuous sampling. Here, the signal is known in sections periodically spaced at intervals of T. The duty cycle is α. Reconstruction of f (t) from this data includes a number of important reconstruction problems as special cases. (a) By keeping αT constant, we can approach the extrapolation problem by letting T go to ∞. (b) Redefine the origin in Figure 10.1e to be centered in a zero interval. Under the same assumption as (a), we can similarly approach the interpolation problem. (c) Redefine the origin as in (b). Then the interpolation problem can be solved by discarding data to make it periodically sampled. (d) Keep T constant and let α → 0. The result is reconstructing f (t) from discrete samples as discussed in Chapter 5. Indeed, this model has been used to derive the sampling theorem [246]. Figures 10.1b-e all illustrate continuously sampled versions of f (t).

Time-Frequency Representations

The Fourier transform is not particularly conducive in the illustration of the evolution of frequency with respect to time. A representation of the temporal evolution of the spectral content of a signal is referred to as a time-frequency representation (TFR). The TFR, in essence, attempts to measure the instantaneous spectrum of a dynamic signal at each point in time. Musical scores, in their most fundamental interpretation, are TFR’s. The fundamental frequency of the note is represented by the vertical location of the note on the staff. Time progresses as we read notes from left to right. The musical score shown in Figure 9.1 is an example. Temporal assignment is given by the note types. The 120 next to the quarter note indicates the piece should be played at 120 beats per minute. Thus, the duration of a quarter note is one half second. The frequency of the A above middle C is, by international standards, 440 Hertz. Adjacent notes notes have a ratio of 21/12. The note, A#, for example, has a frequency of 440 × 21/12 = 466.1637615 Hertz. Middle C, nine half tones (a.k.a. semitones or chromatic steps) below A, has a frequency of 440 × 2−9/12 = 261.6255653 Hertz. The interval of an octave doubles the frequency. The frequency of an octave above A is twelve half tones, or, 440 × 212/12 = 880 Hertz. The frequency spacings in the time-frequency representation of musical scores such as Figure 9.1 are thus logarithmic. This is made more clear in the alternate representation of the musical score in Figure 9.2 where time is on the horizontal axis and frequency on the vertical. At every point in time where there is no rest, a frequency is assigned. To make chords, numerous frequencies can be assigned to a point in time. Further discussion of the technical theory of western harmony is in Section 13.1.

Fourier Transforms in Probability, Random Variables and Stochastic Processes

In this Chapter, we present application of Fourier analysis to probability, random variables and stochastic processes [1089, 1097, 1387, 1329]. Arandom variable, X, is the assignment of a number to the outcome of a random experiment. We can, for example, flip a coin and assign an outcome of a heads as X = 1 and a tails X = 0. Often the number is equated to the numerical outcome of the experiment, such as the number of dots on the face of a rolled die or the measurement of a voltage in a noisy circuit. The cumulative distribution function is defined by FX(x) = Pr[X ≤ x]. (4.1) The probability density function is the derivative fX(x) = d /dxFX(x). Our treatment of random variables focuses on use of Fourier analysis. Due to this viewpoint, the development we use is unconventional and begins immediately in the next section with discussion of properties of the probability density function.

Fourier Analysis in Systems Theory

In the most general sense, any process wherein a stimulus generates a corresponding response can be dubbed a system. For a temporal system with single input, f (t), and single output, g(t), the relation can be written as . . . g(t) = S{ f (t)} (3.1) . . . where S{·} is the system operator. This is illustrated in Figure 3.1. There exist numerous system types. We define them here in terms of continuous signals. The equivalents in discrete time are given as an exercise. For homogeneous systems, amplifying or attenuating the input likewise amplifying or attenuating the output. For any constant, a,. . . S{a f(t)} = aS{ f (t)} (3.2) If the response of the sum is the sum of the responses, the system is said to be additive. Specifically,. . . S{ f1(t) + f2(t)} = S{ f1(t)} + S{ f2(t)} (3.3) . . . Systems that are both homogeneous and additive are said to be linear. The criteria in (3.2) and (3.3) can be combined into a single necessary and sufficient condition for linearity.. . . S{a f1(t) + bf2(t)} = aS{ f1(t)} + bS{ f2(t)} (3.4) . . . where a and b are constants. All linear systems produce a zero output when the input is zero. . . . S{0} = 0. (3.5). . . To show this, we use (3.4) with a = −b and f1(t) = f2(t). Note that, because of (3.5), the system defined by . . . g(t) = b f(t) + c . . . where b and c¹ 0 are constants, is not linear. It is not homogeneous since . . . S{a f} = b f + c ≠aS{ f} = a (b f + c) .

Signal and Image Synthesis: Alternating Projections Onto Convex Sets

Alternating projections onto convex sets (POCS) [319, 918, 1324, 1333] is a powerful tool for signal and image restoration and synthesis. The desirable properties of a reconstructed signal may be defined by a convex set of constraint parameters. Iteratively projecting onto these convex constraint sets can result in a signal which contains all desired properties. Convex signal sets are frequently encountered in practice and include the sets of bandlimited signals, duration limited signals, causal signals, signals that are the same (e.g., zero) on some given interval, bounded signals, signals of a given area and complex signals with a specified phase. POCS was initially introduced by Bregman [156] and Gubin et al. [558] and was later popularized by Youla & Webb [1550] and Sezan & Stark [1253]. POCS has been applied to such topics as acoustics [300, 1381], beamforming [426], bioinformatics [484], cellular radio control [1148], communications systems [29, 769, 1433], deconvolution and extrapolation [718, 907, 1216], diffraction [421], geophysics [4], image compression [1091, 1473], image processing [311, 321, 470, 471, 672, 736, 834, 1065, 1069, 1093, 1473, 1535, 1547, 1596], holography [880, 1381], interpolation [358, 559, 1266], neural networks [1254, 1543, 909, 913, 1039], pattern recognition [1444, 1588], optimization [598, 1359, 1435], radiotherapy [298, 814, 1385], remote sensing [1223], robotics [740], sampling theory [399, 1334, 1542], signal recovery [320, 737, 1104, 1428, 1594], speech processing [1450], superresolution [399, 633, 654, 834, 1393, 1521], television [736, 786], time-frequency analysis [1037, 1043], tomography [1103, 713, 1212, 1213, 1275, 916, 1322, 1060, 1040], video processing [560, 786, 1092], and watermarking [19, 1470]. Although signal processing applications ofPOCS use sets of signals,POCSis best visualized viewing the operations on sets of points. In this section, POCS is introduced geometrically in two and three dimensions. Such visualization of POCS is invaluable in application of the theory.

Generalizations of the Sampling Theorem

There have been numerous interesting and useful generalizations of the sampling theorem. Some are straightforward variations on the fundamental cardinal series. Oversampling, for example, results in dependent samples and allows much greater flexibility in the choice of interpolation functions. In Chapter 7, we will see that it can also result in better performance in the presence of sample data noise. Bandlimited signal restoration from samples of various filtered versions of the signal is the topic addressed in Papoulis’ generalization [1086, 1087] of the sampling theorem. Included as special cases are recurrent nonuniform sampling and simultaneously sampling a signal and one or more of its derivatives. Kramer [772] generalized the sampling theorem to signals that were bandlimited in other than the Fourier sense. We also demonstrate that the cardinal series is a special case of Lagrangian polynomial interpolation. Sampling in two or more dimensions is the topic of Section 8.9. There are a number of functions other than the sinc which can be used to weight a signal’s samples in such a manner as to uniquely characterize the signal. Use of these generalized interpolation functions allows greater flexibility in dealing with sampling theorem type characterizations. If a bandlimited signal has bandwidth B, then it can also be considered to have bandwidthW ≥ B.

Introduction

Jean Baptiste Joseph Fourier’s powerful idea of decomposition of a signal into sinusoidal components has found application in almost every engineering and science field. An incomplete list includes acoustics [1497], array imaging [1304], audio [1290], biology [826], biomedical engineering [1109], chemistry [438, 925], chromatography [1481], communications engineering [968], control theory [764], crystallography [316, 498, 499, 716], electromagnetics [250], imaging [151], image processing [1239] including segmentation [1448], nuclear magnetic resonance (NMR) [436, 1009], optics [492, 514, 517, 1344], polymer characterization [647], physics [262], radar [154, 1510], remote sensing [84], signal processing [41, 154], structural analysis [384], spectroscopy [84, 267, 724, 1220, 1293, 1481, 1496], time series [124], velocity measurement [1448], tomography [93, 1241, 1242, 1327, 1330, 1325, 1331], weather analysis [456], and X-ray diffraction [1378], Jean Baptiste Joseph Fourier’s last name has become an adjective in the terms like Fourier series [395], Fourier transform [41, 51, 149, 154, 160, 437, 447, 926, 968, 1009, 1496], Fourier analysis [151, 379, 606, 796, 1472, 1591], Fourier theory [1485], the Fourier integral [395, 187, 1399], Fourier inversion [1325], Fourier descriptors [826], Fourier coefficients [134], Fourier spectra [624, 625] Fourier reconstruction [1330], Fourier spectrometry [84, 355], Fourier spectroscopy [1220, 1293, 1438], Fourier array imaging [1304], Fourier transform nuclear magnetic resonance (NMR) [429, 1004], Fourier vision [1448], Fourier optics [419, 517, 1343], and Fourier acoustics [1496]. Applied Fourier analysis is ubiquitous simply because of the utility of its descriptive power. It is second only to the differential equation in the modelling of physical phenomena. In contrast with other linear transforms, the Fourier transform has a number of physical manifestations. Here is a short list of everyday occurrences as seen through the lens of the Fourier paradigm. • Diffracting coherent waves in sonar and optics in the far field are given by the two dimensional Fourier transform of the diffracting aperture. Remarkably, in free space, the physics of spreading light naturally forms a two dimensional Fourier transform. • The sampling theorem, born of Fourier analysis, tells us how fast to sample an audio waveform to make a discrete time CD or an image to make a DVD.

Applications

Fourier discovered the Fourier series as a solution to a boundary value problem [33, 303, 512, 620] related to the heat wave equation. Fourier’s work on heat is still in print [455]. In this section, we derive the wave equation for the vibrating string and show how the Fourier series is used in its solution. The solution, in turn, gives rise to the physics of harmonics used as the foundation of music harmony. We contrast the natural harmony of the overtones to that available from the tempered scale of western music. The tempered scale is able to accurately approximate the beauty of natural harmony using a uniformly calibrated frequency scale. The wave equation is manifest in analysis of physical phenomena that display wave like properties. This includes electromagnetic waves, heat waves, and acoustic waves. We consider the case of the simple vibrating string. A string under horizontal tension T is subjected to a small vertical displacement, y = y(x, t), that is a function of time, t, and location, x. As illustrated in Figure 13.1, attention is focused on an incremental piece of the string from x to x +Dx. Under the small displacement assumption, there is no movement of the string horizontally (i.e., in the x direction), and the horizontal forces must sum to zero. T = T1 cos(θ1) = T2 cos(θ2). Let the linear mass density (i.e., mass per unit length) of the string be ρ. The mass of the incremental piece of string is then ρ. The total vertical force acting on the string is T2 cos(θ2) − T1 cos(θ1).

Multidimensional Signal Analysis

N dimensional signals are characterized as values in an N dimensional space. Each point in the space is assigned a value, possibly complex. Each dimension in the space can be discrete, continuous, or on a time scale. A black and white movie can be modelled as a three dimensional signal.Acolor picture can be modelled as three signals in two dimensions, one each, for example, for red, green and blue. This chapter explores Fourier characterization of different types of multidimensional signals and corresponding applications. Some signal characterizations are straightforward extensions of their one dimensional counterparts. Others, even in two dimensions, have properties not found in one dimensional signals. We are fortunate to be able to visualize structures in two, three, and sometimes four dimensions. It assists in the intuitive generalization of properties to higher dimensions. Fourier characterization of multidimensional signals allows straightforward modelling of reconstruction of images from their tomographic projections. Doing so is the foundation of certain medical and industrial imaging, including CAT (for computed axial tomography) scans. Multidimensional Fourier series are based on models found in nature in periodically replicated crystal Bravais lattices [987, 1188]. As is one dimension, the Fourier series components can be found from sampling the Fourier transform of a single period of the periodic signal. The multidimensional cosine transform, a relative of the Fourier transform, is used in image compression such as JPG images. Multidimensional signals can be filtered. The McClellan transform is a powerful method for the design of multidimensional filters, including generalization of the large catalog of zero phase one dimensional FIR filters into higher dimensions. As in one dimension, the multidimensional sampling theorem is the Fourier dual of the Fourier series. Unlike one dimension, sampling can be performed at the Nyquist density with a resulting dependency among sample values. This property can be used to reduce the sampling density of certain images below that of Nyquist, or to restore lost samples from those remaining. Multidimensional signal and image analysis is also the topic of Chapter 9 on time frequency representations, and Chapter 11 where POCS is applied signals in higher dimensions.