Tracking: B

In Chapter Tracking: A, we explained the estimation of a random variable based on observations. We also described the Kalman filter and we gave a number of examples. In this chapter, we derive the Kalman filter and explain some of its properties. We also discuss the extended Kalman filter.Section 10.1 explains how to update an estimate as one makes additional observations. Section 10.2 derives the Kalman filter. The properties of the Kalman filter are explained in Sect. 10.3. Section 10.4 shows how the Kalman filter is extended to nonlinear systems.

Digital Link—A

In a digital link, a transmitter converts bits into signals and a receiver converts the signals it receives into bits. The receiver faces a decision problem that we study in Sect. 7.1. The main tool is Bayes’ Rule. The key notions are maximum a posteriori and maximum likelihood estimates. Transmission systems use codes to reduce the number of bits they need to transmit. Section 7.2 explains the Huffman codes that minimize the expected number of bits needed to transmit symbols; the idea is to use fewer bits for more likely symbols. Section 7.3 explores a commonly used model of a communication channel: the binary symmetric channel. It explains how to calculate the probability of errors. Section 7.4 studies a more complex modulation scheme employed by most smartphones and computers: QAM. Section 7.5 is devoted to a central problem in decision making: how to infer which situation is in force from observations. Does a test reveal the presence of a disease; how to balance the probability of false positive and that of false negative? The main result of that section is the Neyman–Pearson Theorem that the section illustrates with many examples.

Route Planning: B

There is a class of control problems that admit a particularly elegant solution: the linear quadratic Gaussian (LQG) problems. In these problems, the state dynamics and observations are linear, the cost is quadratic, and the noise is Gaussian. Section 14.1 explains the theory of LQG problems when one observes the state. Section 14.2 discusses the situation when the observations are noisy and shows the remarkable certainty equivalence property of the solution. Section 14.3 explains how noisy observations affect Markov decision problems.

Tracking—A

This chapter explains how to estimate an unobserved random variable or vector from available observations. This problem arises in many examples, as illustrated in Sect. 9.1. The basic problem is defined in Sect. 9.2. One commonly used approach is the linear least squares estimate explained in Sect. 9.3. A related notion is the linear regression covered in Sect. 9.4. Section 9.5 comments on the problem of overfitting. Sections 9.6 and 9.7 explain the minimum means squares estimate that may be a nonlinear function of the observations and the remarkable fact that it is linear for jointly Gaussian random variables. Section 9.8 is devoted to the Kalman filter, which is a recursive algorithm for calculating the linear least squares estimate of the state of a system given previous observations.

PageRank: A

Application: Ranking the most relevant pages in web searchTopics: Finite Discrete Time Markov Chains, SLLN

Networks: A

Social networks connect people and enable them to exchange information. News and rumors spread through these networks. We explore models of such propagations. The technology behind social networks is the internet where packets travel from queue to queue. We explain some key results about queueing networks.Section 5.1 explores a model of how rumors spread in a social network. Epidemiologists use similar models to study the spread of viruses. Section 5.2 explains the cascade of choices in a social network where one person’s choice is influenced by those of people she knows. Section 5.3 shows how seeding the market with advertising or free products affects adoptions. Section 5.4 studies a model of how media can influence the eventual consensus in a social network. Section 5.5 explores the randomness of the consensus in a group. Sections 5.6 and 5.7 present a different class of network models where customers queue for service. Section 5.6 studies a single queue and Sect. 5.7 analyzes a network of queues. Section 5.8 explains a classical optimization problem in a communication network: how to choose the capacities of different links. Section 5.9 discusses the suitability of queueing networks as models of the internet. Section 5.10 presents a classical result about a class of queueing networks known as product-form networks.

Speech Recognition: B

Online learning algorithms update their estimates as additional observations are made. Section 12.1 explains a simple example: online linear regression. The stochastic gradient projection algorithm is a general technique to update estimates based on additional observations; it is widely used in machine learning. Section 12.2 presents the theory behind that algorithm. When analyzing large amounts of data, one faces the problems of identifying the most relevant data and of how to use efficiently the available data. Section 12.3 explains three examples of how these questions are addressed: the LASSO algorithm, compressed sensing, and the matrix completion problem. Section 12.4 discusses deep neural networks for which the stochastic gradient projection algorithm is easy to implement.

Route Planning: A

This chapter is concerned with making successive decisions in the presence of uncertainty. The decisions affect the cost at each step but also the “state” of the system. We start with a simple example: choosing a route with uncertain travel times. We then examine a more general model: controlling a Markov chain.Section 13.1 presents a model of route section when the travel times are random. Section 13.2 shows one formulation where one plans the trip long in advance. Section 13.3 explains how the problem changes if one is able to adjust the route based on real-time information. That section introduces the main ideas of stochastic dynamic programming. Section 13.4 discusses a generalization of the route planning problem: a Markov decision problem. Section 13.5 solves the problem when the horizon is infinite.

Multiplexing: A

This chapter explores the fluctuations of random variables away from their mean value. You flip a fair coin 100 times. How likely is it that you get 60 heads? Conversely, if you get 60 heads, how likely is it that the coin is fair? Such questions are fundamental in extracting information from data.In Sect. 3.1, we start by exploring the rate available to a user when a random number of them share a link, as illustrated in Sect. 3.1. Such calculations are central to network provisioning. The main analytical tool is the Central Limit Theorem explained in Sect. 3.2 where Gaussian random variables are also introduced and confidence intervals are defined. To share a common link, devices may be attached to a switch. For instance, the desktop computers in a building are typically connected to a switch that then sends the data to a common high-speed link. We explore the delays that packets face through the buffer of a switch in Sect. 3.3. The analysis uses a Markov chain model of the buffer. To share a wireless radio channel, devices use a multiple access protocol that regulates the transmissions. We study such schemes in Sect. 3.4. We use probabilistic models of the protocols.

Speech Recognition: A

Speech recognition can be formulated as the problem of guessing a sequence of words that produces a sequence of sounds. The human brain is remarkably good at solving this problem, even though the same words correspond to many different sounds, because of accents or characteristics of the voice. Moreover, the environment is always noisy, to that the listeners hear a corrupted version of the speech.Computers are getting much better at speech recognition and voice command systems are now common for smartphones (Siri), automobiles (GPS, music, and climate control), call centers, and dictation systems. In this chapter, we explain the main ideas behind the algorithms for speech recognition and for related applications.The starting point is a model of the random sequence (e.g., words) to be recognized and of how this sequence is related to the observation (e.g., voice). The main model is called a hidden Markov chain. The idea is that the successive parts of speech form a Markov chain and that each word maps randomly to some sounds. The same model is used to decode strings of symbols in communication systems.Section 11.1 is a general discussion of learning. The hidden Markov chain model used in speech recognition and in error decoding is introduced in Sect. 11.2. That section explains the Viterbi algorithm. Section 11.3 discusses expectation maximization and clustering algorithms. Section 11.4 covers learning for hidden Markov chains.