Regression and Least Squares
This chapter focuses on linear regression, the process of identifying the unique model that best explains a set of observed data among a specified class of general models. Regression thus occupies a uniquely important position at the very interface of modeling and data analysis. Regression arises very often, in various guises, in handling and analyzing data. Since it is one of the most basic, useful, and frequently employed data analysis tools, and since some understanding of regression is needed in later sections, regression will be discussed in some detail. Topics covered include setting up the problem; the linear system Ax = b, least squares, special problems giving rise to linear systems, statistical issues in regression analysis, and multidimensional regression and linear model identification.