14.—Dynamic Programming applied to Some Non-linear Stochastic Control Systems

SynopsisOptimal switching curves are investigated for some second-order control systems with random disturbances, saturating control, and a mean-square-error performance index.It is first shown that the optimal switching curves are asymptotic at infinity to the optimal switching curves for corresponding deterministic systems.The systems are then discretised in time and state, and ordinary dynamic programming is applied to the resulting Markov chain models. The discretisation techniques are described in some detail.The same techniques are applied to a deterministic system for which the optimal switching curve is known, and it is found that the resulting discretisation errors are considerable.Thus it turns out that ordinary discrete dynamic programming gives only a rough approximation to the optimal switching curve. It seems desirable to tackle the equations of continuous dynamic programming by techniques more refined than crude discretisation. One such technique is suggested.