We consider two models for the control of a satellite–in the first, fuel is expended in a linear fashion to move a satellite following a diffusion–where the aim is to keep the satellite above a critical level for as long as possible (or indeed to reach a higher, ‘safe’ level). Under suitable assumptions for the drift and diffusion coefficients, it is shown that the stochastic maximum of the time to fall below the critical level is attained by a policy which imposes a reflecting boundary at the critical level until the fuel is exhausted and jumps the satellite directly to the safe level if this is ever possible. In the second model, there is a nonlinear response to the expenditure of fuel, and no safe level. It is shown that the optimal policy for maximizing the expected discounted time for the satellite to crash is similar, in that equal packets of fuel are used to jump the satellite upwards each time it reaches the critical level.
Regression-based Monte Carlo methods for stochastic control models: variable annuities with lifelong guarantees
