This paper considers the problem of optimally controlling the drift of a Brownian motion with a finite set of possible drift rates so as to minimize the long-run average cost, consisting of fixed costs for changing the drift rate, processing costs for maintaining the drift rate, holding costs on the state of the process, and costs for instantaneous controls to keep the process within a prescribed range. We show that, under mild assumptions on the processing costs and the fixed costs for changing the drift rate, there is a strongly ordered optimal policy, that is, an optimal policy that limits the use of each drift rate to a single interval; when the process reaches the upper limit of that interval, the policy either changes to the next lower drift rate deterministically or resorts to instantaneous controls to keep the process within the prescribed range, and when the process reaches the lower limit of the interval, the policy either changes to the next higher drift rate deterministically or again resorts to instantaneous controls to keep the process within the prescribed range. We prove the optimality of such a policy by constructing smooth relative value functions satisfying the associated simplified optimality criteria. This paper shows that, under the proportional changeover cost assumption, each drift rate is active in at most one contiguous range and that the transitions between drift rates are strongly ordered. The results reduce the complexity of proving the optimality of such a policy by proving the existence of optimal relative value functions that constitute a nondecreasing sequence of functions. As a consequence, the constructive arguments lead to a practical procedure for solving the problem that is tens of thousands of times faster than previously reported methods.
Sequential scheduling of priority queues and Arm-acquiring bandits