On gradual-impulse control of continuous-time Markov decision processes with exponential utility

We consider a gradual-impulse control problem of continuous-time Markov decision processes, where the system performance is measured by the expectation of the exponential utility of the total cost. We show, under natural conditions on the system primitives, the existence of a deterministic stationary optimal policy out of a more general class of policies that allow multiple simultaneous impulses, randomized selection of impulses with random effects, and accumulation of jumps. After characterizing the value function using the optimality equation, we reduce the gradual-impulse control problem to an equivalent simple discrete-time Markov decision process, whose action space is the union of the sets of gradual and impulsive actions.

Infinite horizon impulse control problem with jumps and continuous switching costs

PurposeThe purpose of this paper is to show the existence results for adapted solutions of infinite horizon doubly reflected backward stochastic differential equations with jumps. These results are applied to get the existence of an optimal impulse control strategy for an infinite horizon impulse control problem.Design/methodology/approachThe main methods used to achieve the objectives of this paper are the properties of the Snell envelope which reduce the problem of impulse control to the existence of a pair of right continuous left limited processes. Some numerical results are provided to show the main results.FindingsIn this paper, the authors found the existence of a couple of processes via the notion of doubly reflected backward stochastic differential equation to prove the existence of an optimal strategy which maximizes the expected profit of a firm in an infinite horizon problem with jumps.Originality/valueIn this paper, the authors found new tools in stochastic analysis. They extend to the infinite horizon case the results of doubly reflected backward stochastic differential equations with jumps. Then the authors prove the existence of processes using Envelope Snell to find an optimal strategy of our control problem.

Optimal Trading with Online Parameter Revisions

The aim of this paper is to explain how parameters adjustments can be integrated in the design or the control of automates of trading. Typically, we are interested in the online estimation of the market impacts generated by robots or single orders, and how they/the controller should react in an optimal way to the information generated by the observation of the realized impacts. This can be formulated as an optimal impulse control problem with unknown parameters, on which a prior is given. We explain how a mix of the classical Bayesian updating rule and of optimal control techniques allows one to derive the dynamic programming equation satisfied by the corresponding value function, from which the optimal policy can be inferred. We provide an example of convergent finite difference scheme and consider typical examples of applications.

OPTIMAL DIVIDEND–REINSURANCE WITH TWO TYPES OF PREMIUM PRINCIPLES

A combined optimal dividend/reinsurance problem with two types of insurance claims, namely the expected premium principle and the variance premium principle, is discussed. Dividend payments are considered with both fixed and proportional transaction costs. The objective of an insurer is to determine an optimal dividend–reinsurance policy so as to maximize the expected total value of discounted dividend payments to shareholders up to ruin time. The problem is formulated as an optimal regular-impulse control problem. Closed-form solutions for the value function and optimal dividend–reinsurance strategy are obtained in some particular cases. Finally, some numerical analysis is given to illustrate the effects of safety loading on optimal reinsurance strategy.