Solving the Hamilton-Jacobi-Bellman equation of stochastic control by a semigroup perturbation method

1984 ◽  
Vol 16 (1) ◽  
pp. 16-16
Author(s):  
Domokos Vermes
Keyword(s):  
Bellman Equation ◽  
Value Function ◽  
Sufficient Optimality Condition ◽  
Hamilton Jacobi Bellman Equation ◽  
Deterministic Processes ◽  
Random Jumps ◽  
Hamilton Jacobi Bellman ◽  
Necessary And Sufficient ◽  
The Value Function ◽  
Deterministic Trajectories

We consider the optimal control of deterministic processes with countably many (non-accumulating) random iumps. A necessary and sufficient optimality condition can be given in the form of a Hamilton-jacobi-Bellman equation which is a functionaldifferential equation with boundary conditions in the case considered. Its solution, the value function, is continuously differentiable along the deterministic trajectories if. the random jumps only are controllable and it can be represented as a supremum of smooth subsolutions in the general case, i.e. when both the deterministic motion and the random jumps are controlled (cf. the survey by M. H. A. Davis (p.14)).

scholarly journals Dynamic programming principle and associated Hamilton-Jacobi-Bellman equation for stochastic recursive control problem with non-Lipschitz aggregator

2018 ◽  
Vol 24 (1) ◽  
pp. 355-376 ◽  
Author(s):  
Jiangyan Pu ◽  
Qi Zhang
Keyword(s):  
Dynamic Programming ◽  
Control Problem ◽  
Bellman Equation ◽  
Backward Stochastic Differential Equation ◽  
Dynamic Programming Principle ◽  
Unknown Variable ◽  
Hamilton Jacobi Bellman Equation ◽  
Hamilton Jacobi Bellman ◽  
The Stability ◽  
The Value Function

In this work we study the stochastic recursive control problem, in which the aggregator (or generator) of the backward stochastic differential equation describing the running cost is continuous but not necessarily Lipschitz with respect to the first unknown variable and the control, and monotonic with respect to the first unknown variable. The dynamic programming principle and the connection between the value function and the viscosity solution of the associated Hamilton-Jacobi-Bellman equation are established in this setting by the generalized comparison theorem for backward stochastic differential equations and the stability of viscosity solutions. Finally we take the control problem of continuous-time Epstein−Zin utility with non-Lipschitz aggregator as an example to demonstrate the application of our study.

scholarly journals Optimal dividend strategies for two collaborating insurance companies

2017 ◽  
Vol 49 (2) ◽  
pp. 515-548 ◽  
Author(s):  
Hansjörg Albrecher ◽  
Pablo Azcue ◽  
Nora Muler
Keyword(s):  
Value Function ◽  
Insurance Companies ◽  
Two Dimensional ◽  
Dividend Payments ◽  
Optimal Dividend ◽  
Optimal Value ◽  
Hamilton Jacobi Bellman Equation ◽  
Hamilton Jacobi Bellman ◽  
The One ◽  
The Value Function

Abstract We consider a two-dimensional optimal dividend problem in the context of two insurance companies with compound Poisson surplus processes, who collaborate by paying each other's deficit when possible. We study the stochastic control problem of maximizing the weighted sum of expected discounted dividend payments (among all admissible dividend strategies) until ruin of both companies, by extending results of univariate optimal control theory. In the case that the dividends paid by the two companies are equally weighted, the value function of this problem compares favorably with the one of merging the two companies completely. We identify the optimal value function as the smallest viscosity supersolution of the respective Hamilton–Jacobi–Bellman equation and provide an iterative approach to approximate it numerically. Curve strategies are identified as the natural analogue of barrier strategies in this two-dimensional context. A numerical example is given for which such a curve strategy is indeed optimal among all admissible dividend strategies, and for which this collaboration mechanism also outperforms the suitably weighted optimal dividend strategies of the two stand-alone companies.

scholarly journals Optimal Expected Utility of Dividend Payments with Proportional Reinsurance under VaR Constraints and Stochastic Interest Rate

2020 ◽  
Vol 2020 ◽  
pp. 1-13
Author(s):  
Yuzhen Wen ◽  
Chuancun Yin
Keyword(s):  
Optimal Strategy ◽  
Bellman Equation ◽  
Insurance Company ◽  
Time Value ◽  
Dividend Payments ◽  
Discounted Utility ◽  
Stochastic Interest ◽  
Hamilton Jacobi Bellman Equation ◽  
Hamilton Jacobi Bellman ◽  
The Value Function

In this paper, we consider the problem of maximizing the expected discounted utility of dividend payments for an insurance company taking into account the time value of ruin. We assume the preference of the insurer is of the CRRA form. The discounting factor is modeled as a geometric Brownian motion. We introduce the VaR control levels for the insurer to control its loss in reinsurance strategies. By solving the corresponding Hamilton-Jacobi-Bellman equation, we obtain the value function and the corresponding optimal strategy. Finally, we provide some numerical examples to illustrate the results and analyze the VaR control levels on the optimal strategy.

Continuous-Time Portfolio Choice and Pricing

2017 ◽  
Author(s):  
Kerry E. Back
Keyword(s):  
Portfolio Choice ◽  
Bellman Equation ◽  
Value Function ◽  
Optimal Portfolio ◽  
Growth Optimal Portfolio ◽  
Complete Market ◽  
Investment Opportunity Set ◽  
Hamilton Jacobi Bellman Equation ◽  
Static Approach ◽  
Hamilton Jacobi Bellman

The Euler equation is defined. The static approach can be used to derive an optimal portfolio in a complete market and when the investment opportunity set is constant. In the latter case, the optimal portfolio is proportional to the growth‐optimal portfolio and two‐fund separation holds. Dynamic programming and the Hamilton‐Jacobi‐Bellman equation are explained. An optimal portfolio consists of myopic and hedging demands. The envelope condition is explained. CRRA utility implies a CRRA value function. The CCAPM and ICAPM are derived.

Numerical constructions of optimal feedback in models of chemotherapy of a malignant tumor

2019 ◽  
Vol 17 (01) ◽  
pp. 1940004 ◽  
Author(s):  
Natalia G. Novoselova
Keyword(s):  
Malignant Tumor ◽  
Value Function ◽  
Generalized Solution ◽  
Piecewise Monotone ◽  
Optimal Feedback ◽  
Hamilton Jacobi Bellman Equation ◽  
Line Of Discontinuity ◽  
Hamilton Jacobi Bellman ◽  
The Given ◽  
The Value Function

In this paper, a problem of chemotherapy of a malignant tumor is considered. Dynamics is piecewise monotone and a therapy function has two maxima. The aim of therapy is to minimize the number of tumor cells at the given final instance. The main result of this work is the construction of optimal feedbacks in the chemotherapy task. The construction of optimal feedback is based on the value function in the corresponding problem of optimal control (therapy). The value function is represented as a minimax generalized solution of the Hamilton–Jacobi–Bellman equation. It is proved that optimal feedback is a discontinuous function and the line of discontinuity satisfies the Rankin–Hugoniot conditions. Other results of the work are illustrative numerical examples of the construction of optimal feedbacks and Rankin–Hugoniot lines.

Interception of automated adversarial drone swarms in partially observed environments

2021 ◽  
pp. 1-14
Author(s):  
Daniel Saranovic ◽  
Martin Pavlovski ◽  
William Power ◽  
Ivan Stojkovic ◽  
Zoran Obradovic
Keyword(s):  
Bellman Equation ◽  
Pid Controllers ◽  
Partially Observed ◽  
Hamilton Jacobi Bellman Equation ◽  
Partial Feedback ◽  
Point Solution ◽  
Physical Limitations ◽  
Defensive Mechanisms ◽  
Hamilton Jacobi Bellman ◽  
New Framework

As the prevalence of drones increases, understanding and preparing for possible adversarial uses of drones and drone swarms is of paramount importance. Correspondingly, developing defensive mechanisms in which swarms can be used to protect against adversarial Unmanned Aerial Vehicles (UAVs) is a problem that requires further attention. Prior work on intercepting UAVs relies mostly on utilizing additional sensors or uses the Hamilton-Jacobi-Bellman equation, for which strong conditions need to be met to guarantee the existence of a saddle-point solution. To that end, this work proposes a novel interception method that utilizes the swarm’s onboard PID controllers for setting the drones’ states during interception. The drone’s states are constrained only by their physical limitations, and only partial feedback of the adversarial drone’s positions is assumed. The new framework is evaluated in a virtual environment under different environmental and model settings, using random simulations of more than 165,000 swarm flights. For certain environmental settings, our results indicate that the interception performance of larger swarms under partial observation is comparable to that of a one-drone swarm under full observation of the adversarial drone.

scholarly journals Optimal Portfolios for Financial Markets with Wishart Volatility

2013 ◽  
Vol 50 (4) ◽  
pp. 1025-1043 ◽  
Author(s):  
Nicole Bäuerle ◽  
Zejing Li
Keyword(s):  
Value Function ◽  
Numerical Study ◽  
Heston Model ◽  
Hard Part ◽  
Control Approach ◽  
Portfolio Strategy ◽  
Hamilton Jacobi Bellman ◽  
The One ◽  
Hedging Demand ◽  
The Value Function

We consider a multi asset financial market with stochastic volatility modeled by a Wishart process. This is an extension of the one-dimensional Heston model. Within this framework we study the problem of maximizing the expected utility of terminal wealth for power and logarithmic utility. We apply the usual stochastic control approach and obtain, explicitly, the optimal portfolio strategy and the value function in some parameter settings. In particular, we do this when the drift of the assets is a linear function of the volatility matrix. In this case the affine structure of the model can be exploited. In some cases we obtain a Feynman-Kac representation of the candidate value function. Though the approach we use is quite standard, the hard part is to identify when the solution of the Hamilton-Jacobi-Bellman equation is finite. This involves a couple of matrix analytic arguments. In a numerical study we discuss the influence of the investors' risk aversion on the hedging demand.

Sign in / Sign up

Export Citation Format

Share Document