Strategic Interactions and Uncertainty in Decisions to Curb Greenhouse Gas Emissions

This paper examines the strategic interactions of two large regions making choices about greenhouse gas emissions in the face of rising global temperatures. Three central features are highlighted: uncertainty, the incentive for free riding, and asymmetric characteristics of decision makers. Optimal decisions are modelled in a fully dynamic, feedback Stackelberg pollution game. Global average temperature is modelled as a mean reverting stochastic process. A numerical solution of a coupled system of Hamilton-Jacobi-Bellman equations is implemented and the probability distribution of outcomes is illustrated with Monte Carlo simulation. When players are identical, the outcome of the game is much worse than the social planner’s outcome. An increase in temperature volatility reduces player utility, making cooperative action through a social planner more urgent. Asymmetric damages or asymmetric preferences for emissions reductions are shown to have important effffects on the strategic interactions of players.

Utility Indifference Option Pricing Model with a Non-Constant Risk-Aversion under Transaction Costs and Its Numerical Approximation

Our goal is to analyze the system of Hamilton-Jacobi-Bellman equations arising in derivative securities pricing models. The European style of an option price is constructed as a difference of the certainty equivalents to the value functions solving the system of HJB equations. We introduce the transformation method for solving the penalized nonlinear partial differential equation. The transformed equation involves possibly non-constant the risk aversion function containing the negative ratio between the second and first derivatives of the utility function. Using comparison principles we derive useful bounds on the option price. We also propose a finite difference numerical discretization scheme with some computational examples.

Solving high-dimensional Hamilton–Jacobi–Bellman PDEs using neural networks: perspectives from the theory of controlled diffusions and measures on path space

Optimal control of diffusion processes is intimately connected to the problem of solving certain Hamilton–Jacobi–Bellman equations. Building on recent machine learning inspired approaches towards high-dimensional PDEs, we investigate the potential of iterative diffusion optimisation techniques, in particular considering applications in importance sampling and rare event simulation, and focusing on problems without diffusion control, with linearly controlled drift and running costs that depend quadratically on the control. More generally, our methods apply to nonlinear parabolic PDEs with a certain shift invariance. The choice of an appropriate loss function being a central element in the algorithmic design, we develop a principled framework based on divergences between path measures, encompassing various existing methods. Motivated by connections to forward-backward SDEs, we propose and study the novel log-variance divergence, showing favourable properties of corresponding Monte Carlo estimators. The promise of the developed approach is exemplified by a range of high-dimensional and metastable numerical examples.

Guaranteed Deterministic Approach to Superhedging: Case of Binary European Option

For the superreplication problem with discrete time, a guaranteed deterministic formulation is considered: the problem is to guarantee coverage of the contingent liability on sold option under all admissible scenarios. These scenarios are defined by means of a priori defined compacts dependent on price prehistory: the price increments at each point in time must lie in the corresponding compacts. In a general case, we consider a market with trading constraints and assume the absence of transaction costs. The formulation of the problem is game theoretic and leads to the Bellman–Isaacs equations. This paper analyses the solution to these equations for a specific pricing problem, i.e., for a binary option of the European type, within a multiplicative market model, with no trading constraints. A number of solution properties and an algorithm for the numerical solution of the Bellman equations are derived. The interest in this problem, from a mathematical prospective, is related to the discontinuity of the option payoff function.

Stability and convergence of second order backward differentiation schemes for parabolic Hamilton–Jacobi–Bellman equations

We study a second order Backward Differentiation Formula (BDF) scheme for the numerical approximation of linear parabolic equations and nonlinear Hamilton–Jacobi–Bellman (HJB) equations. The lack of monotonicity of the BDF scheme prevents the use of well-known convergence results for solutions in the viscosity sense. We first consider one-dimensional uniformly parabolic equations and prove stability with respect to perturbations, in the $$L^2$$ L 2 norm for linear and semi-linear equations, and in the $$H^1$$ H 1 norm for fully nonlinear equations of HJB and Isaacs type. These results are then extended to two-dimensional semi-linear equations and linear equations with possible degeneracy. From these stability results we deduce error estimates in $$L^2$$ L 2 norm for classical solutions to uniformly parabolic semi-linear HJB equations, with an order that depends on their Hölder regularity, while full second order is recovered in the smooth case. Numerical tests for the Eikonal equation and a controlled diffusion equation illustrate the practical accuracy of the scheme in different norms.

Forward and Backward Bellman Equations Improve the Efficiency of the EM Algorithm for DEC-POMDP

Decentralized partially observable Markov decision process (DEC-POMDP) models sequential decision making problems by a team of agents. Since the planning of DEC-POMDP can be interpreted as the maximum likelihood estimation for the latent variable model, DEC-POMDP can be solved by the EM algorithm. However, in EM for DEC-POMDP, the forward–backward algorithm needs to be calculated up to the infinite horizon, which impairs the computational efficiency. In this paper, we propose the Bellman EM algorithm (BEM) and the modified Bellman EM algorithm (MBEM) by introducing the forward and backward Bellman equations into EM. BEM can be more efficient than EM because BEM calculates the forward and backward Bellman equations instead of the forward–backward algorithm up to the infinite horizon. However, BEM cannot always be more efficient than EM when the size of problems is large because BEM calculates an inverse matrix. We circumvent this shortcoming in MBEM by calculating the forward and backward Bellman equations without the inverse matrix. Our numerical experiments demonstrate that the convergence of MBEM is faster than that of EM.

Policy iteration for Hamilton–Jacobi–Bellman equations with control constraints

Policy iteration is a widely used technique to solve the Hamilton Jacobi Bellman (HJB) equation, which arises from nonlinear optimal feedback control theory. Its convergence analysis has attracted much attention in the unconstrained case. Here we analyze the case with control constraints both for the HJB equations which arise in deterministic and in stochastic control cases. The linear equations in each iteration step are solved by an implicit upwind scheme. Numerical examples are conducted to solve the HJB equation with control constraints and comparisons are shown with the unconstrained cases.