scholarly journals Some Results on Bellman Equations of Optimal Production Control in a Stochastic Manufacturing System

2009 ◽  
Vol 2009 ◽  
pp. 1-23
Author(s):  
Azizul Baten ◽  
Anton Abdulbasah Kamil
Keyword(s):  
Production Cost ◽  
Manufacturing System ◽  
Production Control ◽  
Inventory Problem ◽  
Present Value ◽  
Hjb Equations ◽  
Bellman Equations ◽  
Production Inventory ◽  
Production Cost Control ◽  
Hamilton Jacobi Bellman

The paper studies the production inventory problem of minimizing the expected discounted present value of production cost control in a manufacturing system with degenerate stochastic demand. We establish the existence of a unique solution of the Hamilton-Jacobi-Bellman (HJB) equations associated with this problem. The optimal control is given by a solution to the corresponding HJB equation.

Optimal Production Control in Stochastic Manufacturing Systems with Degenerate Demand

2011 ◽  
Vol 1 (1) ◽  
pp. 89-96 ◽  
Author(s):  
Azizul Baten ◽  
Anton Abdulbasah Kamil
Keyword(s):  
Control Problem ◽  
Manufacturing Systems ◽  
Production Control ◽  
Hjb Equation ◽  
Dynamic Programming Principle ◽  
Inventory Problem ◽  
Production Inventory ◽  
Production Cost Control ◽  
Inventory Control Problem ◽  
Ito's Lemma

AbstractThe paper studies the production inventory problem of minimizing the expected discounted present value of production cost control in manufacturing systems with degenerate stochastic demand. We have developed the optimal inventory production control problem by deriving the dynamics of the inventory-demand ratio that evolves according to a stochastic neoclassical differential equation through Ito's Lemma. We have also established the Riccati based solution of the reduced (one- dimensional) HJB equation corresponding to production inventory control problem through the technique of dynamic programming principle. Finally, the optimal control is shown to exist from the optimality conditions in the HJB equation.

scholarly journals Policy iteration for Hamilton–Jacobi–Bellman equations with control constraints

2021 ◽  
Author(s):  
Sudeep Kundu ◽  
Karl Kunisch
Keyword(s):  
Linear Equations ◽  
Hjb Equation ◽  
Policy Iteration ◽  
Optimal Feedback Control ◽  
Iteration Step ◽  
Control Constraints ◽  
Hjb Equations ◽  
Bellman Equations ◽  
Hamilton Jacobi Bellman ◽  
Feedback Control Theory

AbstractPolicy iteration is a widely used technique to solve the Hamilton Jacobi Bellman (HJB) equation, which arises from nonlinear optimal feedback control theory. Its convergence analysis has attracted much attention in the unconstrained case. Here we analyze the case with control constraints both for the HJB equations which arise in deterministic and in stochastic control cases. The linear equations in each iteration step are solved by an implicit upwind scheme. Numerical examples are conducted to solve the HJB equation with control constraints and comparisons are shown with the unconstrained cases.

scholarly journals Integrated Control Policy for a Multiple Machines and Multiple Product Types Manufacturing System Production Process with Uncertain Fault

Processes ◽  
2020 ◽  
Vol 8 (8) ◽  
pp. 952
Author(s):  
Jia You ◽  
Ming Li ◽  
Kai Guo ◽  
Hao Li
Keyword(s):  
Production Cost ◽  
Manufacturing Systems ◽  
Manufacturing System ◽  
Cost Control ◽  
Integrated Control ◽  
Control Policy ◽  
Point Control ◽  
Multiple Product ◽  
Production Cost Control ◽  
Multiple Machines

The optimization of production cost has always been a key issue in manufacturing systems; for the single product type manufacturing systems, lots of research studies have proved the validity of the hedging point control policy in production cost control. However, due to the complexity of the multiple machines and multiple product types manufacturing systems with uncertain fault, it is difficult to achieve a good control effect only by using the hedging point control policy. To optimize the total production cost under constantly changing demands, an integrated control policy that combines the prioritized hedging point (PHP) control policy with the production capacity planning during production is proposed, and the decision variables are obtained by a particle swarm optimization (PSO) algorithm. The simulation experiments show the effectiveness of the proposed integrated control policy in production cost control for the multiple machines and multiple product types manufacturing system.

scholarly journals Stability and convergence of second order backward differentiation schemes for parabolic Hamilton–Jacobi–Bellman equations

2021 ◽  
Author(s):  
Olivier Bokanowski ◽  
Athena Picarelli ◽  
Christoph Reisinger
Keyword(s):  
Parabolic Equations ◽  
Linear Equations ◽  
Second Order ◽  
Stability And Convergence ◽  
Classical Solutions ◽  
Hjb Equations ◽  
Bellman Equations ◽  
Linear Parabolic Equations ◽  
Viscosity Sense ◽  
Hamilton Jacobi Bellman

AbstractWe study a second order Backward Differentiation Formula (BDF) scheme for the numerical approximation of linear parabolic equations and nonlinear Hamilton–Jacobi–Bellman (HJB) equations. The lack of monotonicity of the BDF scheme prevents the use of well-known convergence results for solutions in the viscosity sense. We first consider one-dimensional uniformly parabolic equations and prove stability with respect to perturbations, in the $$L^2$$ L 2 norm for linear and semi-linear equations, and in the $$H^1$$ H 1 norm for fully nonlinear equations of HJB and Isaacs type. These results are then extended to two-dimensional semi-linear equations and linear equations with possible degeneracy. From these stability results we deduce error estimates in $$L^2$$ L 2 norm for classical solutions to uniformly parabolic semi-linear HJB equations, with an order that depends on their Hölder regularity, while full second order is recovered in the smooth case. Numerical tests for the Eikonal equation and a controlled diffusion equation illustrate the practical accuracy of the scheme in different norms.

scholarly journals Finite Difference Methods for the Hamilton–Jacobi–Bellman Equations Arising in Regime Switching Utility Maximization

2020 ◽  
Vol 85 (3) ◽  
Author(s):  
Jingtang Ma ◽  
Jianjun Ma
Keyword(s):  
Finite Difference ◽  
Regime Switching ◽  
Utility Maximization ◽  
Finite Difference Methods ◽  
Hjb Equations ◽  
Functional Operator ◽  
Bellman Equations ◽  
Numerical Comparisons ◽  
Difference Methods ◽  
Hamilton Jacobi Bellman

AbstractFor solving the regime switching utility maximization, Fu et al. (Eur J Oper Res 233:184–192, 2014) derive a framework that reduce the coupled Hamilton–Jacobi–Bellman (HJB) equations into a sequence of decoupled HJB equations through introducing a functional operator. The aim of this paper is to develop the iterative finite difference methods (FDMs) with iteration policy to the sequence of decoupled HJB equations derived by Fu et al. (2014). The convergence of the approach is proved and in the proof a number of difficulties are overcome, which are caused by the errors from the iterative FDMs and the policy iterations. Numerical comparisons are made to show that it takes less time to solve the sequence of decoupled HJB equations than the coupled ones.

scholarly journals Utility Indifference Option Pricing Model with a Non-Constant Risk-Aversion under Transaction Costs and Its Numerical Approximation

2021 ◽  
Vol 14 (9) ◽  
pp. 399
Author(s):  
Pedro Pólvora ◽  
Daniel Ševčovič
Keyword(s):  
Risk Aversion ◽  
Nonlinear Partial Differential Equation ◽  
Option Price ◽  
Transformation Method ◽  
Value Functions ◽  
Hjb Equations ◽  
Bellman Equations ◽  
Option Pricing Model ◽  
Utility Indifference ◽  
Hamilton Jacobi Bellman

Our goal is to analyze the system of Hamilton-Jacobi-Bellman equations arising in derivative securities pricing models. The European style of an option price is constructed as a difference of the certainty equivalents to the value functions solving the system of HJB equations. We introduce the transformation method for solving the penalized nonlinear partial differential equation. The transformed equation involves possibly non-constant the risk aversion function containing the negative ratio between the second and first derivatives of the utility function. Using comparison principles we derive useful bounds on the option price. We also propose a finite difference numerical discretization scheme with some computational examples.

General fully coupled FBSDES involving the value function and related nonlocal HJB equations combined with algebraic equations

2020 ◽  
pp. 2150032
Author(s):  
Tao Hao ◽  
Qingfeng Zhu
Keyword(s):  
Differential Equations ◽  
Stochastic Differential Equations ◽  
Value Function ◽  
Dynamic Programming Principle ◽  
Algebraic Equations ◽  
Hjb Equations ◽  
Bellman Equations ◽  
Hamilton Jacobi Bellman ◽  
Fully Coupled ◽  
The Value Function

Recently, Hao and Li [Fully coupled forward-backward SDEs involving the value function. Nonlocal Hamilton–Jacobi–Bellman equations, ESAIM: Control Optim, Calc. Var. 22(2016) 519–538] studied a new kind of forward-backward stochastic differential equations (FBSDEs), namely the fully coupled FBSDEs involving the value function in the case where the diffusion coefficient [Formula: see text] in forward stochastic differential equations depends on control, but does not depend on [Formula: see text]. In our paper, we generalize their work to the case where [Formula: see text] depends on both control and [Formula: see text], which is called the general fully coupled FBSDEs involving the value function. The existence and uniqueness theorem of this kind of equations under suitable assumptions is proved. After obtaining the dynamic programming principle for the value function [Formula: see text], we prove that the value function [Formula: see text] is the minimum viscosity solution of the related nonlocal Hamilton–Jacobi–Bellman equation combined with an algebraic equation.

scholarly journals Junction conditions for finite horizon optimal control problems on multi-domains with continuous and discontinuous solutions

2019 ◽  
Vol 25 ◽  
pp. 79
Author(s):  
Daria Ghilli ◽  
Zhiping Rao ◽  
Hasnaa Zidani
Keyword(s):  
Finite Horizon ◽  
Local Controllability ◽  
Continuous Case ◽  
Control Problems ◽  
Junction Conditions ◽  
Discontinuous Solutions ◽  
Hjb Equations ◽  
Bellman Equations ◽  
Lower Semi ◽  
Hamilton Jacobi Bellman

This paper deals with junction conditions for Hamilton–Jacobi–Bellman (HJB) equations for finite horizon control problems on multi-domains. We consider two different cases where the final cost is continuous or lower semi-continuous. In the continuous case, we extend the results in Z. Rao and H. Zidani, Hamilton-Jacobi-Bellman equations on multi-domains, in Control and Optimization with PDE Constraints, Vol. 164 of International Series of Numerical Mathematics. Birkhäuser, Basel (2013) 93–116. in a more general framework with switching running costs and weaker controllability assumptions. The comparison principle has been established to guarantee the uniqueness and the stability results for the HJB system on such multi-domains. In the lower semi-continuous case, we characterize the value function as the unique lower semi-continuous viscosity solution of the HJB system, under a local controllability assumption.

Maintenance and setup planning in manufacturing systems under uncertainties

2018 ◽  
Vol 24 (2) ◽  
pp. 170-184 ◽  
Author(s):  
Guy Richard Kibouka ◽  
Donatien Nganga-Kouya ◽  
Jean-Pierre Kenné ◽  
Vladimir Polotski ◽  
Victor Songmene
Keyword(s):  
Manufacturing Systems ◽  
Manufacturing System ◽  
System Capacity ◽  
Operational Mode ◽  
Hjb Equations ◽  
Optimal Production ◽  
Control Laws ◽  
Content Type ◽  
Industrial Manufacturing ◽  
Hamilton Jacobi Bellman

PurposeThe purpose of this paper is to find the optimal production and setup policies for a manufacturing system that produces two different types of parts. The manufacturing system consists of one machine subject to random failures and repairs. Reconfiguring the machine to switch production from one type of product to another generates a non-production time and a significant cost.Design/methodology/approachThis paper proposes an approach based on the development of optimal production and setup policies, taking into account the possibilities of undertaking the setup for all modes of the machine, and covering them at the end of setup. New optimality conditions are developed in terms of modified Hamilton-Jacobi-Bellman (HJB) equations and recursive numerical methods are applied to solve such equations.FindingsThe proposed approach led to determine more realistic production rates of both parts and setup sequences for the different modes of the machine that significantly influence the inventory and the system capacity. A numerical example and sensitivity analysis are used to determine the structure of the optimal policies and to show the helpfulness and robustness of the results obtained.Practical implicationsFollowing the steps of the proposed approach will provide the control policies for industrial manufacturing systems with setup permitted at all modes of the machine, and when the setup does not necessarily restore the machine to its operational mode. The proposed optimal policy takes into account the stochastic nature of the machine mode at the end of setup and we show that ignoring it leads to non-natural policies and underestimates significantly the safety stock thresholds.Originality/valueConsidering the assumptions presented in this paper leads to a new structure of the control laws for the production planning of manufacturing systems with setup.

Sign in / Sign up

Export Citation Format

Share Document