Dynamic programming and feedback analysis of the two dimensional tidal dynamics system

2020 ◽  
Vol 26 ◽  
pp. 109
Author(s):  
Manil T. Mohan

In this work, we consider the controlled two dimensional tidal dynamics equations in bounded domains. A distributed optimal control problem is formulated as the minimization of a suitable cost functional subject to the controlled 2D tidal dynamics equations. The existence of an optimal control is shown and the dynamic programming method for the optimal control of 2D tidal dynamics system is also described. We show that the feedback control can be obtained from the solution of an infinite dimensional Hamilton-Jacobi equation. The non-differentiability and lack of smoothness of the value function forced us to use the method of viscosity solutions to obtain a solution of the infinite dimensional Hamilton-Jacobi equation. The Bellman principle of optimality for the value function is also obtained. We show that a viscosity solution to the Hamilton-Jacobi equation can be used to derive the Pontryagin maximum principle, which give us the first order necessary conditions of optimality. Finally, we characterize the optimal control using the adjoint variable.

Author(s):  
Shihong Wang ◽  
Zuoyi Zhou

AbstractWe study the averaging of the Hamilton-Jacobi equation with fast variables in the viscosity solution sense in infinite dimensions. We prove that the viscosity solution of the original equation converges to the viscosity solution of the averaged equation and apply this result to the limit problem of the value function for an optimal control problem with fast variables.


Author(s):  
Zhili Tian ◽  
Weidong Han ◽  
Warren B. Powell

Problem definition: Clinical trials are crucial to new drug development. This study investigates optimal patient enrollment in clinical trials with interim analyses, which are analyses of treatment responses from patients at intermediate points. Our model considers uncertainties in patient enrollment and drug treatment effectiveness. We consider the benefits of completing a trial early and the cost of accelerating a trial by maximizing the net present value of drug cumulative profit. Academic/practical relevance: Clinical trials frequently account for the largest cost in drug development, and patient enrollment is an important problem in trial management. Our study develops a dynamic program, accurately capturing the dynamics of the problem, to optimize patient enrollment while learning the treatment effectiveness of an investigated drug. Methodology: The model explicitly captures both the physical state (enrolled patients) and belief states about the effectiveness of the investigated drug and a standard treatment drug. Using Bayesian updates and dynamic programming, we establish monotonicity of the value function in state variables and characterize an optimal enrollment policy. We also introduce, for the first time, the use of backward approximate dynamic programming (ADP) for this problem class. We illustrate the findings using a clinical trial program from a leading firm. Our study performs sensitivity analyses of the input parameters on the optimal enrollment policy. Results: The value function is monotonic in cumulative patient enrollment and the average responses of treatment for the investigated drug and standard treatment drug. The optimal enrollment policy is nondecreasing in the average response from patients using the investigated drug and is nonincreasing in cumulative patient enrollment in periods between two successive interim analyses. The forward ADP algorithm (or backward ADP algorithm) exploiting the monotonicity of the value function reduced the run time from 1.5 months using the exact method to a day (or 20 minutes) within 4% of the exact method. Through an application to a leading firm’s clinical trial program, the study demonstrates that the firm can have a sizable gain of drug profit following the optimal policy that our model provides. Managerial implications: We developed a new model for improving the management of clinical trials. Our study provides insights of an optimal policy and insights into the sensitivity of value function to the dropout rate and prior probability distribution. A firm can have a sizable gain in the drug’s profit by managing its trials using the optimal policies and the properties of value function. We illustrated that firms can use the ADP algorithms to develop their patient enrollment strategies.


Sign in / Sign up

Export Citation Format

Share Document