Approximate Solutions to Stochastic Dynamic Programs

1997 ◽  
Vol 13 (3) ◽  
pp. 392-405 ◽  
Author(s):  
Steven Stern
Keyword(s):  
Structural Estimation ◽  
Approximate Solutions ◽  
Expected Value ◽  
Stochastic Dynamic ◽  
Approximation Methods ◽  
Distributional Assumption ◽  
Approximation Errors ◽  
Dynamic Programs ◽  
Estimation Problems ◽  
Expected Values

This paper examines the properties of various approximation methods for solving stochastic dynamic programs in structural estimation problems. The problem addressed is evaluating the expected value of the maximum of available choices. The paper shows that approximating this by the maximum of expected values frequently has poor properties. It also shows that choosing a convenient distributional assumptions for the errors and then solving exactly conditional on the distributional assumption leads to small approximation errors even if the distribution is misspecified.

scholarly journals Reward Value Coding Distinct From Risk Attitude-Related Uncertainty Coding in Human Reward Systems

2007 ◽  
Vol 97 (2) ◽  
pp. 1621-1632 ◽  
Author(s):  
Philippe N. Tobler ◽  
John P. O'Doherty ◽  
Raymond J. Dolan ◽  
Wolfram Schultz
Keyword(s):  
Risk Aversion ◽  
Risk Attitude ◽  
Mean Value ◽  
Expected Value ◽  
Individual Risk ◽  
Functional Magnetic Resonance ◽  
Coding Regions ◽  
Monetary Rewards ◽  
Expected Values ◽  
Value Coding

When deciding between different options, individuals are guided by the expected (mean) value of the different outcomes and by the associated degrees of uncertainty. We used functional magnetic resonance imaging to identify brain activations coding the key decision parameters of expected value (magnitude and probability) separately from uncertainty (statistical variance) of monetary rewards. Participants discriminated behaviorally between stimuli associated with different expected values and uncertainty. Stimuli associated with higher expected values elicited monotonically increasing activations in distinct regions of the striatum, irrespective of different combinations of magnitude and probability. Stimuli associated with higher uncertainty (variance) elicited increasing activations in the lateral orbitofrontal cortex. Uncertainty-related activations covaried with individual risk aversion in lateral orbitofrontal regions and risk-seeking in more medial areas. Furthermore, activations in expected value-coding regions in prefrontal cortex covaried differentially with uncertainty depending on risk attitudes of individual participants, suggesting that separate prefrontal regions are involved in risk aversion and seeking. These data demonstrate the distinct coding in key reward structures of the two basic and crucial decision parameters, expected value, and uncertainty.

Dynamic Programs with Shared Resources and Signals: Dynamic Fluid Policies and Asymptotic Optimality

2021 ◽  
Author(s):  
David B. Brown ◽  
Jingwei Zhang
Keyword(s):  
Inventory Management ◽  
Fluid Model ◽  
Asymptotic Optimality ◽  
Capital Budgeting ◽  
Stochastic Dynamic ◽  
Performance Bounds ◽  
Sequential Decision ◽  
Shared Resources ◽  
Management Problems ◽  
Dynamic Programs

Allocating Resources Across Systems Coupled by Shared Information Many sequential decision problems involve repeatedly allocating a limited resource across subsystems that are jointly affected by randomly evolving exogenous factors. For example, in adaptive clinical trials, a decision maker needs to allocate patients to treatments in an effort to learn about the efficacy of treatments, but the number of available patients may vary randomly over time. In capital budgeting problems, firms may allocate resources to conduct R&D on new products, but funding budgets may evolve randomly. In many inventory management problems, firms need to allocate limited production capacity to satisfy uncertain demands at multiple locations, and these demands may be correlated due to vagaries in shared market conditions. In this paper, we develop a model involving “shared resources and signals” that captures these and potentially many other applications. The framework is naturally described as a stochastic dynamic program, but this problem is quite difficult to solve. We develop an approximation method based on a “dynamic fluid relaxation”: in this approximation, the subsystem state evolution is approximated by a deterministic fluid model, but the exogenous states (the signals) retain their stochastic evolution. We develop an algorithm for solving the dynamic fluid relaxation. We analyze the corresponding feasible policies and performance bounds from the dynamic fluid relaxation and show that these are asymptotically optimal as the number of subsystems grows large. We show that competing state-of-the-art approaches used in the literature on weakly coupled dynamic programs in general fail to provide asymptotic optimality. Finally, we illustrate the approach on the aforementioned dynamic capital budgeting and multilocation inventory management problems.

scholarly journals A New Approach to Approximate Solutions for Nonlinear Differential Equation

2018 ◽  
Vol 2018 ◽  
pp. 1-8
Author(s):  
Safia Meftah
Keyword(s):  
Differential Equation ◽  
Differential Equations ◽  
Small Parameter ◽  
Perturbation Method ◽  
Periodic Solutions ◽  
Asymptotic Expansions ◽  
Nonlinear Differential Equations ◽  
Approximate Solutions ◽  
Approximation Methods ◽  
New Approach

The question discussed in this study concerns one of the most helpful approximation methods, namely, the expansion of a solution of a differential equation in a series in powers of a small parameter. We used the Lindstedt-Poincaré perturbation method to construct a solution closer to uniformly valid asymptotic expansions for periodic solutions of second-order nonlinear differential equations.

scholarly journals Information Relaxations, Duality, and Convex Stochastic Dynamic Programs

2014 ◽  
Vol 62 (6) ◽  
pp. 1394-1415 ◽  
Author(s):  
David B. Brown ◽  
James E. Smith
Keyword(s):  
Stochastic Dynamic ◽  
Dynamic Programs

Structural Properties of Stochastic Dynamic Programs

2002 ◽  
Vol 50 (5) ◽  
pp. 796-809 ◽  
Author(s):  
James E. Smith ◽  
Kevin F. McCardle
Keyword(s):  
Structural Properties ◽  
Stochastic Dynamic ◽  
Dynamic Programs
Sign in / Sign up

Export Citation Format

Share Document