scholarly journals Optimization problem under change of regime of interest rate

2016 ◽  
Vol 16 (05) ◽  
pp. 1650015 ◽  
Author(s):  
Bogdan Iftimie ◽  
Monique Jeanblanc ◽  
Thomas Lim
Keyword(s):  
Interest Rate ◽  
Optimization Problem ◽  
Dual Problem ◽  
Value Function ◽  
Expected Value ◽  
Value Functions ◽  
Dual Approach ◽  
Primal Problem ◽  
Terminal Wealth ◽  
The Value Function

In this paper, we study the problem of maximization of the expected value of the sum of the utility of the terminal wealth and the utility of the consumption, in a case where some sudden jumps in the risk-free interest rate create incompleteness. To solve the problem we use the dual approach. We characterize the value function of the dual problem by a BSDE and the duality between the primal and the dual value functions is exploited to study the BSDE associated to the primal problem.

scholarly journals Hill Climbing on Value Estimates for Search-control in Dyna

2019 ◽  
Author(s):  
Yangchen Pan ◽  
Hengshuai Yao ◽  
Amir-massoud Farahmand ◽  
Martha White
Keyword(s):  
Value Function ◽  
Gradient Algorithm ◽  
Hill Climbing ◽  
Value Functions ◽  
Current Estimate ◽  
Sampling Distributions ◽  
Current Value ◽  
Empirical Demonstration ◽  
Search Control ◽  
The Value Function

Dyna is an architecture for model based reinforcement learning (RL), where simulated experience from a model is used to update policies or value functions. A key component of Dyna is search control, the mechanism to generate the state and action from which the agent queries the model, which remains largely unexplored. In this work, we propose to generate such states by using the trajectory obtained from Hill Climbing (HC) the current estimate of the value function. This has the effect of propagating value from high value regions and of preemptively updating value estimates of the regions that the agent is likely to visit next. We derive a noisy projected natural gradient algorithm for hill climbing, and highlight a connection to Langevin dynamics. We provide an empirical demonstration on four classical domains that our algorithm, HC Dyna, can obtain significant sample efficiency improvements. We study the properties of different sampling distributions for search control, and find that there appears to be a benefit specifically from using the samples generated by climbing on current value estimates from low value to high value region.

scholarly journals Dynkin game under g-expectation in continuous time

2020 ◽  
Vol 9 (2) ◽  
pp. 459-470
Author(s):  
Helin Wu ◽  
Yong Ren ◽  
Feng Hu
Keyword(s):  
Differential Equation ◽  
Stochastic Differential Equation ◽  
Continuous Time ◽  
Value Function ◽  
Backward Stochastic Differential Equation ◽  
Saddle Points ◽  
Value Functions ◽  
Dynkin Game ◽  
The Value Function

Abstract In this paper, we investigate some kind of Dynkin game under g-expectation induced by backward stochastic differential equation (short for BSDE). The lower and upper value functions $$\underline{V}_t=ess\sup \nolimits _{\tau \in {\mathcal {T}_t}} ess\inf \nolimits _{\sigma \in {\mathcal {T}_t}}\mathcal {E}^g_t[R(\tau ,\sigma )]$$ V ̲ t = e s s sup τ ∈ T t e s s inf σ ∈ T t E t g [ R ( τ , σ ) ] and $$\overline{V}_t=ess\inf \nolimits _{\sigma \in {\mathcal {T}_t}} ess\sup \nolimits _{\tau \in {\mathcal {T}_t}}\mathcal {E}^g_t[R(\tau ,\sigma )]$$ V ¯ t = e s s inf σ ∈ T t e s s sup τ ∈ T t E t g [ R ( τ , σ ) ] are defined, respectively. Under some suitable assumptions, a pair of saddle points is obtained and the value function of Dynkin game $$V(t)=\underline{V}_t=\overline{V}_t$$ V ( t ) = V ̲ t = V ¯ t follows. Furthermore, we also consider the constrained case of Dynkin game.

Canonical Dual Solutions to Quadratic Optimization over One Quadratic Constraint

2015 ◽  
Vol 32 (01) ◽  
pp. 1540007 ◽  
Author(s):  
Wenxun Xing ◽  
Shu-Cherng Fang ◽  
Ruey-Lin Sheu ◽  
Liping Zhang
Keyword(s):  
Optimization Problem ◽  
Quadratic Optimization ◽  
Global Minimizer ◽  
Dual Approach ◽  
Primal Problem ◽  
Quadratic Constraint ◽  
Quadratic Optimization Problem ◽  
Feasible Domain ◽  
Bound Estimation ◽  
Slater’S Condition

A quadratic optimization problem with one nonconvex quadratic constraint is studied using the canonical dual approach. Under the dual Slater's condition, we show that the canonical dual has a smooth concave objective function over a convex feasible domain, and this dual has a finite supremum unless the original quadratic optimization problem is infeasible. This supremum, when it exists, always equals to the minimum value of the primal problem. Moreover, a global minimizer of the primal problem can be provided by a dual-to-primal conversion plus a "boundarification" technique. Application to solving a quadratic programming problem over a ball is included and an error bound estimation is provided.

Geometric Asymptotic Approximation of Value Functions

2009 ◽  
Vol 9 (1) ◽  
Author(s):  
Axel Anderson
Keyword(s):  
Value Function ◽  
Payoff Function ◽  
The State ◽  
Second Derivative ◽  
Value Functions ◽  
State Variable ◽  
Specific Formula ◽  
Geometric Term ◽  
Dynamic Stochastic ◽  
The Value Function

This paper characterizes the behavior of value functions in dynamic stochastic discounted programming models near fixed points of the state space. When the second derivative of the flow payoff function is bounded, the value function is proportional to a linear function plus geometric term. A specific formula for the exponent of this geometric term is provided. This exponent continuously falls in the rate of patience.If the state variable is a martingale, the second derivative of the value function is unbounded. If the state variable is instead a strict local submartingale, then the same holds for the first derivative of the value function. Thus, the proposed approximation is more accurate than Taylor series approximation.The approximation result is used to characterize locally optimal policies in several fundamental economic problems.

Bilevel Integer Programs with Stochastic Right-Hand Sides

2021 ◽  
Author(s):  
Junlong Zhang ◽  
Osman Y. Özaltın
Keyword(s):  
Structural Properties ◽  
Large Scale ◽  
Value Function ◽  
Integer Program ◽  
Value Functions ◽  
Integer Programs ◽  
Solution Algorithms ◽  
Right Hand ◽  
Solution Methods ◽  
The Value Function

We develop an exact value function-based approach to solve a class of bilevel integer programs with stochastic right-hand sides. We first study structural properties and design two methods to efficiently construct the value function of a bilevel integer program. Most notably, we generalize the integer complementary slackness theorem to bilevel integer programs. We also show that the value function of a bilevel integer program can be characterized by its values on a set of so-called bilevel minimal vectors. We then solve the value function reformulation of the original bilevel integer program with stochastic right-hand sides using a branch-and-bound algorithm. We demonstrate the performance of our solution methods on a set of randomly generated instances. We also apply the proposed approach to a bilevel facility interdiction problem. Our computational experiments show that the proposed solution methods can efficiently optimize large-scale instances. The performance of our value function-based approach is relatively insensitive to the number of scenarios, but it is sensitive to the number of constraints with stochastic right-hand sides. Summary of Contribution: Bilevel integer programs arise in many different application areas of operations research including supply chain, energy, defense, and revenue management. This paper derives structural properties of the value functions of bilevel integer programs. Furthermore, it proposes exact solution algorithms for a class of bilevel integer programs with stochastic right-hand sides. These algorithms extend the applicability of bilevel integer programs to a larger set of decision-making problems under uncertainty.

scholarly journals A Hybrid Approach of Bundle and Benders Applied Large Mixed Linear Integer Problem

2013 ◽  
Vol 2013 ◽  
pp. 1-11 ◽  
Author(s):  
Placido Rogerio Pinheiro ◽  
Paulo Roberto Oliveira
Keyword(s):  
Dual Problem ◽  
Value Function ◽  
Benders Decomposition ◽  
Linear Problem ◽  
Hybrid Approach ◽  
Mixed Integer ◽  
Constraint Matrix ◽  
Coupling Constraints ◽  
Integer Problem ◽  
The Value Function

Consider a large mixed integer linear problem where structure of the constraint matrix is sparse, with independent blocks, and coupling constraints and variables. There is one of the groups of constraints to make difficult the application of Benders scheme decomposition. In this work, we propose the following algorithm; a Lagrangian relaxation is made on the mentioned set of constraints; we presented a process heuristic for the calculation of the multiplier through the resolution of the dual problem, structured starting from the bundle methods. According to the methodology proposed, for each iteration of the algorithm, we propose Benders decomposition where quotas are provided for the value function andε-subgradient.

scholarly journals The Use of the Duality Principle to Solve Optimization Problems

2018 ◽  
Vol 6 (1) ◽  
pp. 33
Author(s):  
Rowland Jerry Okechukwu Ekeocha ◽  
Chukwunedum Uzor ◽  
Clement Anetor
Keyword(s):  
Optimization Problem ◽  
Dual Problem ◽  
Optimization Problems ◽  
Linear Program ◽  
Duality Gap ◽  
Duality Principle ◽  
Primal Problem ◽  
Optimal Values ◽  
Primal And Dual Problems ◽  
Primal And Dual

<p><span>The duality principle provides that optimization problems may be viewed from either of two perspectives, the primal problem or the dual problem. The solution to the dual problem provides a lower bound to the solution of the primal (minimization) problem. However the optimal values of the primal and dual problems need not be equal. Their difference is called the duality gap. For convex optimization problems, the duality gap is zero under a constraint qualification condition.<span>  </span>In other words given any linear program, there is another related linear program called the dual. In this paper, an understanding of the dual linear program will be developed. This understanding will give important insights into the algorithm and solution of optimization problem in linear programming. <span> </span>Thus the main concepts of duality will be explored by the solution of simple optimization problem.</span></p>

scholarly journals When Inaccuracies in Value Functions Do Not Propagate on Optima and Equilibria

Mathematics ◽  
2020 ◽  
Vol 8 (7) ◽  
pp. 1109 ◽  
Author(s):  
Agnieszka Wiszniewska-Matyszkiel ◽  
Rajani Singh
Keyword(s):  
Dynamic Optimization ◽  
Dynamic Games ◽  
Value Function ◽  
Optimization Problems ◽  
A Priori ◽  
Value Functions ◽  
Feedback Controls ◽  
Time Dynamic ◽  
Coupled Equations ◽  
The Value Function

We study general classes of discrete time dynamic optimization problems and dynamic games with feedback controls. In such problems, the solution is usually found by using the Bellman or Hamilton–Jacobi–Bellman equation for the value function in the case of dynamic optimization and a set of such coupled equations for dynamic games, which is not always possible accurately. We derive general rules stating what kind of errors in the calculation or computation of the value function do not result in errors in calculation or computation of an optimal control or a Nash equilibrium along the corresponding trajectory. This general result concerns not only errors resulting from using numerical methods but also errors resulting from some preliminary assumptions related to replacing the actual value functions by some a priori assumed constraints for them on certain subsets. We illustrate the results by a motivating example of the Fish Wars, with singularities in payoffs.

Optimal Insurance-Package and Investment Problem for an Insurer

2018 ◽  
Vol 6 (1) ◽  
pp. 85-96
Author(s):  
Delei Sheng ◽  
Linfang Xing
Keyword(s):  
Value Function ◽  
Investment Strategy ◽  
Optimal Combination ◽  
Hjb Equations ◽  
Terminal Wealth ◽  
Optimal Insurance ◽  
Yield Rate ◽  
Investment Problem ◽  
Hamilton Jacobi Bellman ◽  
The Value Function

AbstractAn insurance-package is a combination being tie-in at least two different categories of insurances with different underwriting-yield-rate. In this paper, the optimal insurance-package and investment problem is investigated by maximizing the insurer’s exponential utility of terminal wealth to find the optimal combination-share and investment strategy. Using the methods of stochastic analysis and stochastic optimal control, the Hamilton-Jacobi-Bellman (HJB) equations are established, the optimal strategy and the value function are obtained in closed form. By comparing with classical results, it shows that the insurance-package can enhance the utility of terminal wealth, meanwhile, reduce the insurer’s claim risk.

Optimal stopping with random intervention times

2002 ◽  
Vol 34 (01) ◽  
pp. 141-157 ◽  
Author(s):  
Paul Dupuis ◽  
Hui Wang
Keyword(s):  
Optimal Stopping ◽  
Continuous Time ◽  
Value Function ◽  
Value Functions ◽  
Optimal Exercise Boundary ◽  
Exercise Boundary ◽  
Time Optimal ◽  
Optimal Stopping Problems ◽  
Necessary And Sufficient ◽  
The Value Function

We consider a class of optimal stopping problems where the ability to stop depends on an exogenous Poisson signal process - we can only stop at the Poisson jump times. Even though the time variable in these problems has a discrete aspect, a variational inequality can be obtained by considering an underlying continuous-time structure. Depending on whether stopping is allowed at t = 0, the value function exhibits different properties across the optimal exercise boundary. Indeed, the value function is only 𝒞 0 across the optimal boundary when stopping is allowed at t = 0 and 𝒞 2 otherwise, both contradicting the usual 𝒞 1 smoothness that is necessary and sufficient for the application of the principle of smooth fit. Also discussed is an equivalent stochastic control formulation for these stopping problems. Finally, we derive the asymptotic behaviour of the value functions and optimal exercise boundaries as the intensity of the Poisson process goes to infinity or, roughly speaking, as the problems converge to the classical continuous-time optimal stopping problems.

Sign in / Sign up

Export Citation Format

Share Document