scholarly journals Solving Factored MDPs with Hybrid State and Action Variables

2006 ◽  
Vol 27 ◽  
pp. 153-201 ◽  
Author(s):  
B. Kveton ◽  
M. Hauskrecht ◽  
C. Guestrin
Keyword(s):  
Linear Programming ◽  
Optimization Problems ◽  
Scale Up ◽  
Efficient Solutions ◽  
Compact Representation ◽  
Optimal Value ◽  
Markov Decision ◽  
Computational Aspects ◽  
Approximate Linear Programming ◽  
Continuous And Discrete Variables

Efficient representations and solutions for large decision problems with continuous and discrete variables are among the most important challenges faced by the designers of automated decision support systems. In this paper, we describe a novel hybrid factored Markov decision process (MDP) model that allows for a compact representation of these problems, and a new hybrid approximate linear programming (HALP) framework that permits their efficient solutions. The central idea of HALP is to approximate the optimal value function by a linear combination of basis functions and optimize its weights by linear programming. We analyze both theoretical and computational aspects of this approach, and demonstrate its scale-up potential on several hybrid optimization problems.

scholarly journals Abstract Cores in Implicit Hitting Set MaxSat Solving (Extended Abstract)

2021 ◽  
Author(s):  
Jeremias Berg ◽  
Fahiem Bacchus ◽  
Alex Poole
Keyword(s):  
Linear Programming ◽  
Optimization Problems ◽  
State Of The Art ◽  
Empirical Evaluation ◽  
Boolean Satisfiability ◽  
Compact Representation ◽  
Maximum Satisfiability ◽  
Hitting Set ◽  
Combinatorial Optimization Problems ◽  
Core Extraction

Maximum satisfiability (MaxSat) solving is an active area of research motivated by numerous successful applications to solving NP-hard combinatorial optimization problems. One of the most successful approaches for solving MaxSat instances from real world domains are the so called implicit hitting set (IHS) solvers. IHS solvers decouple MaxSat solving into separate core-extraction (i.e. reasoning) and optimization steps which are tackled by a Boolean satisfiability (SAT) and an integer linear programming (IP) solver, respectively. While the approach shows state-of-the-art performance on many industrial instances, it is known that there exists instances on which IHS solvers need to extract an exponential number of cores before terminating. Motivated by the simplest of these problematic instances, we propose abstract cores, a compact representation for a potentially exponential number of regular cores. We demonstrate how to incorporate abstract core reasoning into the IHS algorithm and report on an empirical evaluation demonstrating, that including abstract cores into a state-of-the-art IHS solver improves its performance enough to surpass the best performing solvers of the 2019 MaxSat Evaluation.

Network-Based Approximate Linear Programming for Discrete Optimization

2020 ◽  
Vol 68 (6) ◽  
pp. 1767-1786
Author(s):  
Selvaprabu Nadarajah ◽  
Andre A. Cire
Keyword(s):  
Linear Programming ◽  
Discrete Optimization ◽  
Optimization Problems ◽  
Linear Programs ◽  
Solution Quality ◽  
Marketing Analytics ◽  
Dynamic Programs ◽  
Approximate Linear Programming ◽  
Commercial Solver ◽  
Discrete Optimization Problems

Several prescriptive tasks in business and engineering as well as prediction in machine learning entail the solution of challenging discrete optimization problems. We recast the typical optimization formulation of these problems as high-dimensional dynamic programs and approach their approximation via linear programming. We develop tractable approximate linear programs with supporting theory by bringing together tools from state-space aggregations, networks, and perfect graphs (i.e., graph completions). We embed these models in a simple branch-and-bound scheme to solve applications in marketing analytics and the maintenance of energy or city-owned assets. We find that the resulting technique substantially outperforms a state-of-the-art commercial solver as well as aggregation-heuristics in terms of both solution quality and time. Our results motivate further consideration of networks and graph theory in approximate linear programming for solving deterministic and stochastic discrete optimization problems.

scholarly journals First-order sensitivity of the optimal value in a Markov decision model with respect to deviations in the transition probability function

2020 ◽  
Vol 92 (1) ◽  
pp. 165-197 ◽  
Author(s):  
Patrick Kern ◽  
Axel Simroth ◽  
Henryk Zähle
Keyword(s):  
Optimization Problems ◽  
Transition Probabilities ◽  
Broad Class ◽  
Transition Probability ◽  
Mathematical Finance ◽  
Model Complexity ◽  
First Order ◽  
Optimal Value ◽  
Markov Decision ◽  
Markov Decision Model

Abstract Markov decision models (MDM) used in practical applications are most often less complex than the underlying ‘true’ MDM. The reduction of model complexity is performed for several reasons. However, it is obviously of interest to know what kind of model reduction is reasonable (in regard to the optimal value) and what kind is not. In this article we propose a way how to address this question. We introduce a sort of derivative of the optimal value as a function of the transition probabilities, which can be used to measure the (first-order) sensitivity of the optimal value w.r.t. changes in the transition probabilities. ‘Differentiability’ is obtained for a fairly broad class of MDMs, and the ‘derivative’ is specified explicitly. Our theoretical findings are illustrated by means of optimization problems in inventory control and mathematical finance.

scholarly journals On quantitative stability in infinite-dimensional optimization under uncertainty

2021 ◽  
Author(s):  
M. Hoffhues ◽  
W. Römisch ◽  
T. M. Surowiec
Keyword(s):  
Stochastic Optimization ◽  
Constrained Optimization ◽  
Optimization Problems ◽  
Optimization Under Uncertainty ◽  
Optimal Solutions ◽  
Probability Metrics ◽  
Pde Constrained Optimization ◽  
Quantitative Stability ◽  
Infinite Dimensional ◽  
Optimal Value

AbstractThe vast majority of stochastic optimization problems require the approximation of the underlying probability measure, e.g., by sampling or using observations. It is therefore crucial to understand the dependence of the optimal value and optimal solutions on these approximations as the sample size increases or more data becomes available. Due to the weak convergence properties of sequences of probability measures, there is no guarantee that these quantities will exhibit favorable asymptotic properties. We consider a class of infinite-dimensional stochastic optimization problems inspired by recent work on PDE-constrained optimization as well as functional data analysis. For this class of problems, we provide both qualitative and quantitative stability results on the optimal value and optimal solutions. In both cases, we make use of the method of probability metrics. The optimal values are shown to be Lipschitz continuous with respect to a minimal information metric and consequently, under further regularity assumptions, with respect to certain Fortet-Mourier and Wasserstein metrics. We prove that even in the most favorable setting, the solutions are at best Hölder continuous with respect to changes in the underlying measure. The theoretical results are tested in the context of Monte Carlo approximation for a numerical example involving PDE-constrained optimization under uncertainty.

scholarly journals Structure of Pareto Solutions of Generalized Polyhedral-Valued Vector Optimization Problems in Banach Spaces

2013 ◽  
Vol 2013 ◽  
pp. 1-10
Author(s):  
Qinghai He ◽  
Weili Kong
Keyword(s):  
Banach Spaces ◽  
Vector Optimization ◽  
Optimization Problems ◽  
Vector Optimization Problem ◽  
Solution Set ◽  
Pareto Optimal ◽  
Pareto Solution ◽  
Value Set ◽  
Set Valued Mapping ◽  
Optimal Value

In general Banach spaces, we consider a vector optimization problem (SVOP) in which the objective is a set-valued mapping whose graph is the union of finitely many polyhedra or the union of finitely many generalized polyhedra. Dropping the compactness assumption, we establish some results on structure of the weak Pareto solution set, Pareto solution set, weak Pareto optimal value set, and Pareto optimal value set of (SVOP) and on connectedness of Pareto solution set and Pareto optimal value set of (SVOP). In particular, we improved and generalize, Arrow, Barankin, and Blackwell’s classical results in Euclidean spaces and Zheng and Yang’s results in general Banach spaces.

Sign in / Sign up

Export Citation Format

Share Document