scholarly journals Efficient Methods for Near-Optimal Sequential Decision Making under Uncertainty

2010 ◽  
pp. 125-153
Author(s):  
Christos Dimitrakakis
Keyword(s):  
Decision Making ◽  
Sequential Decision Making ◽  
Decision Making Under Uncertainty ◽  
Sequential Decision

A State-Space Representation Model and Learning Algorithm for Real-Time Decision-Making Under Uncertainty

2007 ◽  
Author(s):  
Andreas A. Malikopoulos ◽  
Panos Y. Papalambros ◽  
Dennis N. Assanis
Keyword(s):  
Decision Making ◽  
Optimal Control ◽  
Real Time ◽  
Learning Algorithm ◽  
Control Policy ◽  
Sequential Decision Making ◽  
Space Representation ◽  
Decision Making Under Uncertainty ◽  
Sequential Decision ◽  
State Space Representation

Modeling dynamic systems incurring stochastic disturbances for deriving a control policy is a ubiquitous task in engineering. However, in some instances obtaining a model of a system may be impractical or impossible. Alternative approaches have been developed using a simulation-based stochastic framework, in which the system interacts with its environment in real time and obtains information that can be processed to produce an optimal control policy. In this context, the problem of developing a policy for controlling the system’s behavior is formulated as a sequential decision-making problem under uncertainty. This paper considers real-time sequential decision-making under uncertainty modeled as a Markov Decision Process (MDP). A state-space representation model is constructed through a learning mechanism and is used to improve system performance over time. The model allows decision making based on gradually enhanced knowledge of system response as it transitions from one state to another, in conjunction with actions taken at each state. A learning algorithm is implemented realizing in real time the optimal control policy associated with the state transitions. The proposed method is demonstrated on the single cart-pole balancing problem and a vehicle cruise control problem.

Sequential Decision-Making Under Uncertainty Using Hybrid Probability-Possibility Functions

2021 ◽  
pp. 54-66
Author(s):  
Didier Dubois ◽  
Hélène Fargier ◽  
Romain Guillaume ◽  
Agnès Rico
Keyword(s):  
Decision Making ◽  
Sequential Decision Making ◽  
Decision Making Under Uncertainty ◽  
Sequential Decision

scholarly journals Markov Decision Processes: A Tool for Sequential Decision Making under Uncertainty

2009 ◽  
Vol 30 (4) ◽  
pp. 474-483 ◽  
Author(s):  
Oguzhan Alagoz ◽  
Heather Hsu ◽  
Andrew J. Schaefer ◽  
Mark S. Roberts
Keyword(s):  
Decision Making ◽  
Markov Decision Processes ◽  
Living Donor ◽  
Decision Processes ◽  
Medical Decision ◽  
Sequential Decision Making ◽  
Optimal Timing ◽  
Decision Making Under Uncertainty ◽  
Sequential Decision ◽  
Markov Decision

We provide a tutorial on the construction and evaluation of Markov decision processes (MDPs), which are powerful analytical tools used for sequential decision making under uncertainty that have been widely used in many industrial and manufacturing applications but are underutilized in medical decision making (MDM). We demonstrate the use of an MDP to solve a sequential clinical treatment problem under uncertainty. Markov decision processes generalize standard Markov models in that a decision process is embedded in the model and multiple decisions are made over time. Furthermore, they have significant advantages over standard decision analysis. We compare MDPs to standard Markov-based simulation models by solving the problem of the optimal timing of living-donor liver transplantation using both methods. Both models result in the same optimal transplantation policy and the same total life expectancies for the same patient and living donor. The computation time for solving the MDP model is significantly smaller than that for solving the Markov model. We briefly describe the growing literature of MDPs applied to medical decisions.

An Introduction to Fully and Partially Observable Markov Decision Processes

2012 ◽  
pp. 33-62 ◽  
Author(s):  
Pascal Poupart
Keyword(s):  
Decision Making ◽  
Markov Decision Processes ◽  
Decision Processes ◽  
Sequential Decision Making ◽  
Decision Making Under Uncertainty ◽  
Sequential Decision ◽  
Markov Decision ◽  
The Common ◽  
Partially Observable Markov ◽  
Partially Observable

The goal of this chapter is to provide an introduction to Markov decision processes as a framework for sequential decision making under uncertainty. The aim of this introduction is to provide practitioners with a basic understanding of the common modeling and solution techniques. Hence, we will not delve into the details of the most recent algorithms, but rather focus on the main concepts and the issues that impact deployment in practice. More precisely, we will review fully and partially observable Markov decision processes, describe basic algorithms to find good policies and discuss modeling/computational issues that arise in practice.

Sign in / Sign up

Export Citation Format

Share Document