scholarly journals Planning Under Uncertainty Applications in Power Plants Using Factored Markov Decision Processes

Energies ◽  
2020 ◽  
Vol 13 (9) ◽  
pp. 2302 ◽  
Author(s):  
Alberto Reyes ◽  
L. Enrique Sucar ◽  
Pablo H. Ibargüengoytia ◽  
Eduardo F. Morales
Keyword(s):  
Markov Decision Processes ◽  
Power Plants ◽  
Combined Cycle ◽  
Decision Processes ◽  
Machine Learning Techniques ◽  
Planning Under Uncertainty ◽  
Hydroelectric Power ◽  
Steam Generation ◽  
Power Domain ◽  
Markov Decision

Due to its ability to deal with non-determinism and partial observability, represent goals as an immediate reward function and find optimal solutions, planning under uncertainty using factored Markov Decision Processes (FMDPs) has increased its importance and usage in power plants and power systems. In this paper, three different applications using this approach are described: (i) optimal dam management in hydroelectric power plants, (ii) inspection and surveillance in electric substations, and (iii) optimization of steam generation in a combined cycle power plant. For each case, the technique has demonstrated to find optimal action policies in uncertain settings, present good response and compilation times, deal with stochastic variables and be a good alternative to traditional control systems. The main contributions of this work are as follows, a methodology to approximate a decision model using machine learning techniques, and examples of how to specify and solve problems in the electric power domain in terms of a FMDP.

scholarly journals Speeding Up the Convergence of Value Iteration in Partially Observable Markov Decision Processes

2001 ◽  
Vol 14 ◽  
pp. 29-51 ◽  
Author(s):  
N. L. Zhang ◽  
W. Zhang
Keyword(s):  
Markov Decision Processes ◽  
Decision Processes ◽  
Benchmark Problems ◽  
Test Problems ◽  
Value Iteration ◽  
Planning Under Uncertainty ◽  
Markov Decision ◽  
Partially Observable Markov ◽  
Partially Observable ◽  
Number Of Iterations

Partially observable Markov decision processes (POMDPs) have recently become popular among many AI researchers because they serve as a natural model for planning under uncertainty. Value iteration is a well-known algorithm for finding optimal policies for POMDPs. It typically takes a large number of iterations to converge. This paper proposes a method for accelerating the convergence of value iteration. The method has been evaluated on an array of benchmark problems and was found to be very effective: It enabled value iteration to converge after only a few iterations on all the test problems.

BOUNDED-PARAMETER PARTIALLY OBSERVABLE MARKOV DECISION PROCESSES: FRAMEWORK AND ALGORITHM

2013 ◽  
Vol 21 (06) ◽  
pp. 821-863 ◽  
Author(s):  
YAODONG NI ◽  
ZHI-QIANG LIU
Keyword(s):  
Markov Decision Processes ◽  
Real Life ◽  
Decision Processes ◽  
Iteration Algorithm ◽  
Value Iteration ◽  
Planning Under Uncertainty ◽  
Markov Decision ◽  
Real Life Situation ◽  
Partially Observable Markov ◽  
Partially Observable

Partially observable Markov decision processes (POMDPs) are powerful for planning under uncertainty. However, it is usually impractical to employ a POMDP with exact parameters to model the real-life situation precisely, due to various reasons such as limited data for learning the model, inability of exact POMDPs to model dynamic situations, etc. In this paper, assuming that the parameters of POMDPs are imprecise but bounded, we formulate the framework of bounded-parameter partially observable Markov decision processes (BPOMDPs). A modified value iteration is proposed as a basic strategy for tackling parameter imprecision in BPOMDPs. In addition, we design the UL-based value iteration algorithm, in which each value backup is based on two sets of vectors called U-set and L-set. We propose four strategies for computing U-set and L-set. We analyze theoretically the computational complexity and the reward loss of the algorithm. The effectiveness and robustness of the algorithm are shown empirically.

Extreme-point solutions in Markov decision processes

1983 ◽  
Vol 20 (04) ◽  
pp. 835-842
Author(s):  
David Assaf
Keyword(s):  
Convex Function ◽  
Extreme Point ◽  
Markov Decision Processes ◽  
Convex Functions ◽  
Sufficient Conditions ◽  
Decision Processes ◽  
Markov Decision ◽  
Full Solution

The paper presents sufficient conditions for certain functions to be convex. Functions of this type often appear in Markov decision processes, where their maximum is the solution of the problem. Since a convex function takes its maximum at an extreme point, the conditions may greatly simplify a problem. In some cases a full solution may be obtained after the reduction is made. Some illustrative examples are discussed.

Sign in / Sign up

Export Citation Format

Share Document