On dynamic programming recursions for multiplicative Markov decision chains

Author(s):  
Karel Sladký
Author(s):  
L. Mandow ◽  
J. L. Perez-de-la-Cruz ◽  
N. Pozas

AbstractThis paper addresses the problem of approximating the set of all solutions for Multi-objective Markov Decision Processes. We show that in the vast majority of interesting cases, the number of solutions is exponential or even infinite. In order to overcome this difficulty we propose to approximate the set of all solutions by means of a limited precision approach based on White’s multi-objective value-iteration dynamic programming algorithm. We prove that the number of calculated solutions is tractable and show experimentally that the solutions obtained are a good approximation of the true Pareto front.


1980 ◽  
Vol 17 (04) ◽  
pp. 996-1003
Author(s):  
D. R. Robinson

It is known that when costs are unbounded satisfaction of the appropriate dynamic programming ‘optimality' equation by a policy is not sufficient to guarantee its average optimality. A ‘lowest-order potential' condition is introduced which, along with the dynamic programming equation, is sufficient to establish the optimality of the policy. Also, it is shown that under fairly general conditions, if the lowest-order potential condition is not satisfied there exists a non-memoryless policy with smaller average cost than the policy satisfying the dynamic programming equation.


1993 ◽  
Vol 25 (04) ◽  
pp. 979-996
Author(s):  
Arie Hordijk ◽  
Ger Koole

In this paper we study scheduling problems of multiclass customers on identical parallel processors. A new type of arrival process, called a Markov decision arrival process, is introduced. This arrival process can be controlled and allows for an indirect dependence on the numbers of customers in the queues. As a special case we show the optimality of LEPT and the µc-rule in the last node of a controlled tandem network for various cost structures. A unifying proof using dynamic programming is given.


1992 ◽  
Vol 29 (03) ◽  
pp. 633-644
Author(s):  
K. D. Glazebrook ◽  
Michael P. Bailey ◽  
Lyn R. Whitaker

In response to the computational complexity of the dynamic programming/backwards induction approach to the development of optimal policies for semi-Markov decision processes, we propose a class of heuristics resulting from an inductive process which proceeds forwards in time. These heuristics always choose actions in such a way as to minimize some measure of the current cost rate. We describe a procedure for calculating such cost rate heuristics. The quality of the performance of such policies is related to the speed of evolution (in a cost sense) of the process. A simple model of preventive maintenance is described in detail. Cost rate heuristics for this problem are calculated and assessed computationally.


Symmetry ◽  
2019 ◽  
Vol 11 (3) ◽  
pp. 303 ◽  
Author(s):  
Sijia Hao ◽  
Rui Song ◽  
Shiwei He ◽  
Zekang Lan

Rescheduling is often needed when trains stay in segments or stations longer than specified in the timetable due to disturbances. Under crowded situations, it is more challenging to return to normal with heavy passenger flow. Considering making a trade-off between passenger loss and operating costs, we present a train regulation combined with a passenger control model by analyzing the interactive relationship between passenger behaviors and train operation. In this paper, we convert the problem into a Markov decision process and then propose the management strategy of regulating the running time and controlling the number of boarding passengers. Owing to the high dimensions of the large-scale problem, we applied the Approximate Dynamic Programming (ADP) approach, which approximates the value function with state features to improve computational efficiency. Finally, we designed three experimental scenarios to verify the effectiveness of our proposed model and approach. The results show that both the proposed model and the approach have a good performance in the cases with different passenger flows and different disturbances.


Sign in / Sign up

Export Citation Format

Share Document