Multiobjective dynamic programming in bipolar multistage method

The multicriteria Bipolar method can be extended and used to control multicriteria, multistage decision processes. In this extension, at each stage of the given multistage process two sets of reference points are determined, constituting a reference system for the evaluation of stage alternatives. Multistage alternatives, which are compositions of stage alternatives, are assigned to one of six predefined hierarchical classes and then ranked. The aim of this paper is to show the possibility of finding the best multistage alternative, using Bellman’s optimality principle and optimality equations. Of particular importance is a theorem on the non-dominance of the best multistage alternative, proven here. The methodology proposed allows to avoid reviewing each multistage alternative, which is important in large-size problems. The method is illustrated by a numerical example and a brief description of the sustainable regional development problem. The problem can be solved by means of the proposed procedure.

On zero-sum two-person undiscounted semi-Markov games with a multichain structure

Zero-sum two-person finite undiscounted (limiting ratio average) semi-Markov games (SMGs) are considered with a general multichain structure. We derive the strategy evaluation equations for stationary strategies of the players. A relation between the payoff in the multichain SMG and that in the associated stochastic game (SG) obtained by a data-transformation is established. We prove that the multichain optimality equations (OEs) for an SMG have a solution if and only if the associated SG has optimal stationary strategies. Though the solution of the OEs may not be optimal for an SMG, we establish the significance of studying the OEs for a multichain SMG. We provide a nice example of SMGs in which one player has no optimal strategy in the stationary class but has an optimal semistationary strategy (that depends only on the initial and current state of the game). For an SMG with absorbing states, we prove that solutions in the game where all players are restricted to semistationary strategies are solutions for the unrestricted game. Finally, we prove the existence of stationary optimal strategies for unichain SMGs and conclude that the unichain condition is equivalent to require that the game satisfies some recurrence/ergodicity/weakly communicating conditions.

Bias and Overtaking Optimality for Continuous-Time Jump Markov Decision Processes in Polish Spaces

In this paper we study the bias and the overtaking optimality criteria for continuous-time jump Markov decision processes in general state and action spaces. The corresponding transition rates are allowed to be unbounded, and the reward rates may have neither upper nor lower bounds. Under appropriate hypotheses, we prove the existence of solutions to the bias optimality equations, the existence of bias optimal policies, and an equivalence relation between bias and overtaking optimality.

Bias and Overtaking Optimality for Continuous-Time Jump Markov Decision Processes in Polish Spaces

In this paper we study the bias and the overtaking optimality criteria for continuous-time jump Markov decision processes in general state and action spaces. The corresponding transition rates are allowed to be unbounded, and the reward rates may have neither upper nor lower bounds. Under appropriate hypotheses, we prove the existence of solutions to the bias optimality equations, the existence of bias optimal policies, and an equivalence relation between bias and overtaking optimality.