Hierarchical games with feedback on the assumption of a lower-level player's benevolence

A new optimality principle is proposed that generalizes the Stackelberg equilibrium principle. Its connection with the classical definition is investigated. The technique of working with the new definition is discussed. As an example, solutions are found in two hierarchical games with feedback.

A Modified MSA for Stochastic Control Problems

The classical Method of Successive Approximations (MSA) is an iterative method for solving stochastic control problems and is derived from Pontryagin’s optimality principle. It is known that the MSA may fail to converge. Using careful estimates for the backward stochastic differential equation (BSDE) this paper suggests a modification to the MSA algorithm. This modified MSA is shown to converge for general stochastic control problems with control in both the drift and diffusion coefficients. Under some additional assumptions the rate of convergence is shown. The results are valid without restrictions on the time horizon of the control problem, in contrast to iterative methods based on the theory of forward-backward stochastic differential equations.

Multiobjective dynamic programming in bipolar multistage method

The multicriteria Bipolar method can be extended and used to control multicriteria, multistage decision processes. In this extension, at each stage of the given multistage process two sets of reference points are determined, constituting a reference system for the evaluation of stage alternatives. Multistage alternatives, which are compositions of stage alternatives, are assigned to one of six predefined hierarchical classes and then ranked. The aim of this paper is to show the possibility of finding the best multistage alternative, using Bellman’s optimality principle and optimality equations. Of particular importance is a theorem on the non-dominance of the best multistage alternative, proven here. The methodology proposed allows to avoid reviewing each multistage alternative, which is important in large-size problems. The method is illustrated by a numerical example and a brief description of the sustainable regional development problem. The problem can be solved by means of the proposed procedure.

Обоснование выбора метода для распределения региональных ресурсов МЧС

The paper describes the choice of a method for the correct and correct allocation of resources within the Ministry of Emergency Situations management. As such a method, it was decided to choose the Bellman algorithm. A mathematical model is introduced that allows applying the chosen optimality principle. The choice of the general type of function that is most suitable for resource allocation is given.

Dynamic Programming and Hamilton–Jacobi–Bellman Equations on Time Scales

Bellman optimality principle for the stochastic dynamic system on time scales is derived, which includes the continuous time and discrete time as special cases. At the same time, the Hamilton–Jacobi–Bellman (HJB) equation on time scales is obtained. Finally, an example is employed to illustrate our main results.

How to Do Things with Words: A Bayesian Approach

Communication changes the beliefs of the listener and of the speaker. The value of a communicative act stems from the valuable belief states which result from this act. To model this we build on the Interactive POMDP (IPOMDP) framework, which extends POMDPs to allow agents to model others in multi-agent settings, and we include communication that can take place between the agents to formulate Communicative IPOMDPs (CIPOMDPs). We treat communication as a type of action and therefore, decisions regarding communicative acts are based on decision-theoretic planning using the Bellman optimality principle and value iteration, just as they are for all other rational actions. As in any form of planning, the results of actions need to be precisely specified. We use the Bayes’ theorem to derive how agents update their beliefs in CIPOMDPs; updates are due to agents’ actions, observations, messages they send to other agents, and messages they receive from others. The Bayesian decision-theoretic approach frees us from the commonly made assumption of cooperative discourse – we consider agents which are free to be dishonest while communicating and are guided only by their selfish rationality. We use a simple Tiger game to illustrate the belief update, and to show that the ability to rationally communicate allows agents to improve efficiency of their interactions.

The Confucian doctrine of the Mean, the optimality principle, and social harmony

The Confucian doctrine of the Mean teaches that too much is as bad as too little. The Aristotelian doctrine of the Mean coincidently articulates that there can be too much or too little of nearly every human passion and action. In neoclassical economics, it is assumed that people tend to take any action at the optimal (not too much and not too little) level to maximise the net happiness from the action. This article argues that the Confucian doctrine of the Mean concurs with the optimality principle, and therefore that the optimality principle is a representation of human nature and can be understood as universal human wisdom. It follows that people can adopt both the Confucian doctrine of the Mean and the optimality principle as worldly common wisdom beyond the blunt dichotomy of spiritual orientalism and materialistic individualism. Too much emphasis on the technical differentials between the two has undermined the common wisdom embedded in them.