scholarly journals A particular upper expectation as global belief model for discrete-time finite-state uncertain processes

2020 ◽  
Author(s):  
Natan T'Joens ◽  
Jasper De Bock ◽  
Gert de Cooman
Keyword(s):  
Discrete Time ◽  
Finite State ◽  
Belief Model

On quasi-stationary distributions in absorbing continuous-time finite Markov chains

1967 ◽  
Vol 4 (1) ◽  
pp. 192-196 ◽  
Author(s):  
J. N. Darroch ◽  
E. Seneta
Keyword(s):  
Markov Chains ◽  
Discrete Time ◽  
Continuous Time ◽  
Present Note ◽  
Time Parameter ◽  
Stationary Distributions ◽  
Finite State ◽  
Absorbing Markov Chains ◽  
Finite Markov Chains ◽  
Finite State Space

In a recent paper, the authors have discussed the concept of quasi-stationary distributions for absorbing Markov chains having a finite state space, with the further restriction of discrete time. The purpose of the present note is to summarize the analogous results when the time parameter is continuous.

State-Feedback Zero-Sum Dynamic Games

2017 ◽  
Author(s):  
João P. Hespanha
Keyword(s):  
Discrete Time ◽  
Information Structure ◽  
Dynamic Games ◽  
State Feedback ◽  
Linear Quadratic ◽  
Feedback Information ◽  
Time Dynamic ◽  
Finite State ◽  
Solution Methods ◽  
Zero Sum

This chapter focuses on the computation of the saddle-point equilibrium of a zero-sum discrete time dynamic game in a state-feedback policy. It begins by considering solution methods for two-player zero sum dynamic games in discrete time, assuming a finite horizon stage-additive cost that Player 1 wants to minimize and Player 2 wants to maximize, and taking into account a state feedback information structure. The discussion then turns to discrete time dynamic programming, the use of MATLAB to solve zero-sum games with finite state spaces and finite action spaces, and discrete time linear quadratic dynamic games. The chapter concludes with a practice exercise that requires computing the cost-to-go for each state of the tic-tac-toe game, and the corresponding solution.

The Computation of Average Optimal Policies in Denumerable State Markov Decision Chains

1997 ◽  
Vol 29 (01) ◽  
pp. 114-137
Author(s):  
Linn I. Sennott
Keyword(s):  
Discrete Time ◽  
Average Cost ◽  
Queueing Systems ◽  
State Spaces ◽  
Original Process ◽  
Optimal Policies ◽  
Finite State ◽  
Markov Decision ◽  
Optimal Average ◽  
Infinite State

This paper studies the expected average cost control problem for discrete-time Markov decision processes with denumerably infinite state spaces. A sequence of finite state space truncations is defined such that the average costs and average optimal policies in the sequence converge to the optimal average cost and an optimal policy in the original process. The theory is illustrated with several examples from the control of discrete-time queueing systems. Numerical results are discussed.

Finite state and discrete time approximation for filters

2001 ◽  
Vol 47 (4) ◽  
pp. 2485-2495 ◽  
Author(s):  
Anna Gerardi ◽  
Paola Tardelli
Keyword(s):  
Discrete Time ◽  
Time Approximation ◽  
Discrete Time Approximation ◽  
Finite State

Stable Encoding of Finite-State Machines in Discrete-Time Recurrent Neural Nets with Sigmoid Units

2000 ◽  
Vol 12 (9) ◽  
pp. 2129-2174 ◽  
Author(s):  
Rafael C. Carrasco ◽  
Mikel L. Forcada ◽  
M. Ángeles Valdés-Muñoz ◽  
Ramón P. Ñeco
Keyword(s):  
Discrete Time ◽  
Finite State Machine ◽  
Finite State Machines ◽  
Activation Function ◽  
Neural Nets ◽  
State Machines ◽  
Computational Power ◽  
Simple Strategy ◽  
Finite State ◽  
Detailed Presentation

There has been a lot of interest in the use of discrete-time recurrent neural nets (DTRNN) to learn finite-state tasks, with interesting results regarding the induction of simple finite-state machines from input–output strings. Parallel work has studied the computational power of DTRNN in connection with finite-state computation. This article describes a simple strategy to devise stable encodings of finite-state machines in computationally capable discrete-time recurrent neural architectures with sigmoid units and gives a detailed presentation on how this strategy may be applied to encode a general class of finite-state machines in a variety of commonly used first- and second-order recurrent neural networks. Unlike previous work that either imposed some restrictions to state values or used a detailed analysis based on fixed-point attractors, our approach applies to any positive, bounded, strictly growing, continuous activation function and uses simple bounding criteria based on a study of the conditions under which a proposed encoding scheme guarantees that the DTRNN is actually behaving as a finite-state machine.

scholarly journals On the total reward variance for continuous-time Markov reward chains

2006 ◽  
Vol 43 (04) ◽  
pp. 1044-1052 ◽  
Author(s):  
Nico M. Van Dijk ◽  
Karel Sladký
Keyword(s):  
Growth Rate ◽  
Discrete Time ◽  
Continuous Time ◽  
Asymptotically Linear ◽  
State Spaces ◽  
Total Reward ◽  
Finite State ◽  
Markov Reward ◽  
Discrete Time Case

As an extension of the discrete-time case, this note investigates the variance of the total cumulative reward for continuous-time Markov reward chains with finite state spaces. The results correspond to discrete-time results. In particular, the variance growth rate is shown to be asymptotically linear in time. Expressions are provided to compute this growth rate.

scholarly journals Finite-TimeH∞Control for a Class of Discrete-Time Markov Jump Systems with Actuator Saturation via Dynamic Antiwindup Design

2014 ◽  
Vol 2014 ◽  
pp. 1-9
Author(s):  
Junjie Zhao ◽  
Jing Wang ◽  
Bo Li
Keyword(s):  
Discrete Time ◽  
Finite Time ◽  
Actuator Saturation ◽  
Linear Matrix ◽  
Markov Jump ◽  
Markov Jump Systems ◽  
Time Control ◽  
Saturating Actuators ◽  
Finite State ◽  
Jump Systems

We deal with the finite-time control problem for discrete-time Markov jump systems subject to saturating actuators. A finite-state Markovian process is given to govern the transition of the jumping parameters. A controller designed for unconstrained systems combined with a dynamic antiwindup compensator is given to guarantee that the resulting system is mean-square locally asymptotically finite-time stabilizable. The proposed conditions allow us to find dynamic anti-windup compensator which stabilize the closed-loop systems in the finite-time sense. All these conditions can be expressed in the form of linear matrix inequalities and therefore are numerically tractable, as shown in the example included in the paper.

Sign in / Sign up

Export Citation Format

Share Document