scholarly journals Serial dependence of a Markov process

1965 ◽  
Vol 5 (3) ◽  
pp. 299-314 ◽  
Author(s):  
B. D. Craven
Keyword(s):  
Time Series ◽  
Markov Process ◽  
State Space ◽  
Discrete Time ◽  
Borel Subset ◽  
Serial Dependence ◽  
State Spaces ◽  
Infinite Dimensional ◽  
Continuous State Space ◽  
Continuous State

Consider a Markov process defined in discrete time t = 1, 2, 3, hellip on a state space S. The state of the Process at time time t will be specifies by a random varable Vt, taking values in S. This paper presents some results concerning the behaviour of the saquence V1, V2, V3hellip, considered as a time series. In general, S will be assumed to be a Borel subset of an h-dimensional Euclideam space, where h is finite. The results apply, in particular, to a continuous state space, taking S to be an interval of the realine, or to discrete process having finitely or enumerably many states. Certain results, which are indicated in what follows, apply also to more general (infinite-dimensional) state spaces.

Asymptotic behaviour of continuous time, continuous state-space branching processes

1974 ◽  
Vol 11 (04) ◽  
pp. 669-677 ◽  
Author(s):  
D. R. Grey
Keyword(s):  
Asymptotic Behaviour ◽  
State Space ◽  
Discrete Time ◽  
Continuous Time ◽  
Branching Processes ◽  
Discrete State ◽  
Space And Time ◽  
Continuous State Space ◽  
Continuous State ◽  
Discrete State Space

Results on the behaviour of Markov branching processes as time goes to infinity, hitherto obtained for models which assume a discrete state-space or discrete time or both, are here generalised to a model with both state-space and time continuous. The results are similar but the methods not always so.

scholarly journals Jump Markov models and transition state theory: the quasi-stationary distribution approach

2016 ◽  
Vol 195 ◽  
pp. 469-495 ◽  
Author(s):  
Giacomo Di Gesù ◽  
Tony Lelièvre ◽  
Dorian Le Peutrec ◽  
Boris Nectoux
Keyword(s):  
Markov Process ◽  
Transition State ◽  
State Space ◽  
Stationary Distribution ◽  
Transition State Theory ◽  
State Theory ◽  
Jump Markov Process ◽  
Continuous State Space ◽  
Continuous State ◽  
Quasi Stationary Distribution

We are interested in the connection between a metastable continuous state space Markov process (satisfyinge.g.the Langevin or overdamped Langevin equation) and a jump Markov process in a discrete state space. More precisely, we use the notion of quasi-stationary distribution within a metastable state for the continuous state space Markov process to parametrize the exit event from the state. This approach is useful to analyze and justify methods which use the jump Markov process underlying a metastable dynamics as a support to efficiently sample the state-to-state dynamics (accelerated dynamics techniques). Moreover, it is possible by this approach to quantify the error on the exit event when the parametrization of the jump Markov model is based on the Eyring–Kramers formula. This therefore provides a mathematical framework to justify the use of transition state theory and the Eyring–Kramers formula to build kinetic Monte Carlo or Markov state models.

Asymptotic behaviour of continuous time, continuous state-space branching processes

1974 ◽  
Vol 11 (4) ◽  
pp. 669-677 ◽  
Author(s):  
D. R. Grey
Keyword(s):  
Asymptotic Behaviour ◽  
State Space ◽  
Discrete Time ◽  
Continuous Time ◽  
Branching Processes ◽  
Discrete State ◽  
Space And Time ◽  
Continuous State Space ◽  
Continuous State ◽  
Discrete State Space

Results on the behaviour of Markov branching processes as time goes to infinity, hitherto obtained for models which assume a discrete state-space or discrete time or both, are here generalised to a model with both state-space and time continuous. The results are similar but the methods not always so.

scholarly journals A limit theorem for Markov chains with continuous state space

1963 ◽  
Vol 3 (3) ◽  
pp. 351-358 ◽  
Author(s):  
P. D. Finch
Keyword(s):  
State Space ◽  
Transition Probabilities ◽  
Borel Subset ◽  
Homogeneous Markov Chain ◽  
Continuous State Space ◽  
Continuous State ◽  
One Step ◽  
Step Transition ◽  
Image Position ◽  
The Right

Let R denote the set of real numbers, B the σ-field of all Borel subsets of R. A homogeneous Markov Chain with state space a Borel subset Ω of R is a sequence {an}, n≧ 0, of random variables, taking values in Ω, with one-step transition probabilities P(1) (ξ, A) defined by for each choice of ξ, ξ0, …, ξn−1 in ω and all Borel subsets A of ω The fact that the right-hand side of (1.1) does not depend on the ξi, 0 ≧ i > n, is of course the Markovian property, the non-dependence on n is the homogeneity of the chain.

scholarly journals Reinforcement learning versus swarm intelligence for autonomous multi-HAPS coordination

2021 ◽  
Vol 3 (6) ◽  
Author(s):  
Ogbonnaya Anicho ◽  
Philip B. Charlesworth ◽  
Gurvinder S. Baicher ◽  
Atulya K. Nagar
Keyword(s):  
Reinforcement Learning ◽  
State Space ◽  
Swarm Intelligence ◽  
Performance Indicators ◽  
Convergence Rates ◽  
Tuning Parameters ◽  
Continuous State Space ◽  
Continuous State ◽  
User Coverage ◽  
Better Than

AbstractThis work analyses the performance of Reinforcement Learning (RL) versus Swarm Intelligence (SI) for coordinating multiple unmanned High Altitude Platform Stations (HAPS) for communications area coverage. It builds upon previous work which looked at various elements of both algorithms. The main aim of this paper is to address the continuous state-space challenge within this work by using partitioning to manage the high dimensionality problem. This enabled comparing the performance of the classical cases of both RL and SI establishing a baseline for future comparisons of improved versions. From previous work, SI was observed to perform better across various key performance indicators. However, after tuning parameters and empirically choosing suitable partitioning ratio for the RL state space, it was observed that the SI algorithm still maintained superior coordination capability by achieving higher mean overall user coverage (about 20% better than the RL algorithm), in addition to faster convergence rates. Though the RL technique showed better average peak user coverage, the unpredictable coverage dip was a key weakness, making SI a more suitable algorithm within the context of this work.

scholarly journals Instantaneous brain dynamics mapped to a continuous state space

NeuroImage ◽  
2017 ◽  
Vol 162 ◽  
pp. 344-352 ◽  
Author(s):  
Jacob C.W. Billings ◽  
Alessio Medda ◽  
Sadia Shakil ◽  
Xiaohong Shen ◽  
Amrit Kashyap ◽  
...  
Keyword(s):  
State Space ◽  
Brain Dynamics ◽  
Continuous State Space ◽  
Continuous State
Sign in / Sign up

Export Citation Format

Share Document