scholarly journals Adaptive importance sampling of random walks on continuous state spaces

1998 ◽  
Author(s):  
K. Baggerly ◽  
D. Cox ◽  
R. Picard
Keyword(s):  
Random Walks ◽  
Importance Sampling ◽  
State Spaces ◽  
Adaptive Importance Sampling ◽  
Continuous State

Exponential convergence of adaptive importance sampling for Markov chains

2000 ◽  
Vol 37 (2) ◽  
pp. 342-358 ◽  
Author(s):  
Keith Baggerly ◽  
Dennis Cox ◽  
Rick Picard
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Importance Sampling ◽  
Exponential Convergence ◽  
State Spaces ◽  
Adaptive Importance Sampling ◽  
Finite State ◽  
The Mean ◽  
General Conditions

We consider adaptive importance sampling for a Markov chain with scoring. It is shown that convergence to the zero-variance importance sampling chain for the mean total score occurs exponentially fast under general conditions. These results extend previous work in Kollman (1993) and in Kollman et al. (1999) for finite state spaces.

Exponential convergence of adaptive importance sampling for Markov chains

2000 ◽  
Vol 37 (02) ◽  
pp. 342-358 ◽  
Author(s):  
Keith Baggerly ◽  
Dennis Cox ◽  
Rick Picard
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Importance Sampling ◽  
Exponential Convergence ◽  
State Spaces ◽  
Adaptive Importance Sampling ◽  
Finite State ◽  
The Mean ◽  
General Conditions

We consider adaptive importance sampling for a Markov chain with scoring. It is shown that convergence to the zero-variance importance sampling chain for the mean total score occurs exponentially fast under general conditions. These results extend previous work in Kollman (1993) and in Kollman et al. (1999) for finite state spaces.

On coupling of random walks and renewal processes

1996 ◽  
Vol 33 (1) ◽  
pp. 122-126
Author(s):  
Torgny Lindvall ◽  
L. C. G. Rogers
Keyword(s):  
Random Walk ◽  
Random Walks ◽  
Simple Random Walk ◽  
Renewal Processes ◽  
Renewal Theorem ◽  
One Dimensional ◽  
Continuous State Space ◽  
Continuous State ◽  
Efficient Coupling ◽  
Blackwell's Renewal Theorem

The use of Mineka coupling is extended to a case with a continuous state space: an efficient coupling of random walks S and S' in can be made such that S' — S is virtually a one-dimensional simple random walk. This insight settles a zero-two law of ergodicity. One more proof of Blackwell's renewal theorem is also presented.

scholarly journals A New Improved Penalty Avoiding Rational Policy Making Algorithm for Keepaway with Continuous State Spaces

2009 ◽  
Vol 13 (6) ◽  
pp. 675-682 ◽  
Author(s):  
Takuji Watanabe ◽  
◽  
Kazuteru Miyazaki ◽  
Hiroaki Kobayashi ◽  
◽  
...  
Keyword(s):  
Basis Function ◽  
Function Approximation ◽  
Policy Making ◽  
Basis Functions ◽  
State Spaces ◽  
Soccer Game ◽  
Continuous State ◽  
Current Input ◽  
Multiagent Environments ◽  
Parp 1

The penalty avoiding rational policy making algorithm (PARP) [1] previously improved to save memory and cope with uncertainty, i.e., IPARP [2], requires that states be discretized in real environments with continuous state spaces, using function approximation or some other method. Especially, in PARP, a method that discretizes state using a basis functions is known [3]. Because this creates a new basis function based on the current input and its next observation, however, an unsuitable basis function may be generated in some asynchronous multiagent environments. We therefore propose a uniform basis function and range extent of the basis function is estimated before learning. We show the effectiveness of our proposal using a soccer game task called “Keepaway.”

Sign in / Sign up

Export Citation Format

Share Document