scholarly journals Approximating High-dimensional Dynamic Models: Sieve Value Function Iteration

Author(s):  
Peter Arcidiacono ◽  
Patrick Bayer ◽  
Federico A. Bugni ◽  
Jonathan James
2012 ◽  
Author(s):  
Peter Arcidiacono ◽  
Patrick Bayer ◽  
Federico Bugni ◽  
Jonathan James

Automatica ◽  
2012 ◽  
Vol 48 (11) ◽  
pp. 2740-2749 ◽  
Author(s):  
Simona Dobre ◽  
Thierry Bastogne ◽  
Christophe Profeta ◽  
Muriel Barberi-Heyob ◽  
Alain Richard

2007 ◽  
Vol 05 (01) ◽  
pp. 31-46 ◽  
Author(s):  
OLLI HAAVISTO ◽  
HEIKKI HYÖTYNIEMI ◽  
CHRISTOPHE ROOS

Combined interaction of all the genes forms a central part of the functional system of a cell. Thus, especially the data-based modeling of the gene expression network is currently one of the main challenges in the field of systems biology. However, the problem is an extremely high-dimensional and complex one, so that normal identification methods are usually not applicable specially if aiming at dynamic models. We propose in this paper a subspace identification approach, which is well suited for high-dimensional system modeling and the presented modified version can also handle the underdetermined case with less data samples than variables (genes). The algorithm is applied to two public stress-response data sets collected from yeast Saccharomyces cerevisiae. The obtained dynamic state space model is tested by comparing the simulation results with the measured data. It is shown that the identified model can relatively well describe the dynamics of the general stress-related changes in the expression of the complete yeast genome. However, it seems inevitable that more precise modeling of the dynamics of the whole genome would require experiments especially designed for systemic modeling.


2013 ◽  
Vol 756-759 ◽  
pp. 3967-3971
Author(s):  
Bo Yan Ren ◽  
Zheng Qin ◽  
Feng Fei Zhao

Linear value function approximation with binary features is important in the research of Reinforcement Learning (RL). When updating the value function, it is necessary to generate a feature vector which contains the features that should be updated. In high dimensional domains, the generation process will take lot more time, which reduces the performance of algorithm a lot. Hence, this paper introduces Optional Feature Vector Generation (OFVG) algorithm as an improved method to generate feature vectors that can be combined with any online, value-based RL method that uses and expands binary features. This paper shows empirically that OFVG performs well in high dimensional domains.


2015 ◽  
Vol 20 (7) ◽  
pp. 1850-1872 ◽  
Author(s):  
Richard Dennis ◽  
Tatiana Kirsanova

Time inconsistency is an essential feature of many policy problems. This paper presents and compares three methods for computing Markov-perfect optimal policies in stochastic nonlinear business cycle models. The methods considered include value function iteration, generalized Euler equations, and parameterized shadow prices. In the context of a business cycle model in which a fiscal authority chooses government spending and income taxation optimally, although lacking the ability to commit, we show that the solutions obtained using value function iteration and generalized Euler equations are somewhat more accurate than that obtained using parameterized shadow prices. Among these three methods, we show that value function iteration can be applied easily, even to environments that include a risk-sensitive fiscal authority and/or inequality constraints on government spending. We show that the risk-sensitive fiscal authority lowers government spending and income taxation, reducing the disincentive to accumulate wealth that households face.


Author(s):  
Karel Horák ◽  
Branislav Bošanský ◽  
Christopher Kiekintveld ◽  
Charles Kamhoua

Value methods for solving stochastic games with partial observability model the uncertainty of the players as a probability distribution over possible states, where the dimension of the belief space is the number of states. For many practical problems, there are exponentially many states which causes scalability problems. We propose an abstraction technique that addresses this curse of dimensionality by projecting the high-dimensional beliefs onto characteristic vectors of significantly lower dimension (e.g., marginal probabilities). Our main contributions are (1) a novel compact representation of the uncertainty in partially observable stochastic games and (2) a novel algorithm using this representation that is based on existing state-of-the-art algorithms for solving stochastic games with partial observability. Experimental evaluation confirms that the new algorithm using the compact representation dramatically increases scalability compared to the state of the art.


Sign in / Sign up

Export Citation Format

Share Document