Solving Partially Observable Reinforcement Learning Problems with Recurrent Neural Networks

2012 ◽  
pp. 709-733 ◽  
Author(s):  
Siegmund Duell ◽  
Steffen Udluft ◽  
Volkmar Sterzing
Keyword(s):  
Neural Networks ◽  
Reinforcement Learning ◽  
Recurrent Neural Networks ◽  
Learning Problems ◽  
Partially Observable

scholarly journals Multi-Zone Unit for Recurrent Neural Networks

2020 ◽  
Vol 34 (04) ◽  
pp. 5150-5157
Author(s):  
Fandong Meng ◽  
Jinchao Zhang ◽  
Yang Liu ◽  
Jie Zhou
Keyword(s):  
Neural Networks ◽  
Recurrent Neural Networks ◽  
Critical Role ◽  
Transition Function ◽  
Learning Problems ◽  
Arbitrary Length ◽  
Transition Functions ◽  
Multiple Datasets ◽  
Analysis Task ◽  
Hidden States

Recurrent neural networks (RNNs) have been widely used to deal with sequence learning problems. The input-dependent transition function, which folds new observations into hidden states to sequentially construct fixed-length representations of arbitrary-length sequences, plays a critical role in RNNs. Based on single space composition, transition functions in existing RNNs often have difficulty in capturing complicated long-range dependencies. In this paper, we introduce a new Multi-zone Unit (MZU) for RNNs. The key idea is to design a transition function that is capable of modeling multiple space composition. The MZU consists of three components: zone generation, zone composition, and zone aggregation. Experimental results on multiple datasets of the character-level language modeling task and the aspect-based sentiment analysis task demonstrate the superiority of the MZU.

ANALYTICAL REVIEW OF MULTI-AGENT REINFORCEMENT LEARNING PROBLEMS

2020 ◽  
pp. 48-56
Author(s):  
Yu. V. Dubenko
Keyword(s):  
Reinforcement Learning ◽  
Intelligent Agents ◽  
Russian Language ◽  
Learning Problems ◽  
Multi Agent Systems ◽  
Hierarchical Reinforcement Learning ◽  
Collective Interaction ◽  
Analytical Review ◽  
Multi Agent ◽  
Partially Observable

This paper is devoted to the problem of collective artificial intelligence in solving problems by intelligent agents in external environments. The environments may be: fully or partially observable, deterministic or stochastic, static or dynamic, discrete or continuous. The paper identifies problems of collective interaction of intelligent agents when they solve a class of tasks, which need to coordinate actions of agent group, e. g. task of exploring the territory of a complex infrastructure facility. It is revealed that the problem of reinforcement training in multi-agent systems is poorly presented in the press, especially in Russian-language publications. The article analyzes reinforcement learning, describes hierarchical reinforcement learning, presents basic methods to implement reinforcement learning. The concept of macro-action by agents integrated in groups is introduced. The main problems of intelligent agents collective interaction for problem solving (i. e. calculation of individual rewards for each agent; agent coordination issues; application of macro actions by agents integrated into groups; exchange of experience generated by various agents as part of solving a collective problem) are identified. The model of multi-agent reinforcement learning is described in details. The article describes problems of this approach building on existing solutions. Basic problems of multi-agent reinforcement learning are formulated in conclusion.

A Monte Carlo EM Approach for Partially Observable Diffusion Processes: Theory and Applications to Neural Networks

2002 ◽  
Vol 14 (7) ◽  
pp. 1507-1544 ◽  
Author(s):  
Javier R. Movellan ◽  
Paul Mineiro ◽  
R. J. Williams
Keyword(s):  
Neural Networks ◽  
Monte Carlo ◽  
Speech Recognition ◽  
Recurrent Neural Networks ◽  
Diffusion Processes ◽  
Visual Speech ◽  
Stochastic Version ◽  
Inner Products ◽  
Partially Observable ◽  
Monte Carlo Em

We present a Monte Carlo approach for training partially observable diffusion processes. We apply the approach to diffusion networks, a stochastic version of continuous recurrent neural networks. The approach is aimed at learning probability distributions of continuous paths, not just expected values. Interestingly, the relevant activation statistics used by the learning rule presented here are inner products in the Hilbert space of square integrable functions. These inner products can be computed using Hebbian operations and do not require backpropagation of error signals. Moreover, standard kernel methods could potentially be applied to compute such inner products. We propose that the main reason that recurrent neural networks have not worked well in engineering applications (e.g., speech recognition) is that they implicitly rely on a very simplistic likelihood model. The diffusion network approach proposed here is much richer and may open new avenues for applications of recurrent neural networks. We present some analysis and simulations to support this view. Very encouraging results were obtained on a visual speech recognition task in which neural networks outperformed hidden Markov models.

scholarly journals Local stability conditions for discrete-time cascade locally recurrent neural networks

2010 ◽  
Vol 20 (1) ◽  
pp. 23-34 ◽  
Author(s):  
Krzysztof Patan
Keyword(s):  
Neural Networks ◽  
Discrete Time ◽  
Recurrent Neural Networks ◽  
Local Stability ◽  
Learning Problems ◽  
Stability Conditions ◽  
Neuron Models ◽  
Optimization Task ◽  
Dynamic Type ◽  
Locally Recurrent

Local stability conditions for discrete-time cascade locally recurrent neural networksThe paper deals with a specific kind of discrete-time recurrent neural network designed with dynamic neuron models. Dynamics are reproduced within each single neuron, hence the network considered is a locally recurrent globally feedforward. A crucial problem with neural networks of the dynamic type is stability as well as stabilization in learning problems. The paper formulates local stability conditions for the analysed class of neural networks using Lyapunov's first method. Moreover, a stabilization problem is defined and solved as a constrained optimization task. In order to tackle this problem, a gradient projection method is adopted. The efficiency and usefulness of the proposed approach are justified by using a number of experiments.

Sign in / Sign up

Export Citation Format

Share Document