scholarly journals Online Probabilistic Goal Recognition over Nominal Models

2019 ◽  
Author(s):  
Ramon Fraga Pereira ◽  
Mor Vered ◽  
Felipe Meneguzzi ◽  
Miquel Ramírez
Keyword(s):  
Expert Knowledge ◽  
Error Rates ◽  
Prediction Errors ◽  
Model Learning ◽  
Dynamic Constraints ◽  
Posterior Probability Distribution ◽  
Model Based ◽  
Finite Set ◽  
Domain Models ◽  
Goal Recognition

This paper revisits probabilistic, model-based goal recognition to study the implications of the use of nominal models to estimate the posterior probability distribution over a finite set of hypothetical goals. Existing model-based approaches rely on expert knowledge to produce symbolic descriptions of the dynamic constraints domain objects are subject to, and these are assumed to produce correct predictions. We abandon this assumption to consider the use of nominal models that are learnt from observations on transitions of systems with unknown dynamics. Leveraging existing work on the acquisition of domain models via learning for Hybrid Planning we adapt and evaluate existing goal recognition approaches to analyze how prediction errors, inherent to system dynamics identification and model learning techniques have an impact over recognition error rates.

scholarly journals Prefrontal solution to the bias-variance tradeoff during reinforcement learning

2020 ◽  
Author(s):  
Dongjae Kim ◽  
Jaeseung Jeong ◽  
Sang Wan Lee
Keyword(s):  
Adaptive Control ◽  
Reinforcement Learning ◽  
Prediction Error ◽  
Brain Regions ◽  
Decision Task ◽  
Prediction Errors ◽  
Model Based ◽  
Model Free ◽  
Bias Variance ◽  
The Brain

AbstractThe goal of learning is to maximize future rewards by minimizing prediction errors. Evidence have shown that the brain achieves this by combining model-based and model-free learning. However, the prediction error minimization is challenged by a bias-variance tradeoff, which imposes constraints on each strategy’s performance. We provide new theoretical insight into how this tradeoff can be resolved through the adaptive control of model-based and model-free learning. The theory predicts the baseline correction for prediction error reduces the lower bound of the bias–variance error by factoring out irreducible noise. Using a Markov decision task with context changes, we showed behavioral evidence of adaptive control. Model-based behavioral analyses show that the prediction error baseline signals context changes to improve adaptability. Critically, the neural results support this view, demonstrating multiplexed representations of prediction error baseline within the ventrolateral and ventromedial prefrontal cortex, key brain regions known to guide model-based and model-free learning.One sentence summaryA theoretical, behavioral, computational, and neural account of how the brain resolves the bias-variance tradeoff during reinforcement learning is described.

scholarly journals A Case Studies Approach to the Analysis of Profiling and Framing Structures for Pervasive Information Systems

2012 ◽  
Vol 4 (2) ◽  
pp. 1-18 ◽  
Author(s):  
José Eduardo Fernandes ◽  
Ricardo J. Machado ◽  
João Á. Carvalho
Keyword(s):  
Information Systems ◽  
Case Studies ◽  
Software Design ◽  
Expert Knowledge ◽  
Structural Approach ◽  
Technological Changes ◽  
Design And Development ◽  
Structure Approach ◽  
Model Based ◽  
Widespread Adoption

Model-Based/Driven Development (MDD) constitutes an approach to software design and development that potentially contributes to: concepts closer to domain and reduction of semantic gaps, automation and less sensitivity to technological changes, and the capture of expert knowledge and reuse. The widespread adoption of pervasive technologies as basis for new systems and applications lead to the need of effectively design pervasive information systems that properly fulfil the goals they were designed for. This paper presents a profiling and framing structure approach for the development of Pervasive Information Systems (PIS). This profiling and framing structure allows the organization of the functionality that can be assigned to computational devices in a system and of the corresponding development structures and models, being. The proposed approach enables a structural approach to PIS development. The paper also presents two case studies that allowed demonstrating the applicability of the approach.

Methodology for a Partly Automated Parameter Identification for the Validation of Multi-Domain Models

2014 ◽  
Author(s):  
D. Kruse ◽  
C. Schweers ◽  
A. Trächtler
Keyword(s):  
Parameter Identification ◽  
Domain Model ◽  
Domain Expert ◽  
Model Based ◽  
New Approaches ◽  
Domain Models ◽  
Identification Tool ◽  
The Way

The paper presents a methodology for a partly automated parameter identification that is to validate multi-domain models. To this end an identification tool under MATLAB has been developed. It enables a partly automated procedure that uses established methods to identify parameters from complex, nonlinear multi-domain models. In order to integrate such multi-domain models into the tool, an interface based on the Functional Mock-up Interface (FMI) standard can be used. The interface makes the required identification parameters from the multi-domain model automatically available to the identification tool. Additionally a guideline is developed which describes the way in which the respective domain expert has to mark the required identification parameters during modeling. The needs for this methodology as well as its application are shown by a practical example from the industry, using Dymola, the FMI-standard, and MATLAB. The practical example deals with the model-based development of a new washing procedure. The paper presents a partly automated parameter identification for the validation of the absorption part of the multi-domain model. Besides, new approaches to the modelling of this kind of absorption effects will be detailed.

scholarly journals Model-Based Influences on Humans' Choices and Striatal Prediction Errors

Neuron ◽  
2011 ◽  
Vol 69 (6) ◽  
pp. 1204-1215 ◽  
Author(s):  
Nathaniel D. Daw ◽  
Samuel J. Gershman ◽  
Ben Seymour ◽  
Peter Dayan ◽  
Raymond J. Dolan
Keyword(s):  
Prediction Errors ◽  
Model Based

Multiple Model-Based Reinforcement Learning

2002 ◽  
Vol 14 (6) ◽  
pp. 1347-1369 ◽  
Author(s):  
Kenji Doya ◽  
Kazuyuki Samejima ◽  
Ken-ichi Katagiri ◽  
Mitsuo Kawato
Keyword(s):  
Reinforcement Learning ◽  
Prediction Models ◽  
Physical Parameters ◽  
Prediction Errors ◽  
Multiple Model ◽  
Model Based ◽  
Continuous State ◽  
Finite State ◽  
Multiple Domains ◽  
State Case

We propose a modular reinforcement learning architecture for nonlinear, nonstationary control tasks, which we call multiple model-based reinforcement learning (MMRL). The basic idea is to decompose a complex task into multiple domains in space and time based on the predictability of the environmental dynamics. The system is composed of multiple modules, each of which consists of a state prediction model and a reinforcement learning controller. The “responsibility signal,” which is given by the softmax function of the prediction errors, is used to weight the outputs of multiple modules, as well as to gate the learning of the prediction models and the reinforcement learning controllers. We formulate MMRL for both discrete-time, finite-state case and continuous-time, continuous-state case. The performance of MMRL was demonstrated for discrete case in a nonstationary hunting task in a grid world and for continuous case in a nonlinear, nonstationary control task of swinging up a pendulum with variable physical parameters.

Sign in / Sign up

Export Citation Format

Share Document