scholarly journals Model-free, Model-based, and General Intelligence

2018 ◽  
Author(s):  
Hector Geffner
Keyword(s):  
Intelligent Systems ◽  
Human Mind ◽  
Writing Programs ◽  
Scalable Algorithms ◽  
Model Based ◽  
Model Free ◽  
Considerable Success ◽  
Black Boxes ◽  
Free Model ◽  
Intelligent Behavior

During the 60s and 70s, AI researchers explored intuitions about intelligence by writing programs that displayed intelligent behavior. Many good ideas came out from this work but programs written by hand were not robust or general. After the 80s, research increasingly shifted to the development of learners capable of inferring behavior and functions from experience and data, and solvers capable of tackling well-defined but intractable models like SAT, classical planning, Bayesian networks, and POMDPs. The learning approach has achieved considerable success but results in black boxes that do not have the flexibility, transparency, and generality of their model-based counterparts. Model-based approaches, on the other hand, require models and scalable algorithms. Model-free learners and model-based solvers have indeed close parallels with Systems 1 and 2 in current theories of the human mind: the first, a fast, opaque, and inflexible intuitive mind; the second, a slow, transparent, and flexible analytical mind. In this paper, I review developments in AI and draw on these theories to discuss the gap between model-free learners and model-based solvers, a gap that needs to be bridged in order to have intelligent systems that are robust and general.

Fault Recognition in the Presence of Error With NURBS-Based Metamodels

2009 ◽  
Author(s):  
Cameron J. Turner
Keyword(s):  
Theoretical Foundation ◽  
Condition Based Maintenance ◽  
On Demand ◽  
Model Based ◽  
Sources Of Error ◽  
Model Free ◽  
Fault Recognition ◽  
Simulated System ◽  
Free Model ◽  
The Impact

Condition-based maintenance (CBM) offers the possibility of replacing the predominant maintenance-as-scheduled paradigm with a maintenance-on-demand paradigm. In all CBM algorithms, faults must first be recognized, then characterized and finally reconciled. Multiple CBM methods have been proposed, including model-free, model-based and metamodel-based methods. However, the signals from real systems are obscured by sources of error. This research examines the impact of error upon a metamodel-based CBM approach using a simulated system to reveal the significance of error in the all-important step of fault recognition. The use of a simulated system allows control of the type and magnitude of both the error and of the fault signals allowing their significance to be evaluated. As a result of this research, a stronger theoretical foundation metamodel-based CBM techniques is established and several promising behaviors are identified.

scholarly journals Model-free and model-based learning processes in the updating of explicit and implicit evaluations

2019 ◽  
Vol 116 (13) ◽  
pp. 6035-6044 ◽  
Author(s):  
Benedek Kurdi ◽  
Samuel J. Gershman ◽  
Mahzarin R. Banaji
Keyword(s):  
Measurement Techniques ◽  
Human Mind ◽  
Dual Process ◽  
Model Based ◽  
Model Free ◽  
Produce Change ◽  
Single Process ◽  
Novel Targets ◽  
Probabilistic Nature

Evaluating stimuli along a good–bad dimension is a fundamental computation performed by the human mind. In recent decades, research has documented dissociations and associations between explicit (i.e., self-reported) and implicit (i.e., indirectly measured) forms of evaluations. However, it is unclear whether such dissociations arise from relatively more superficial differences in measurement techniques or from deeper differences in the processes by which explicit and implicit evaluations are acquired and represented. The present project (totalN= 2,354) relies on the computationally well-specified distinction between model-based and model-free reinforcement learning to investigate the unique and shared aspects of explicit and implicit evaluations. Study 1 used a revaluation procedure to reveal that, whereas explicit evaluations of novel targets are updated via model-free and model-based processes, implicit evaluations depend on the former but are impervious to the latter. Studies 2 and 3 demonstrated the robustness of this effect to (i) the number of stimulus exposures in the revaluation phase and (ii) the deterministic vs. probabilistic nature of initial reinforcement. These findings provide a framework, going beyond traditional dual-process and single-process accounts, to highlight the context-sensitivity and long-term recalcitrance of implicit evaluations as well as variations in their relationship with their explicit counterparts. These results also suggest avenues for designing theoretically guided interventions to produce change in implicit evaluations.

Representation, abstraction, and simple-minded sophisticates

2020 ◽  
Vol 43 ◽  
Author(s):  
Peter Dayan
Keyword(s):  
Decision Theory ◽  
Predictive Coding ◽  
Bayesian Decision Theory ◽  
Bayesian Decision ◽  
Model Based ◽  
Model Free

Abstract Bayesian decision theory provides a simple formal elucidation of some of the ways that representation and representational abstraction are involved with, and exploit, both prediction and its rather distant cousin, predictive coding. Both model-free and model-based methods are involved.

Model-Based and Model-Free Social Cognition

2019 ◽  
Author(s):  
Leor M Hackel ◽  
Jeffrey Jordan Berg ◽  
Björn Lindström ◽  
David Amodio
Keyword(s):  
Reinforcement Learning ◽  
Social Cognition ◽  
Learning Strategies ◽  
Memory Systems ◽  
Learning Task ◽  
Financial Advisors ◽  
Model Based ◽  
Model Free ◽  
Systems Model ◽  
Task Assessment

Do habits play a role in our social impressions? To investigate the contribution of habits to the formation of social attitudes, we examined the roles of model-free and model-based reinforcement learning in social interactions—computations linked in past work to habit and planning, respectively. Participants in this study learned about novel individuals in a sequential reinforcement learning paradigm, choosing financial advisors who led them to high- or low-paying stocks. Results indicated that participants relied on both model-based and model-free learning, such that each independently predicted choice during the learning task and self-reported liking in a post-task assessment. Specifically, participants liked advisors who could provide large future rewards as well as advisors who had provided them with large rewards in the past. Moreover, participants varied in their use of model-based and model-free learning strategies, and this individual difference influenced the way in which learning related to self-reported attitudes: among participants who relied more on model-free learning, model-free social learning related more to post-task attitudes. We discuss implications for attitudes, trait impressions, and social behavior, as well as the role of habits in a memory systems model of social cognition.

scholarly journals Author Correction: Reliance on model-based and model-free control in obesity

2021 ◽  
Vol 11 (1) ◽  
Author(s):  
Lieneke K. Janssen ◽  
Florian P. Mahner ◽  
Florian Schlagenhauf ◽  
Lorenz Deserno ◽  
Annette Horstmann
Keyword(s):  
Model Based ◽  
Model Free ◽  
Model Free Control

An amendment to this paper has been published and can be accessed via a link at the top of the paper.

Model-Based and Model-Free Control of DC-DC Converters with High-Order Dynamics and Limited Measurements

2020 ◽  
pp. 1-1
Author(s):  
Javier Loranca ◽  
Jonathan Carlos Mayo Maldonado ◽  
Gerardo Escobar ◽  
Carlos Villarreal-Hernandez ◽  
Thabiso Maupong ◽  
...  
Keyword(s):  
High Order ◽  
Model Based ◽  
Model Free ◽  
Model Free Control
Sign in / Sign up

Export Citation Format

Share Document