scholarly journals Computational models of adaptive behavior and prefrontal cortex

Author(s):  
Alireza Soltani ◽  
Etienne Koechlin

AbstractThe real world is uncertain, and while ever changing, it constantly presents itself in terms of new sets of behavioral options. To attain the flexibility required to tackle these challenges successfully, most mammalian brains are equipped with certain computational abilities that rely on the prefrontal cortex (PFC). By examining learning in terms of internal models associating stimuli, actions, and outcomes, we argue here that adaptive behavior relies on specific interactions between multiple systems including: (1) selective models learning stimulus–action associations through rewards; (2) predictive models learning stimulus- and/or action–outcome associations through statistical inferences anticipating behavioral outcomes; and (3) contextual models learning external cues associated with latent states of the environment. Critically, the PFC combines these internal models by forming task sets to drive behavior and, moreover, constantly evaluates the reliability of actor task sets in predicting external contingencies to switch between task sets or create new ones. We review different models of adaptive behavior to demonstrate how their components map onto this unifying framework and specific PFC regions. Finally, we discuss how our framework may help to better understand the neural computations and the cognitive architecture of PFC regions guiding adaptive behavior.

Author(s):  
Sergio Castellanos ◽  
Luis-Felipe Rodríguez ◽  
J. Octavio Gutierrez-Garcia

Autonomous agents (AAs) are capable of evaluating their environment from an emotional perspective by implementing computational models of emotions (CMEs) in their architecture. A major challenge for CMEs is to integrate the cognitive information projected from the components included in the AA's architecture. In this chapter, a scheme for modulating emotional stimuli using appraisal dimensions is proposed. In particular, the proposed scheme models the influence of cognition on appraisal dimensions by modifying the limits of fuzzy membership functions associated with each dimension. The computational scheme is designed to facilitate, through input and output interfaces, the development of CMEs capable of interacting with cognitive components implemented in a given cognitive architecture of AAs. A proof of concept based on real-world data to provide empirical evidence that indicates that the proposed mechanism can properly modulate the emotional process is carried out.


Author(s):  
Volkan Ustun ◽  
Paul S. Rosenbloom

Realism is required not only for how synthetic characters look but also for how they behave. Many applications, such as simulations, virtual worlds, and video games, require computational models of intelligence that generate realistic and credible behavior for the participating synthetic characters. Sigma (S) is being built as a computational model of general intelligence with a long-term goal of understanding and replicating the architecture of the mind; i.e., the fixed structure underlying intelligent behavior. Sigma leverages probabilistic graphical models towards a uniform grand unification of not only traditional cognitive capabilities but also key non-cognitive aspects, creating unique opportunities for the construction of new kinds of non-modular behavioral models. These ambitions strive for the complete control of synthetic characters that behave as humanly as possible. In this paper, Sigma is introduced along with two disparate proof-of-concept virtual humans – one conversational and the other a pair of ambulatory agents – that demonstrate its diverse capabilities.


2006 ◽  
Vol 18 (2) ◽  
pp. 283-328 ◽  
Author(s):  
Randall C. O'Reilly ◽  
Michael J. Frank

The prefrontal cortex has long been thought to subserve both working memory (the holding of information online for processing) and executive functions (deciding how to manipulate working memory and perform processing). Although many computational models of working memory have been developed, the mechanistic basis of executive function remains elusive, often amounting to a homunculus. This article presents an attempt to deconstruct this homunculus through powerful learning mechanisms that allow a computational model of the prefrontal cortex to control both itself and other brain areas in a strategic, task-appropriate manner. These learning mechanisms are based on subcortical structures in the midbrain, basal ganglia, and amygdala, which together form an actor-critic architecture. The critic system learns which prefrontal representations are task relevant and trains the actor, which in turn provides a dynamic gating mechanism for controlling working memory updating. Computationally, the learning mechanism is designed to simultaneously solve the temporal and structural credit assignment problems. The model's performance compares favorably with standard backpropagation-based temporal learning mechanisms on the challenging 1-2-AX working memory task and other benchmark working memory tasks.


2013 ◽  
Vol 39 (3) ◽  
pp. 538-544 ◽  
Author(s):  
David M Dietz ◽  
Pamela J Kennedy ◽  
HaoSheng Sun ◽  
Ian Maze ◽  
Amy M Gancarz ◽  
...  

Author(s):  
Eva Hudlicka ◽  
Jonathan Pfautz

Although quintessentially human, emotions have, until recently, been largely ignored in the human factors cognitive engineering / decision-making area. This is surprising, as extensive empirical evidence indicates that emotions, and personality traits, influence human perception and decision-making. This is particularly the case in crisis situations, when extreme affective states may arise (e.g., anxiety). The development of more complete and realistic theories of human perception and decision-making, and associated computational models, will require the inclusion of personality and affective considerations. In this paper, we propose an augmented version of the recognition-primed decision-making theory, which takes into consideration trait and state effects on decision-making. We describe a cognitive architecture that implements this theory, and a generic methodology for modeling trait and state effects within this architecture. Following an initial prototype demonstration, the full architecture is currently being implemented in the context of a military peacekeeping scenario.


1998 ◽  
Vol 08 (05n06) ◽  
pp. 525-539
Author(s):  
HOWARD CARD

In this paper the properties of artificial neural network computations by digital VLSI systems are discussed. We also comment on artificial computational models, learning algorithms, and digital implementations of ANNs in general. The analysis applies to regular arrays or processing elements performing binary integer arithmetic at various bit precisions. Computation rates are limited by power dissipation which is dependent upon required precision and packaging constraints such as pinout. They also depend strongly on the minimum feature size of the CMOS technology. Custom digital implementations with low bit precision are emphasized, because these circuits require less power and silicon area. This may be achieved using stochastic arithmetic, with pseudorandom number generation using cellular automata.


2018 ◽  
Vol 140 (2) ◽  
Author(s):  
Ahmet Erdemir ◽  
Peter J. Hunter ◽  
Gerhard A. Holzapfel ◽  
Leslie M. Loew ◽  
John Middleton ◽  
...  

The role of computational modeling for biomechanics research and related clinical care will be increasingly prominent. The biomechanics community has been developing computational models routinely for exploration of the mechanics and mechanobiology of diverse biological structures. As a result, a large array of models, data, and discipline-specific simulation software has emerged to support endeavors in computational biomechanics. Sharing computational models and related data and simulation software has first become a utilitarian interest, and now, it is a necessity. Exchange of models, in support of knowledge exchange provided by scholarly publishing, has important implications. Specifically, model sharing can facilitate assessment of reproducibility in computational biomechanics and can provide an opportunity for repurposing and reuse, and a venue for medical training. The community's desire to investigate biological and biomechanical phenomena crossing multiple systems, scales, and physical domains, also motivates sharing of modeling resources as blending of models developed by domain experts will be a required step for comprehensive simulation studies as well as the enhancement of their rigor and reproducibility. The goal of this paper is to understand current perspectives in the biomechanics community for the sharing of computational models and related resources. Opinions on opportunities, challenges, and pathways to model sharing, particularly as part of the scholarly publishing workflow, were sought. A group of journal editors and a handful of investigators active in computational biomechanics were approached to collect short opinion pieces as a part of a larger effort of the IEEE EMBS Computational Biology and the Physiome Technical Committee to address model reproducibility through publications. A synthesis of these opinion pieces indicates that the community recognizes the necessity and usefulness of model sharing. There is a strong will to facilitate model sharing, and there are corresponding initiatives by the scientific journals. Outside the publishing enterprise, infrastructure to facilitate model sharing in biomechanics exists, and simulation software developers are interested in accommodating the community's needs for sharing of modeling resources. Encouragement for the use of standardized markups, concerns related to quality assurance, acknowledgement of increased burden, and importance of stewardship of resources are noted. In the short-term, it is advisable that the community builds upon recent strategies and experiments with new pathways for continued demonstration of model sharing, its promotion, and its utility. Nonetheless, the need for a long-term strategy to unify approaches in sharing computational models and related resources is acknowledged. Development of a sustainable platform supported by a culture of open model sharing will likely evolve through continued and inclusive discussions bringing all stakeholders at the table, e.g., by possibly establishing a consortium.


2021 ◽  
Author(s):  
Sunandha Srikanth ◽  
Dylan Le ◽  
Yudi Hu ◽  
Jill K Leutgeb ◽  
Stefan Leutgeb

Oscillatory activity is thought to coordinate neural computations across brain regions, and theta oscillations are critical for learning and memory. Because the frequency of respiratory-related oscillations (RROs) in rodents can overlap with the frequency of theta in the prefrontal cortex (PFC) and the hippocampus, we asked whether odor-cued working memory may be supported by coupling between these two oscillations. We first confirmed that RROs are propagated to the hippocampus and PFC and that RRO frequency overlaps with canonical theta frequency. However, we found low coherence between RROs and local theta oscillations in the hippocampus-PFC network when the two types of oscillations overlapped in frequency. This effect was observed during all behavioral phases including during movement and while odors were actively sampled when stationary. Despite the similarity in frequency, RROs and theta oscillations therefore appear to be limited to supporting computation in distinct networks, which suggests that sustained long-range coordination between oscillation patterns that depend on separate pacemakers is not necessary to support at least one type of working memory.


2019 ◽  
Author(s):  
Charles Findling ◽  
Nicolas Chopin ◽  
Etienne Koechlin

AbstractEveryday life features uncertain and ever-changing situations. In such environments, optimal adaptive behavior requires higher-order inferential capabilities to grasp the volatility of external contingencies. These capabilities however involve complex and rapidly intractable computations, so that we poorly understand how humans develop efficient adaptive behaviors in such environments. Here we demonstrate this counterintuitive result: simple, low-level inferential processes involving imprecise computations conforming to the psychophysical Weber Law actually lead to near-optimal adaptive behavior, regardless of the environment volatility. Using volatile experimental settings, we further show that such imprecise, low-level inferential processes accounted for observed human adaptive performances, unlike optimal adaptive models involving higher-order inferential capabilities, their biologically more plausible, algorithmic approximations and non-inferential adaptive models like reinforcement learning. Thus, minimal inferential capabilities may have evolved along with imprecise neural computations as contributing to near-optimal adaptive behavior in real-life environments, while leading humans to make suboptimal choices in canonical decision-making tasks.


F1000Research ◽  
2019 ◽  
Vol 8 ◽  
pp. 1292 ◽  
Author(s):  
Ersin Yavas ◽  
Sarah Gonzalez ◽  
Michael S. Fanselow

One of the guiding principles of memory research in the preceding decades is multiple memory systems theory, which links specific task demands to specific anatomical structures and circuits that are thought to act orthogonally with respect to each other. We argue that this view does not capture the nature of learning and memory when any degree of complexity is introduced. In most situations, memory requires interactions between these circuits and they can act in a facilitative manner to generate adaptive behavior.


Sign in / Sign up

Export Citation Format

Share Document