scholarly journals Generalizing to generalize: when (and when not) to be compositional in task structure learning

2019 ◽  
Author(s):  
Nicholas T. Franklin ◽  
Michael J. Frank
Keyword(s):  
National Science Foundation ◽  
Structure Learning ◽  
Human Subjects ◽  
Empirical Studies ◽  
Theoretical Work ◽  
State Transitions ◽  
Task Structure ◽  
Learned Behavior ◽  
Learning Agent ◽  
Author Note

AbstractHumans routinely face novel environments in which they have to generalize in order toact adaptively. However, doing so involves the non-trivial challenge of deciding which aspects of a task domain to generalize. While it is sometimes appropriate to simply re-use a learned behavior, often adaptive generalization entails recombining distinct components of knowledge acquired across multiple contexts. Theoretical work has suggested a computational trade-off in which it can be more or less useful to learn and generalize aspects of task structure jointly or compositionally, depending on previous task statistics, but empirical studies are lacking. Here we develop a series of navigation tasks which manipulate the statistics of goal values (“what to do”) and state transitions (“how to do it”) across contexts, and assess whether human subjects generalize these task components separately or conjunctively. We find that human generalization is sensitive to the statistics of the previously experienced task domain, favoring compositional or conjunctive generalization when the task statistics are indicative of such structures, and a mixture of the two when they are more ambiguous. These results support the predictions of a normative “meta-generalization learning” agent that does not only generalize previous knowledge but also generalizes the statistical structure most likely to support generalization.Author NoteThis work was supported in part by the National Science Foundation Proposal 1460604 “How Prefrontal Cortex Augments Reinforcement Learning” to MJF. We thank Mark Ho for providing code used in the behavioral task. We thank Matt Nassar for helpful discussions. Correspondence should be addressed to Nicholas T. Franklin ([email protected]) or Michael J. Frank ([email protected]).

scholarly journals Compositional clustering in task structure learning

2017 ◽  
Author(s):  
Nicholas Franklin ◽  
Michael J. Frank
Keyword(s):  
Computational Models ◽  
Structure Learning ◽  
Transition Function ◽  
Task Structure ◽  
Reward Function ◽  
Transition Functions ◽  
Learning Agent ◽  
Many Instruments ◽  
Reward Functions ◽  
New Policies

AbstractHumans are remarkably adept at generalizing knowledge between experiences in a way that can be difficult for computers. Often, this entails generalizing constituent pieces of experiences that do not fully overlap, but nonetheless share useful similarities with, previously acquired knowledge. However, it is often unclear how knowledge gained in one context should generalize to another. Previous computational models and data suggest that rather than learning about each individual context, humans build latent abstract structures and learn to link these structures to arbitrary contexts, facilitating generalization. In these models, task structures that are more popular across contexts are more likely to be revisited in new contexts. However, these models can only re-use policies as a whole and are unable to transfer knowledge about the transition structure of the environment even if only the goal has changed (or vice-versa). This contrasts with ecological settings, where some aspects of task structure, such as the transition function, will be shared between context separately from other aspects, such as the reward function. Here, we develop a novel non-parametric Bayesian agent that forms independent latent clusters for transition and reward functions, affording separable transfer of their constituent parts across contexts. We show that the relative performance of this agent compared to an agent that jointly clusters reward and transition functions depends environmental task statistics: the mutual information between transition and reward functions and the stochasticity of the observations. We formalize our analysis through an information theoretic account of the priors, and propose a meta learning agent that dynamically arbitrates between strategies across task domains to optimize a statistical tradeoff.Author summaryA musician may learn to generalize behaviors across instruments for different purposes, for example, reusing hand motions used when playing classical on the flute to play jazz on the saxophone. Conversely, she may learn to play a single song across many instruments that require completely distinct physical motions, but nonetheless transfer knowledge between them. This degree of compositionality is often absent from computational frameworks of learning, forcing agents either to generalize entire learned policies or to learn new policies from scratch. Here, we propose a solution to this problem that allows an agent to generalize components of a policy independently and compare it to an agent that generalizes components as a whole. We show that the degree to which one form of generalization is favored over the other is dependent on the features of task domain, with independent generalization of task components favored in environments with weak relationships between components or high degrees of noise and joint generalization of task components favored when there is a clear, discoverable relationship between task components. Furthermore, we show that the overall meta structure of the environment can be learned and leveraged by an agent that dynamically arbitrates between these forms of structure learning.

Delay Discounting in Impulsive Behavior

2019 ◽  
Vol 24 (4) ◽  
pp. 312-321 ◽  
Author(s):  
Diana Moreira ◽  
Fernando Barbosa
Keyword(s):  
Delay Discounting ◽  
Human Subjects ◽  
Empirical Studies ◽  
Cochrane Collaboration ◽  
Initial Assessment ◽  
Impulsive Behavior ◽  
Manual Search ◽  
Study Results ◽  
Depth Analysis ◽  
The Future

Abstract. Delay discounting (DD) is the process of devaluing results that happen in the future. With this review, we intend to identify specificities in the processes of DD in impulsive behavior. Studies were retrieved from multiple literature databases, through rigorous criteria (we included systematic reviews and empirical studies with adult human subjects), following the procedures of the Cochrane Collaboration initiative. Of the 174 documents obtained, 19 were considered eligible for inclusion and were retained for in-depth analysis. In addition, 13 studies from the manual search were included. Thus, a total of 32 studies were selected for review. The objectives/hypotheses, results, and the main conclusion(s) were extracted from each study. Results show that people with pronounced traits of impulsivity discount rewards more markedly, that is, they prefer immediate rewards, though of less value, or postponed losses, even though they worsen in the future. Taken together, the existing data suggest the importance of inserting DD as a tool for initial assessment in conjunction with measures of addiction and stress level, as well as the consideration of new therapies.

scholarly journals Spontaneous task structure formation results in a cost to incidental memory of task stimuli

2018 ◽  
Author(s):  
Christina Bejjani ◽  
Tobias Egner
Keyword(s):  
Structure Formation ◽  
Structure Learning ◽  
Learning Task ◽  
Underlying Structure ◽  
Task Structure ◽  
Cognitive Demands ◽  
Task Set ◽  
Stimulus Response ◽  
Attention To Task ◽  
Task Sets

Humans are characterized by their ability to leverage rules for classifying and linking stimuli to context-appropriate actions. Previous studies have shown that when humans learn stimulus-response associations for two-dimensional stimuli, they implicitly form and generalize hierarchical rule structures (task-sets). However, the cognitive processes underlying structure formation are poorly understood. Across four experiments, we manipulated how trial-unique images mapped onto responses to bias spontaneous task-set formation and investigated structure learning through the lens of incidental stimulus encoding. Participants performed a learning task designed to either promote task-set formation (by “motor-clustering” possible stimulus-action rules), or to discourage it (by using arbitrary category-response mappings). We adjudicated between two hypotheses: Structure learning may promote attention to task stimuli, thus resulting in better subsequent memory. Alternatively, building task-sets might impose cognitive demands (for instance, on working memory) that divert attention away from stimulus encoding. While the clustering manipulation affected task-set formation, there were also substantial individual differences. Importantly, structure learning incurred a cost: spontaneous task-set formation was associated with diminished stimulus encoding. Thus, spontaneous hierarchical task-set formation appears to involve cognitive demands that divert attention away from encoding of task stimuli during structure learning.

Workhorse Models

2015 ◽  
Author(s):  
Pol Antràs
Keyword(s):  
International Trade ◽  
Intellectual History ◽  
Empirical Studies ◽  
Theoretical Work ◽  
Central Unit ◽  
Unit Of Analysis ◽  
Basic Understanding ◽  
History Of ◽  
Key Features ◽  
The Rich

This chapter provides a succinct account of the rich intellectual history of the field of international trade and offers an overview of its modern workhorse models. This field has experienced a true revolution in recent years. Firms rather than countries or industries are now the central unit of analysis. The workhorse trade models used by most researchers both in theoretical work as well as in guiding empirical studies were published in the 2000s. While these benchmark frameworks ignore contractual aspects, they constitute the backbone of the models developed later in this volume, so the chapter provides a basic understanding of their key features.

scholarly journals Health system resilience: a literature review of empirical research

2020 ◽  
Vol 35 (8) ◽  
pp. 1084-1109
Author(s):  
Louise Biddle ◽  
Katharina Wahedi ◽  
Kayvan Bozorgmehr
Keyword(s):  
Health System ◽  
Mixed Methods Research ◽  
Quantitative Research ◽  
Empirical Studies ◽  
Disease Outbreaks ◽  
Building Blocks ◽  
Theoretical Work ◽  
Empirical Literature ◽  
Quantitative And Qualitative Research ◽  
System Resilience

Abstract The concept of health system resilience has gained popularity in the global health discourse, featuring in UN policies, academic articles and conferences. While substantial effort has gone into the conceptualization of health system resilience, there has been no review of how the concept has been operationalized in empirical studies. We conducted an empirical review in three databases using systematic methods. Findings were synthesized using descriptive quantitative analysis and by mapping aims, findings, underlying concepts and measurement approaches according to the resilience definition by Blanchet et al. We identified 71 empirical studies on health system resilience from 2008 to 2019, with an increase in literature in recent years (62% of studies published since 2017). Most studies addressed a specific crisis or challenge (82%), most notably infectious disease outbreaks (20%), natural disasters (15%) and climate change (11%). A large proportion of studies focused on service delivery (48%), while other health system building blocks were side-lined. The studies differed in terms of their disciplinary tradition and conceptual background, which was reflected in the variety of concepts and measurement approaches used. Despite extensive theoretical work on the domains which constitute health system resilience, we found that most of the empirical literature only addressed particular aspects related to absorptive and adaptive capacities, with legitimacy of institutions and transformative resilience seldom addressed. Qualitative and mixed methods research captured a broader range of resilience domains than quantitative research. The review shows that the way in which resilience is currently applied in the empirical literature does not match its theoretical foundations. In order to do justice to the complexities of the resilience concept, knowledge from both quantitative and qualitative research traditions should be integrated in a comprehensive assessment framework. Only then will the theoretical ‘resilience idea’ be able to prove its usefulness for the research community.

scholarly journals Dissociable Processes for Learning the Surface Structure and Abstract Structure of Sensorimotor Sequences

1998 ◽  
Vol 10 (6) ◽  
pp. 734-751 ◽  
Author(s):  
Peter F. Dominey ◽  
Taïssia Lelekov ◽  
Jocelyne Ventre-Dominey ◽  
Marc Jeannerod
Keyword(s):  
Surface Structure ◽  
Information Structure ◽  
Structure Learning ◽  
Human Subjects ◽  
Serial Order ◽  
Learning Performance ◽  
Physiological Basis ◽  
Sequence Elements ◽  
Abstract Structure ◽  
Striatal System

A sensorimotor sequence may contain information structure at several different levels. In this study, we investigated the hypothesis that two dissociable processes are required for the learning of surface structure and abstract structure, respectively, of sensorimotor sequences. Surface structure is the simple serial order of the sequence elements, whereas abstract structure is defined by relationships between repeating sequence elements. Thus, sequences ABCBAC and DEFEDF have different surface structures but share a common abstract structure, 123213, and are therefore isomorphic. Our simulations of sequence learning performance in serial reaction time (SRT) tasks demonstrated that (1) an existing model of the primate fronto-striatal system is capable of learning surface structure but fails to learn abstract structure, which requires an additional capability, (2) surface and abstract structure can be learned independently by these independent processes, and (3) only abstract structure transfers to isomorphic sequences. We tested these predictions in human subjects. For a sequence with predictable surface and abstract structure, subjects in either explicit or implicit conditions learn the surface structure, but only explicit subjects learn and transfer the abstract structure. For sequences with only abstract structure, learning and transfer of this structure occurs only in the explicit group. These results are parallel to those from the simulations and support our dissociable process hypothesis. Based on the synthesis of the current simulation and empirical results with our previous neuropsychological findings, we propose a neuro-physiological basis for these dissociable processes: Surface structure can be learned by processes that operate under implicit conditions and rely on the fronto-striatal system, whereas learning abstract structure requires a more explicit activation of dissociable processes that rely on a distributed network that includes the left anterior cortex.

TASK-BASED PRONUNCIATION TEACHING AND RESEARCH

2017 ◽  
Vol 39 (2) ◽  
pp. 381-399 ◽  
Author(s):  
Joan C. Mora ◽  
Mayya Levkina
Keyword(s):  
Empirical Studies ◽  
First Language ◽  
Task Design ◽  
Future Research ◽  
Task Structure ◽  
Crucial Issue ◽  
Grammatical Development ◽  
Research Domains ◽  
Key Questions ◽  
And Task

AbstractThis article synthesizes the conclusions of the empirical studies in this special issue and outlines key questions in future research. The research reported in this volume has identified several fundamental issues in pronunciation-focused task design that are discussed in detail and on which suggestions for further research are outlined. One crucial issue is how attention to pronunciation resulting in language-related episodes effectively leads to robust gains in accuracy. Another important aspect discussed is the need to adapt task design features to the phonological domain under focus and how to incorporate systematic patterns of first language interference into the task structure. Finally, we propose that future research in task-based pronunciation teaching and second language phonetics and phonology should systematically examine learner factors known to affect task performance and task features established in the research domains of lexical and grammatical development.

scholarly journals Linked supergenes underlie split sex ratio and social organization in an ant

2021 ◽  
Author(s):  
German Lagunas-Robles ◽  
Jessica Purcell ◽  
Alan Brelsford
Keyword(s):  
Sex Ratio ◽  
Sex Allocation ◽  
Empirical Studies ◽  
Population Level ◽  
Theoretical Work ◽  
Female Offspring ◽  
Genomic Region ◽  
Genomic Conflict ◽  
Theoretical Predictions ◽  
Formica Podzolica

AbstractSexually reproducing organisms usually invest equally in male and female offspring. Deviations from this pattern have led researchers to new discoveries in the study of parent-offspring conflict, genomic conflict, and cooperation. Some social insect species exhibit the unusual population-level pattern of split sex ratio, wherein some colonies specialize in the production of future queens and others specialize in the production of males. Theoretical work focused on the relatedness asymmetries emerging from haplodiploid inheritance, whereby queens are equally related to daughters and sons, but their daughter workers are more closely related to sisters than to brothers, led to a series of testable predictions and spawned many empirical studies of this phenomenon. However, not all empirical systems follow predicted patterns, so questions remain about how split sex ratio emerges. Here, we sequence the genomes of 138 Formica glacialis workers from 34 male-producing and 34 gyne-producing colonies to determine whether split sex ratio is under genetic control. We identify a supergene spanning 5.5 Mbp that is closely associated with sex allocation in this system. Strikingly, this supergene is adjacent to another supergene spanning 5 Mbp that is associated with variation in colony queen number. We identify a similar pattern in a second related species, Formica podzolica. The discovery that split sex ratio is determined, at least in part, by a supergene in two species opens a new line of research on the evolutionary drivers of split sex ratio.Significance StatementSome social insects exhibit split sex ratio, wherein some colonies produce future queens and others produce males. This phenomenon spawned many influential theoretical studies and empirical tests, both of which have advanced our understanding of parent-offspring conflicts and cooperation. However, some empirical systems did not follow theoretical predictions, indicating that researchers lack a comprehensive understanding of the drivers of split sex ratio. Here, we show that split sex ratio is associated with a large genomic region in two ant species. The discovery of a genetic basis for sex allocation in ants provides a novel explanation for this phenomenon, particularly in systems where empirical observations deviate from theoretical predictions.

The Theoretical and Empirical Approaches to Uncertainty and Conflict in International Relations

2017 ◽  
Author(s):  
Muhammet A. Bas ◽  
Robert Schub
Keyword(s):  
Asymmetric Information ◽  
International Relations ◽  
International Politics ◽  
Empirical Studies ◽  
Theoretical Work ◽  
War And Peace ◽  
Fundamental Uncertainty ◽  
Many Sources ◽  
Different Origins ◽  
Empirical Approaches

Uncertainty is pervasive in international politics. This uncertainty can have many sources. Each source has different origins and implications for the likelihood of conflict. Existing theories focus on three sources: (1) uncertainty due to asymmetric information about adversary traits that affect war payoffs, (2) uncertainty about adversary intentions, and (3) fundamental uncertainty about conflict-relevant processes. Scholarship details the implications of each type of uncertainty for war and peace as well as the prospects for reducing the uncertainty. While theoretical work is quite rich, empirical studies generally lag behind due to measurement challenges and difficulties in specifying clear, testable implications. Nonetheless, using novel proxies for different forms of uncertainty has generated notable progress.

scholarly journals Structure Learning for Headline Generation

2020 ◽  
Vol 34 (05) ◽  
pp. 9555-9562
Author(s):  
Ruqing Zhang ◽  
Jiafeng Guo ◽  
Yixing Fan ◽  
Yanyan Lan ◽  
Xueqi Cheng
Keyword(s):  
Language Processing ◽  
Structure Learning ◽  
Empirical Studies ◽  
Linear Structure ◽  
Neural Models ◽  
Long Distance ◽  
Convolutional Network ◽  
Document Structure ◽  
Representational Power ◽  
High Level

Headline generation is an important problem in natural language processing, which aims to describe a document by a compact and informative headline. Some recent successes on this task have been achieved by advanced graph-based neural models, which marry the representational power of deep neural networks with the structural modeling ability of the relational sentence graphs. The advantages of graph-based neural models over traditional Seq2Seq models lie in that they can encode long-distance relationship between sentences beyond the surface linear structure. However, since documents are typically weakly-structured data, modern graph-based neural models usually rely on manually designed rules or some heuristics to construct the sentence graph a prior. This may largely limit the power and increase the cost of the graph-based methods. In this paper, therefore, we propose to incorporate structure learning into the graph-based neural models for headline generation. That is, we want to automatically learn the sentence graph using a data-driven way, so that we can unveil the document structure flexibly without prior heuristics or rules. To achieve this goal, we employ a deep & wide network to encode rich relational information between sentences for the sentence graph learning. For the deep component, we leverage neural matching models, either representation-focused or interaction-focused model, to learn semantic similarity between sentences. For the wide component, we encode a variety of discourse relations between sentences. A Graph Convolutional Network (GCN) is then applied over the sentence graph to generate high-level relational representations for headline generation. The whole model could be optimized end-to-end so that the structure and representation could be learned jointly. Empirical studies show that our model can significantly outperform the state-of-the-art headline generation models.

Sign in / Sign up

Export Citation Format

Share Document