Constructing Temporal Abstractions Autonomously in Reinforcement Learning

The idea of temporal abstraction, i.e. learning, planning and representing the world at multiple time scales, has been a constant thread in AI research, spanning sub-fields from classical planning and search to control and reinforcement learning. For example, programming a robot typically involves making decisions over a set of controllers, rather than working at the level of motor torques. While temporal abstraction is a very natural concept, learning such abstractions with no human input has proved quite daunting. In this paper, we present a general architecture, called option-critic, which allows learning temporal abstractions automatically, end-to-end, simply from the agent’s experience. This approach allows continual learning and provides interesting qualitative and quantitative results in several tasks.

Download Full-text

Deep Reinforcement Learning with Hierarchical Structures

Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence ◽

10.24963/ijcai.2021/681 ◽

2021 ◽

Author(s):

Siyuan Li

Keyword(s):

Reinforcement Learning ◽

Time Scales ◽

Hierarchical Structures ◽

Multiple Time Scales ◽

Multiple Time ◽

Top Down ◽

Bottom Up ◽

Hierarchical Reinforcement Learning ◽

Future Work

Hierarchical reinforcement learning (HRL), which enables control at multiple time scales, is a promising paradigm to solve challenging and long-horizon tasks. In this paper, we briefly introduce our work in bottom-up and top-down HRL and outline the directions for future work.

Download Full-text

Mammography Images Segmentation Based on Fuzzy Set and Thresholding

Al-Mustansiriyah Journal of Science ◽

10.23851/mjs.v29i3.644 ◽

2019 ◽

Vol 29 (3) ◽

pp. 168

Author(s):

Ali Mohammed Salih ◽

Mohammed Y. Kamil

Keyword(s):

Fuzzy Set ◽

Fuzzy Inference ◽

Early Recognition ◽

Data Set ◽

Inference System ◽

Breast Masses ◽

Qualitative And Quantitative ◽

Breast Malignancy ◽

The World ◽

Quantitative Results

Breast cancer is the most widespread cancer that influences ladies about the world. Early recognition of breast tumor is a standout amongst the hugest variables influencing the probability of recuperation from the illness. Hence, mammography remains the most precise and best device for distinguishing breast malignancy. This paper presents a method for segment the boundary of breast masses regions in mammograms via a proposed algorithm based on fuzzy set techniques. Firstly, it was used data set (mini-MIAS) for evaluate algorithm. it was preprocessing the data set to remove noise and propose a fuzzy set by using fuzzy inference system by generated two input parameters (employs image gradient), then used thresholding filter. Then it was evaluated this proposed method, qualitative and quantitative results were obtained to demonstrate the efficiency of this method and confirm the possibility of its use in improving the diagnosis.

Download Full-text

Modality in English and Russian Legislative Discourse: A Comparative Study

Respectus Philologicus ◽

10.15388/respectus.2012.27.15335 ◽

2012 ◽

Vol 22 (27) ◽

pp. 37-44

Author(s):

Vladimir Ozyumenko

Keyword(s):

European Union ◽

Comparative Study ◽

Cultural Values ◽

Intercultural Communication ◽

English Language ◽

Qualitative And Quantitative ◽

Communicative Strategies ◽

The World ◽

Quantitative Results

This paper deals with the verbalization and functioning of modality in English and Russian legislative discourse. By comparing the means of modality in English and Russian legislative texts, both in their qualitative and quantitative aspects, it makes an attempt to explain the revealed differences. The data were collected through a comparative study of the English and Russian versions of the UN Charter and the Consolidated Version of the Treaty on European Union. The quantitative results confirm that the English language, as compared to Russian, has a more sophisticated set of modals which are used more regularly. The qualitative findings show some differences in the choices of linguistic forms and communicative strategies. Sociolinguistic and cognitive analyses were then conducted, suggesting that these differences are rooted in culture; that is, in social organization, cultural values, the concept of self, and the relations between authority and individuals. The results advocate the idea that grammar is an ideological instrument for the categorization and classification of things that happen in the world (Thornborrow 2002). Alongside other aspects of language, it provides a lot of sociocultural information. The results are relevant to the study of translation and intercultural communication, as well as to ESL teaching.

Download Full-text

Exact model reduction with delays: closed-form distributions and extensions to fully bi-directional monomolecular reactions

Journal of The Royal Society Interface ◽

10.1098/rsif.2014.0108 ◽

2014 ◽

Vol 11 (95) ◽

pp. 20140108 ◽

Cited By ~ 14

Author(s):

Andre Leier ◽

Manuel Barrio ◽

Tatiana T. Marquez-Lago

Keyword(s):

Closed Form ◽

Biological Research ◽

Multiple Time Scales ◽

Multiple Time ◽

Pharmaceutical Chemical ◽

Computational Costs ◽

Qualitative And Quantitative ◽

Linear Chains ◽

Biochemical Systems ◽

Monomolecular Reactions

In order to systematically understand the qualitative and quantitative behaviour of chemical reaction networks, scientists must derive and analyse associated mathematical models. However, biochemical systems are often very large, with reactions occurring at multiple time scales, as evidenced by signalling pathways and gene expression kinetics. Owing to the associated computational costs, it is then many times impractical, if not impossible, to solve or simulate these systems with an appropriate level of detail. By consequence, there is a growing interest in developing techniques for the simplification or reduction of complex biochemical systems. Here, we extend our recently presented methodology on exact reduction of linear chains of reactions with delay distributions in two ways. First, we report that it is now possible to deal with fully bi-directional monomolecular systems, including degradations, synthesis and generalized bypass reactions. Second, we provide all derivations of associated delays in analytical, closed form. Both advances have a major impact on further reducing computational costs, while still retaining full accuracy. Thus, we expect our new methodology to respond to current simulation needs in pharmaceutical, chemical and biological research.

Download Full-text

Drowning in negativism, self-hate, doubt, madness: Linguistic insights into Sylvia Plath’s experience of depression

Communication & Medicine ◽

10.1558/cam.v11i1.18478 ◽

2015 ◽

Vol 11 (1) ◽

pp. 41-54 ◽

Cited By ~ 1

Author(s):

Zsófia Demjén

Keyword(s):

Lived Experience ◽

Medical Humanities ◽

Sylvia Plath ◽

Individual Experience ◽

Qualitative And Quantitative ◽

The World ◽

The Individual ◽

Linguistic Methods ◽

Linguistic Approach

This paper demonstrates how a range of linguistic methods can be harnessed in pursuit of a deeper understanding of the ‘lived experience’ of psychological disorders. It argues that such methods should be applied more in medical contexts, especially in medical humanities. Key extracts from The Unabridged Journals of Sylvia Plath are examined, as a case study of the experience of depression. Combinations of qualitative and quantitative linguistic methods, and inter- and intra-textual comparisons are used to consider distinctive patterns in the use of metaphor, personal pronouns and (the semantics of) verbs, as well as other relevant aspects of language. Qualitative techniques provide in-depth insights, while quantitative corpus methods make the analyses more robust and ensure the breadth necessary to gain insights into the individual experience. Depression emerges as a highly complex and sometimes potentially contradictory experience for Plath, involving both a sense of apathy and inner turmoil. It involves a sense of a split self, trapped in a state that one cannot overcome, and intense self-focus, a turning in on oneself and a view of the world that is both more negative and more polarized than the norm. It is argued that a linguistic approach is useful beyond this specific case.

Download Full-text

SOME REMARKS ON THE MULTIPLE TIME SCALES OF DIFFUSION IN MINERALS AND MELTS

10.1130/abs/2018am-323387 ◽

2018 ◽

Author(s):

Yan Liang ◽

◽

Daniele J. Cherniak ◽

Chenguang Sun

Keyword(s):

Time Scales ◽

Multiple Time Scales ◽

Multiple Time

Download Full-text

Robust optimal scheduling of integrated energy systems based on hybrid energy storage and multiple time scales

2020 IEEE Sustainable Power and Energy Conference (iSPEC) ◽

10.1109/ispec50848.2020.9351230 ◽

2020 ◽

Author(s):

Mingfang Lu ◽

Xianshan Li ◽

Fei Li

Keyword(s):

Energy Storage ◽

Time Scales ◽

Energy Systems ◽

Optimal Scheduling ◽

Multiple Time Scales ◽

Multiple Time ◽

Hybrid Energy ◽

Hybrid Energy Storage ◽

Integrated Energy Systems

Download Full-text

Odor response adaptation in Drosophila—a continuous individualization process

Cell and Tissue Research ◽

10.1007/s00441-020-03384-6 ◽

2021 ◽

Vol 383 (1) ◽

pp. 143-148

Author(s):

Shadi Jafari ◽

Mattias Alenius

Keyword(s):

Sensory Neurons ◽

Olfactory System ◽

Multiple Time Scales ◽

Multiple Time ◽

Olfactory Perception ◽

Adaptation Mechanisms ◽

Metabolic States ◽

The Central Nervous System ◽

Olfactory Adaptation ◽

Central Adaptation

AbstractOlfactory perception is very individualized in humans and also in Drosophila. The process that individualize olfaction is adaptation that across multiple time scales and mechanisms shape perception and olfactory-guided behaviors. Olfactory adaptation occurs both in the central nervous system and in the periphery. Central adaptation occurs at the level of the circuits that process olfactory inputs from the periphery where it can integrate inputs from other senses, metabolic states, and stress. We will here focus on the periphery and how the fast, slow, and persistent (lifelong) adaptation mechanisms in the olfactory sensory neurons individualize the Drosophila olfactory system.

Download Full-text

Are We Doing ‘Systems’ Research? An Assessment of Methods for Climate Change Adaptation to Hydrohazards in a Complex World

Sustainability ◽

10.3390/su11041163 ◽

2019 ◽

Vol 11 (4) ◽

pp. 1163 ◽

Cited By ~ 4

Author(s):

Melissa Bedinger ◽

Lindsay Beevers ◽

Lila Collet ◽

Annie Visser

Keyword(s):

Climate Change ◽

Human Nature ◽

Climate Change Adaptation ◽

Time Scales ◽

Spatial Scales ◽

Local Context ◽

Multiple Time Scales ◽

Future Research ◽

Multiple Time ◽

Systems Research

Climate change is a product of the Anthropocene, and the human–nature system in which we live. Effective climate change adaptation requires that we acknowledge this complexity. Theoretical literature on sustainability transitions has highlighted this and called for deeper acknowledgment of systems complexity in our research practices. Are we heeding these calls for ‘systems’ research? We used hydrohazards (floods and droughts) as an example research area to explore this question. We first distilled existing challenges for complex human–nature systems into six central concepts: Uncertainty, multiple spatial scales, multiple time scales, multimethod approaches, human–nature dimensions, and interactions. We then performed a systematic assessment of 737 articles to examine patterns in what methods are used and how these cover the complexity concepts. In general, results showed that many papers do not reference any of the complexity concepts, and no existing approach addresses all six. We used the detailed results to guide advancement from theoretical calls for action to specific next steps. Future research priorities include the development of methods for consideration of multiple hazards; for the study of interactions, particularly in linking the short- to medium-term time scales; to reduce data-intensivity; and to better integrate bottom–up and top–down approaches in a way that connects local context with higher-level decision-making. Overall this paper serves to build a shared conceptualisation of human–nature system complexity, map current practice, and navigate a complexity-smart trajectory for future research.

Download Full-text