scholarly journals Learning from Interventions Using Hierarchical Policies for Safe Learning

2020 ◽  
Vol 34 (06) ◽  
pp. 10352-10360
Author(s):  
Jing Bi ◽  
Vikas Dhiman ◽  
Tianyou Xiao ◽  
Chenliang Xu

Learning from Demonstrations (LfD) via Behavior Cloning (BC) works well on multiple complex tasks. However, a limitation of the typical LfD approach is that it requires expert demonstrations for all scenarios, including those in which the algorithm is already well-trained. The recently proposed Learning from Interventions (LfI) overcomes this limitation by using an expert overseer. The expert overseer only intervenes when it suspects that an unsafe action is about to be taken. Although LfI significantly improves over LfD, the state-of-the-art LfI fails to account for delay caused by the expert's reaction time and only learns short-term behavior. We address these limitations by 1) interpolating the expert's interventions back in time, and 2) by splitting the policy into two hierarchical levels, one that generates sub-goals for the future and another that generates actions to reach those desired sub-goals. This sub-goal prediction forces the algorithm to learn long-term behavior while also being robust to the expert's reaction time. Our experiments show that LfI using sub-goals in a hierarchical policy framework trains faster and achieves better asymptotic performance than typical LfD.

Vulcan ◽  
2020 ◽  
Vol 8 (1) ◽  
pp. 100-124
Author(s):  
Adam Givens

Abstract This article analyzes the groundbreaking 1952 plan by US Army leadership to develop a sizeable cargo helicopter program in the face of interservice opposition. It examines the influence that decision had in the next decade on the Army, the helicopter industry, and vtol technology. The Army’s procurement of large helicopters that could transport soldiers and materiel was neither a fait accompli nor based on short-term needs. Rather, archival records reveal that the decision was based on long-range concerns about the postwar health of the helicopter industry, developing the state of the art, and fostering new doctrinal concepts. The procurement had long-term consequences. Helicopters became central to Army war planning, and the ground service’s needs dictated the next generation of helicopter designs. That technology made possible the revolutionary airmobility concept that the Army took into Vietnam and also led to a flourishing commercial helicopter field.


1985 ◽  
Vol 107 (3) ◽  
pp. 339-346 ◽  
Author(s):  
A. Peralta-Duran ◽  
P. H. Wirsching

A probablistic approach to the correlation and extrapolation of creep–rupture data is presented. Time–temperature parameters (TTP) are used to correlate the data, and an analytical expression for the master curve is developed. The expression provides a simple model for the statistical distribution of strength and fits neatly into a probabilistic design format. The analysis focuses on the Larson–Miller and on the Manson–Haferd parameters, but it can be applied to any of the TTP’s. A method is developed for evaluating material dependent constants for TTP’s. It is shown that “optimized” constants can provide a significant improvement in the correlation of the data, thereby reducing modeling error. Attempts were made to quantify the performance of the proposed method in predicting long-term behavior. Bias and uncertainty in predicting long-term behavior from short-term tests were derived for several sets of data. Examples are presented which illustrate the theory and demonstrate the application of state-of-the-art reliability methods to the design of components under creep.


2016 ◽  
Vol 6 (1) ◽  
pp. 20150098 ◽  
Author(s):  
Markus J. Buehler ◽  
Guy M. Genin

Advances in multiscale models and computational power have enabled a broad toolset to predict how molecules, cells, tissues and organs behave and develop. A key theme in biological systems is the emergence of macroscale behaviour from collective behaviours across a range of length and timescales, and a key element of these models is therefore hierarchical simulation. However, this predictive capacity has far outstripped our ability to validate predictions experimentally, particularly when multiple hierarchical levels are involved. The state of the art represents careful integration of multiscale experiment and modelling, and yields not only validation, but also insights into deformation and relaxation mechanisms across scales. We present here a sampling of key results that highlight both challenges and opportunities for integrated multiscale experiment and modelling in biological systems.


2016 ◽  
Vol 2016 ◽  
pp. 1-11 ◽  
Author(s):  
Octavian Pastravanu ◽  
Mihaela-Hanako Matcovschi

The main purpose of this work is to show that the Perron-Frobenius eigenstructure of a positive linear system is involved not only in the characterization of long-term behavior (for which well-known results are available) but also in the characterization of short-term or transient behavior. We address the analysis of the short-term behavior by the help of the “(M,β)-stability” concept introduced in literature for general classes of dynamics. Our paper exploits this concept relative to Hölder vectorp-norms,1≤p≤∞, adequately weighted by scaling operators, focusing on positive linear systems. Given an asymptotically stable positive linear system, for each1≤p≤∞, we prove the existence of a scaling operator (built from the right and left Perron-Frobenius eigenvectors, with concrete expressions depending onp) that ensures the best possible values for the parametersMandβ, corresponding to an “ideal” short-term (transient) behavior. We provide results that cover both discrete- and continuous-time dynamics. Our analysis also captures the differences between the cases where the system dynamics is defined by matrices irreducible and reducible, respectively. The theoretical developments are applied to the practical study of the short-term behavior for two positive linear systems already discussed in literature by other authors.


2021 ◽  
Vol 11 (23) ◽  
pp. 11344
Author(s):  
Wei Ke ◽  
Ka-Hou Chan

Paragraph-based datasets are hard to analyze by a simple RNN, because a long sequence always contains lengthy problems of long-term dependencies. In this work, we propose a Multilayer Content-Adaptive Recurrent Unit (CARU) network for paragraph information extraction. In addition, we present a type of CNN-based model as an extractor to explore and capture useful features in the hidden state, which represent the content of the entire paragraph. In particular, we introduce the Chebyshev pooling to connect to the end of the CNN-based extractor instead of using the maximum pooling. This can project the features into a probability distribution so as to provide an interpretable evaluation for the final analysis. Experimental results demonstrate the superiority of the proposed approach, being compared to the state-of-the-art models.


2020 ◽  
Vol 34 (05) ◽  
pp. 9571-9578 ◽  
Author(s):  
Wei Zhang ◽  
Yue Ying ◽  
Pan Lu ◽  
Hongyuan Zha

Personalized image caption, a natural extension of the standard image caption task, requires to generate brief image descriptions tailored for users' writing style and traits, and is more practical to meet users' real demands. Only a few recent studies shed light on this crucial task and learn static user representations to capture their long-term literal-preference. However, it is insufficient to achieve satisfactory performance due to the intrinsic existence of not only long-term user literal-preference, but also short-term literal-preference which is associated with users' recent states. To bridge this gap, we develop a novel multimodal hierarchical transformer network (MHTN) for personalized image caption in this paper. It learns short-term user literal-preference based on users' recent captions through a short-term user encoder at the low level. And at the high level, the multimodal encoder integrates target image representations with short-term literal-preference, as well as long-term literal-preference learned from user IDs. These two encoders enjoy the advantages of the powerful transformer networks. Extensive experiments on two real datasets show the effectiveness of considering two types of user literal-preference simultaneously and better performance over the state-of-the-art models.


2020 ◽  
Author(s):  
Juanjuan Wang ◽  
HaoRan Yang ◽  
Ning Xu ◽  
Chengqin Wu ◽  
ZengShun Zhao ◽  
...  

Abstract The long-term visual tracking undergoes more challenges and is closer to realistic applications than short-term tracking. However, the performances of most existing methods have been limited in the long-term tracking tasks. In this work, we present a reliable yet simple long-term tracking method, which extends the state-of-the-art Learning Adaptive Discriminative Correlation Filters (LADCF) tracking algorithm with a re-detection component based on the SVM model. The LADCF tracking algorithm localizes the target in each frame and the re-detector is able to efficiently re-detect the target in the whole image when the tracking fails. We further introduce a robust confidence degree evaluation criterion that combines the maximum response criterion and the average peak-to correlation energy (APCE) to judge the confidence level of the predicted target. When the confidence degree is generally high, the SVM is updated accordingly. If the confidence drops sharply, the SVM re-detects the target. We perform extensive experiments on the OTB-2015 and UAV123 datasets. The experimental results demonstrate the effectiveness of our algorithm in long-term tracking.


2021 ◽  
Vol 4 ◽  
Author(s):  
Tiina Laamanen ◽  
Veera Norros ◽  
Sanna Suikkanen ◽  
Mikko Tolkkinen ◽  
Kristiina Vuorio ◽  
...  

Environmental DNA (eDNA) and other molecular based approaches are revolutionizing the field of biomonitoring. These approaches undergo rapid modifications, and it is crucial to develop the best practices by sharing the newest information and knowledge. In our ongoing project we: assess the state-of-the-art of eDNA methods at Finnish Environment Institute SYKE; identify concrete next steps towards the long-term aim of implementing eDNA methods into environmental and biomonitoring; promote information exchange on eDNA methods and advance future research efforts both within SYKE and with our national and international partners. assess the state-of-the-art of eDNA methods at Finnish Environment Institute SYKE; identify concrete next steps towards the long-term aim of implementing eDNA methods into environmental and biomonitoring; promote information exchange on eDNA methods and advance future research efforts both within SYKE and with our national and international partners. Scientific background Well-functioning and intact natural ecosystems are essential for human well-being, provide a variety of ecosystem services and contain a high diversity of organisms. However, human activities such as eutrophication, pollution, land-use or invasive species, are threatening the state and functioning of ecosystems from local to global scale (e.g. Benateau et al. 2019; Reid et al. 2018; Vörösmarty et al. 2010). New molecular techniques in the field and in the laboratory have enabled sampling and identification of much of terrestrial, marine and freshwater biodiversity. These include environmental DNA (eDNA, e.g. Valentini et al. 2016) and bulk-sample DNA metabarcoding approaches (e.g. Elbrecht et al. 2017) and targeted RNA-based methods (e.g. Mäki and Tiirola 2018). The eDNA technique uses DNA that is released from organisms into their environment, from which a signal of organisms’ presence in the system can be obtained. For example, in aquatic ecosystems, eDNA is typically extracted from sediment or filtered water samples (e.g. Deiner et al. 2016), and this approach is distinguished from bulk DNA metabarcoding, where organisms are directly identified from e.g. complete biological monitoring samples (e.g. Elbrecht et al. 2017). Despite the demonstrated potential of environmental and bulk-sample DNA metabarcoding approaches in recent years, there are still significant bottlenecks to their routine use that need to be addressed (e.g. Pawlowski et al. 2020). Methods and implementati on The project is divided into three work packages: WP1 Gathering existing knowledge, identifying knowledge gaps and proposing best practices, WP2 Roadmap to implementation and WP3 eDNA monitoring pilot. Please see more details in the Fig. 1


2021 ◽  
Vol 04 ◽  
Author(s):  
Diego Moreira Schlemper ◽  
Sérgio Henrique Pezzin

: Self-healing coatings are intended to increase long-term durability and reliability and can be enabled by the presence of microcapsules containing a self-healing agent capable of interacting with the matrix and regenerating the system. This review article provides an overview of the state-of-the-art, focusing on the patents published in the field of microcapsule-based self-healing organic coatings, since the early 2000’s. A discussion about coatings for corrosion protection and the different self-healing approaches and mechanisms are also addressed, as well as future challenges and expectations for this kind of coatings.


Sign in / Sign up

Export Citation Format

Share Document