scholarly journals Combining Reinforcement Learning and Causal Models for Robotics Applications

2021 ◽  
Author(s):  
Arquímides Méndez-Molina
Keyword(s):  
Reinforcement Learning ◽  
Learning Task ◽  
Learning Processes ◽  
Causal Models ◽  
Causal Modeling ◽  
Application Area ◽  
Service Robotics ◽  
Preliminary Results ◽  
Starting Point ◽  
Good Starting Point

The relation between Reinforcement learning (RL) and Causal Modeling(CM) is an underexplored area with untapped potential for any learning task. In this extended abstract of our Ph.D. research proposal, we present a way to combine both areas to improve their respective learning processes, especially in the context of our application area (service robotics). The preliminary results obtained so far are a good starting point for thinking about the success of our research project.

scholarly journals Addition for Measuring the Ability to Understand Regularity and to Extrapolate

2021 ◽  
Author(s):  
Ippei Fujisawa ◽  
Ryota Kanai
Keyword(s):  
Reinforcement Learning ◽  
Computer Science ◽  
Simple Structure ◽  
Theoretical Perspective ◽  
General Purpose ◽  
Training Dataset ◽  
Test Dataset ◽  
Starting Point ◽  
Good Starting Point ◽  
Future Direction

In this paper, we discuss the possibility that elementary arithmetic, such as addition and other arithmetic operations, would be useful as a benchmark for measuring the ability to extrapolate which is considered important for the realization of general-purpose intelligence. Understanding addition can be regarded as performing addition of arbitrary digits correctly by memorizing and applying the rules of addition of single digits, and by learning the rules of carrying. We propose a benchmark in which we prepare a small training dataset that is considered sufficient to reveal the algebra of addition, and measure the accuracy using a test dataset that requires large multi-digit operations. Our benchmark has the following advantages over the datasets usually used in recognition tasks and reinforcement learning. Its simple structure makes it easy to generate datasets, adjust and extend the difficulty, identify inductive biases, and allows for discussion from a theoretical perspective in computer science like program synthesis. We hope that the elementary arithmetic benchmark will reveal functions missing in current AI systems, and also provide a good starting point for developing them. In particular, we speculate that the use of the knowledge may be required for a system to compute correctly for arbitrary digits. Finally, based on these insights, we propose a future direction for the development of systems with extrapolation capabilities.

Neuroevolution approach for reinforcement learning task applied to control system of a cart pole system

2019 ◽  
Author(s):  
Marco Boaretto ◽  
Gabriel Chaves Becchi ◽  
Luiza Scapinello Aquino ◽  
Aderson Cleber Pifer ◽  
Helon Vicente Hultmann Ayala ◽  
...  
Keyword(s):  
Control System ◽  
Reinforcement Learning ◽  
Learning Task ◽  
Pole System

Model-Based and Model-Free Social Cognition

2019 ◽  
Author(s):  
Leor M Hackel ◽  
Jeffrey Jordan Berg ◽  
Björn Lindström ◽  
David Amodio
Keyword(s):  
Reinforcement Learning ◽  
Social Cognition ◽  
Learning Strategies ◽  
Memory Systems ◽  
Learning Task ◽  
Financial Advisors ◽  
Model Based ◽  
Model Free ◽  
Systems Model ◽  
Task Assessment

Do habits play a role in our social impressions? To investigate the contribution of habits to the formation of social attitudes, we examined the roles of model-free and model-based reinforcement learning in social interactions—computations linked in past work to habit and planning, respectively. Participants in this study learned about novel individuals in a sequential reinforcement learning paradigm, choosing financial advisors who led them to high- or low-paying stocks. Results indicated that participants relied on both model-based and model-free learning, such that each independently predicted choice during the learning task and self-reported liking in a post-task assessment. Specifically, participants liked advisors who could provide large future rewards as well as advisors who had provided them with large rewards in the past. Moreover, participants varied in their use of model-based and model-free learning strategies, and this individual difference influenced the way in which learning related to self-reported attitudes: among participants who relied more on model-free learning, model-free social learning related more to post-task attitudes. We discuss implications for attitudes, trait impressions, and social behavior, as well as the role of habits in a memory systems model of social cognition.

Organizational Learning and Multiteam Systems

2019 ◽  
pp. 602-620
Author(s):  
Leslie A. DeChurch ◽  
Gina M. Bufton ◽  
Sophie A. Kay ◽  
Chelsea V. Velez ◽  
Noshir Contractor
Keyword(s):  
Organizational Learning ◽  
Knowledge Transfer ◽  
Knowledge Creation ◽  
Learning Processes ◽  
Organizational Systems ◽  
System Perspective ◽  
Multiteam Systems ◽  
Starting Point ◽  
Emergent States ◽  
Superordinate Goal

Multiteam systems consist of two or more teams, each of which pursues subordinate team goals, while working interdependently with at least one other team toward a superordinate goal. Many teams work in these larger organizational systems, where oft-cited challenges involve learning processes within and between teams. This chapter brings a learning perspective to multiteam systems and a multiteam system perspective to organizational learning. Several classic illustrations of organizational learning—for example, the Challenger and Columbia disasters—actually point to failures in organizational learning processes within and between teams. We offer the focus on intrateam knowledge creation and retention and interteam knowledge transfer as a useful starting point for thinking about how to conceptually and operationally define learning in multiteam systems. Furthermore, we think leadership structures and multiteam emergent states are particularly valuable drivers of learning.

scholarly journals Discovery and Optimization of Selective Inhibitors of Meprin α (Part I)

2021 ◽  
Vol 14 (3) ◽  
pp. 203 ◽  
Author(s):  
Shurong Hou ◽  
Juan Diez ◽  
Chao Wang ◽  
Christoph Becker-Pauly ◽  
Gregg B. Fields ◽  
...  
Keyword(s):  
Inhibitory Activity ◽  
De Novo ◽  
Selective Inhibitors ◽  
Preclinical Development ◽  
Starting Point ◽  
Good Starting Point ◽  
Parallel Screening ◽  
Low Cytotoxicity

Meprin α and β are zinc-dependent proteinases implicated in multiple diseases including cancers, fibrosis, and Alzheimer’s. However, until recently, only a few inhibitors of either meprin were reported and no inhibitors are in preclinical development. Moreover, inhibitors of other metzincins developed in previous years are not effective in inhibiting meprins suggesting the need for de novo discovery effort. To address the paucity of tractable meprin inhibitors we developed ultrahigh-throughput assays and conducted parallel screening of >650,000 compounds against each meprin. As a result of this effort, we identified five selective meprin α hits belonging to three different chemotypes (triazole-hydroxyacetamides, sulfonamide-hydroxypropanamides, and phenoxy-hydroxyacetamides). These hits demonstrated a nanomolar to micromolar inhibitory activity against meprin α with low cytotoxicity and >30-fold selectivity against meprin β and other related metzincincs. These selective inhibitors of meprin α provide a good starting point for further optimization.

scholarly journals Advances on the Implementation of Circular Economy Techniques in Rural Areas in Colombia under a Sustainable Development Framework

2021 ◽  
Vol 13 (7) ◽  
pp. 3816
Author(s):  
Javier Rodrigo-Ilarri ◽  
Camilo-A. Vargas-Terranova ◽  
María-Elena Rodrigo-Clavero ◽  
Paula-A. Bustos-Castro
Keyword(s):  
Sustainable Development ◽  
Waste Management ◽  
Circular Economy ◽  
Rural Areas ◽  
Development Framework ◽  
Starting Point ◽  
Management Plans ◽  
Good Starting Point ◽  
Depressed Regions ◽  
First Time

For the first time in the scientific literature, this research shows an analysis of the implementation of circular economy techniques under sustainable development framework in six municipalities with a depressed economy in Colombia. The analysis is based on solid waste data production at a local scale, the valuation of the waste for subsequent recycling, and the identification and quantification of the variables associated with the treatment and final disposal of waste, in accordance with the Colombian regulatory framework. Waste generation data are obtained considering three different scenarios, in which a comparison between the simulated values and those established in the management plans are compared. Important differences have been identified between the waste management programs of each municipality, specifically regarding the components of waste collection, transportation and disposal, participation of environmental reclaimers, and potential use of materials. These differences are fundamentally associated with the different administrative processes considered for each individual municipality. This research is a good starting point for the development of waste management models based on circular economy techniques, through the subsequent implementation of an office tool in depressed regions such as those studied.

scholarly journals Optimizing Consistency and Coverage in Configurational Causal Modeling

2021 ◽  
pp. 004912412199555
Author(s):  
Michael Baumgartner ◽  
Mathias Ambühl
Keyword(s):  
Model Selection ◽  
Causal Inference ◽  
Comparative Methods ◽  
Causal Models ◽  
Causal Modeling ◽  
Model Fit

Consistency and coverage are two core parameters of model fit used by configurational comparative methods (CCMs) of causal inference. Among causal models that perform equally well in other respects (e.g., robustness or compliance with background theories), those with higher consistency and coverage are typically considered preferable. Finding the optimally obtainable consistency and coverage scores for data [Formula: see text], so far, is a matter of repeatedly applying CCMs to [Formula: see text] while varying threshold settings. This article introduces a procedure called ConCovOpt that calculates, prior to actual CCM analyses, the consistency and coverage scores that can optimally be obtained by models inferred from [Formula: see text]. Moreover, we show how models reaching optimal scores can be methodically built in case of crisp-set and multi-value data. ConCovOpt is a tool, not for blindly maximizing model fit, but for rendering transparent the space of viable models at optimal fit scores in order to facilitate informed model selection—which, as we demonstrate by various data examples, may have substantive modeling implications.

scholarly journals Reinforcement Learning Approaches in Social Robotics

Sensors ◽  
2021 ◽  
Vol 21 (4) ◽  
pp. 1292
Author(s):  
Neziha Akalin ◽  
Amy Loutfi
Keyword(s):  
Reinforcement Learning ◽  
Real World ◽  
Social Robotics ◽  
Research Field ◽  
Social Robots ◽  
Learning Approaches ◽  
Reward Function ◽  
Optimal Behavior ◽  
Learning Challenges ◽  
Starting Point

This article surveys reinforcement learning approaches in social robotics. Reinforcement learning is a framework for decision-making problems in which an agent interacts through trial-and-error with its environment to discover an optimal behavior. Since interaction is a key component in both reinforcement learning and social robotics, it can be a well-suited approach for real-world interactions with physically embodied social robots. The scope of the paper is focused particularly on studies that include social physical robots and real-world human-robot interactions with users. We present a thorough analysis of reinforcement learning approaches in social robotics. In addition to a survey, we categorize existent reinforcement learning approaches based on the used method and the design of the reward mechanisms. Moreover, since communication capability is a prominent feature of social robots, we discuss and group the papers based on the communication medium used for reward formulation. Considering the importance of designing the reward function, we also provide a categorization of the papers based on the nature of the reward. This categorization includes three major themes: interactive reinforcement learning, intrinsically motivated methods, and task performance-driven methods. The benefits and challenges of reinforcement learning in social robotics, evaluation methods of the papers regarding whether or not they use subjective and algorithmic measures, a discussion in the view of real-world reinforcement learning challenges and proposed solutions, the points that remain to be explored, including the approaches that have thus far received less attention is also given in the paper. Thus, this paper aims to become a starting point for researchers interested in using and applying reinforcement learning methods in this particular research field.

Industry-Sponsored Medical Education — In the Quest for Professional Integrity and Legal Certainty

2007 ◽  
Vol 14 (4) ◽  
pp. 313-319
Author(s):  
Benedikt Buchner
Keyword(s):  
Medical Education ◽  
Self Regulation ◽  
State Regulation ◽  
General Question ◽  
Legal Certainty ◽  
European Directive ◽  
Starting Point ◽  
Good Starting Point ◽  
Advertising Law ◽  
The Right

AbstractIndustry-sponsored medical education is a much disputed issue. So far, there has been no regulatory framework which provides clear and definite rules as to whether and under what circumstances the sponsorship of medical education is acceptable. State regulation does not exist, or confines itself to a very general principle. Professional regulation, even though applied frequently, is rather vague and indefinite, raising the general question as to whether self-regulation is the right approach at all. Certainly, self-regulation by industry cannot and should not replace other regulatory approaches. Ultimately, advertising law in general and the European Directive 2001/83/EC specifically, might be a good starting point in providing legal certainty and ensuring the independence of medical education. Swiss advertising law illustrates how the principles of the European Directive could be implemented clearly and unambiguously.

Sign in / Sign up

Export Citation Format

Share Document