scholarly journals Reinforcement Learning: Connections, Surprises, and Challenge

AI Magazine ◽  
2019 ◽  
Vol 40 (1) ◽  
pp. 3-15 ◽  
Author(s):  
Andrew G. Barto
Keyword(s):  
Reinforcement Learning ◽  
Real World ◽  
Personal Experience ◽  
Historical Context ◽  
The Way ◽  
Learning Connections

The idea of implementing reinforcement learning in a computer was one of the earliest ideas about the possibility of AI, but reinforcement learning remained on the margin of AI until relatively recently. Today we see reinforcement learning playing essential roles in some of the most impressive AI applications. This article presents observations from the author’s personal experience with reinforcement learning over the most recent 40 years of its history in AI, focusing on striking connections that emerged between largely separate disciplines and on some of the findings that surprised him along the way. These connections and surprises place reinforcement learning in a historical context, and they help explain the success it is finding in modern AI. The article concludes by discussing some of the challenges that need to be faced as reinforcement learning moves out into real world.

Market Reforms in Russia: Can the Vicious Circle Be Teared Open?

2004 ◽  
pp. 114-128
Author(s):  
V. Nimushin
Keyword(s):  
Economic Policy ◽  
Comparative Analysis ◽  
Historical Context ◽  
Vicious Circle ◽  
Modern World ◽  
Market Reforms ◽  
Main Task ◽  
Forward Movement ◽  
Socially Responsible ◽  
The Way

In the framework of broad philosophic and historical context the author conducts comparative analysis of the conditions for assimilating liberal values in leading countries of the modern world and in Russia. He defends the idea of inevitable forward movement of Russia on the way of rationalization and cultivation of all aspects of life, but, to his opinion, it will occur not so fast as the "first wave" reformers thought and in other ideological and sociocultural forms than in Europe and America. The author sees the main task of the reformist forces in Russia in consolidation of the society and inplementation of socially responsible economic policy.

COVID-19 Contact Tracing & Privacy Protection, We Are on the Way (Preprint)

2020 ◽  
Author(s):  
Jing Fan
Keyword(s):  
Real World ◽  
Privacy Protection ◽  
Contact Tracing ◽  
Personal Privacy ◽  
The Way

UNSTRUCTURED Smartphone-based contact tracing is proven to be effective in epidemic containment. To maintain its utilization meanwhile ensure the protection of personal privacy, different countries came up with different practices, new exploratory solutions may come into real-world practice soon as well.

A Fitting Solution

2018 ◽  
Author(s):  
Kaori Kashimura ◽  
Takafumi Kawasaki Jr. ◽  
Nozomi Ikeya ◽  
Dave Randall
Keyword(s):  
Organizational Change ◽  
Power Plant ◽  
Augmented Reality ◽  
Real World ◽  
Work Practices ◽  
Plant Construction ◽  
Tightly Coupled ◽  
Complex Scenario ◽  
The Relationship ◽  
The Way

This chapter provides an ethnography of a complex scenario involving the construction of a power plant and, in so doing, tries to show the importance of a practice-based approach to the problem of technical and organizational change. The chapter reports on fieldwork conducted in a highly complex and tightly coupled environment: power plant construction. The ethnography describes work practices on three different sites and describes and analyses their interlocking dependencies, showing the difficulties encountered at each location and the way in which the delays that result cascade through the different sites. It goes on to describe some technological solutions that are associated with augmented reality and that are being designed in response to the insights gained from the fieldwork. The chapter also reflects more generally on the relationship between fieldwork and design in real-world contexts.

Recent Advances in Reinforcement Learning for Traffic Signal Control

2021 ◽  
Vol 22 (2) ◽  
pp. 12-18 ◽  
Author(s):  
Hua Wei ◽  
Guanjie Zheng ◽  
Vikash Gayah ◽  
Zhenhui Li
Keyword(s):  
Reinforcement Learning ◽  
Real World ◽  
Intelligent Transportation Systems ◽  
Transportation Systems ◽  
Traffic Signal ◽  
Signal Control ◽  
Traffic Signal Control ◽  
Control Methods ◽  
Advantages And Disadvantages ◽  
Recent Advances

Traffic signal control is an important and challenging real-world problem that has recently received a large amount of interest from both transportation and computer science communities. In this survey, we focus on investigating the recent advances in using reinforcement learning (RL) techniques to solve the traffic signal control problem. We classify the known approaches based on the RL techniques they use and provide a review of existing models with analysis on their advantages and disadvantages. Moreover, we give an overview of the simulation environments and experimental settings that have been developed to evaluate the traffic signal control methods. Finally, we explore future directions in the area of RLbased traffic signal control methods. We hope this survey could provide insights to researchers dealing with real-world applications in intelligent transportation systems

Challenges of real-world reinforcement learning: definitions, benchmarks and analysis

2021 ◽  
Author(s):  
Gabriel Dulac-Arnold ◽  
Nir Levine ◽  
Daniel J. Mankowitz ◽  
Jerry Li ◽  
Cosmin Paduraru ◽  
...  
Keyword(s):  
Reinforcement Learning ◽  
Real World

scholarly journals Dealing with multiple experts and non-stationarity in inverse reinforcement learning: an application to real-life problems

2021 ◽  
Author(s):  
Amarildo Likmeta ◽  
Alberto Maria Metelli ◽  
Giorgia Ramponi ◽  
Andrea Tirinzoni ◽  
Matteo Giuliani ◽  
...  
Keyword(s):  
Reinforcement Learning ◽  
Real World ◽  
Real Life ◽  
User Preferences ◽  
Inverse Reinforcement Learning ◽  
Water Release ◽  
Reward Function ◽  
Model Free ◽  
Conflicting Objectives ◽  
Multiple Experts

AbstractIn real-world applications, inferring the intentions of expert agents (e.g., human operators) can be fundamental to understand how possibly conflicting objectives are managed, helping to interpret the demonstrated behavior. In this paper, we discuss how inverse reinforcement learning (IRL) can be employed to retrieve the reward function implicitly optimized by expert agents acting in real applications. Scaling IRL to real-world cases has proved challenging as typically only a fixed dataset of demonstrations is available and further interactions with the environment are not allowed. For this reason, we resort to a class of truly batch model-free IRL algorithms and we present three application scenarios: (1) the high-level decision-making problem in the highway driving scenario, and (2) inferring the user preferences in a social network (Twitter), and (3) the management of the water release in the Como Lake. For each of these scenarios, we provide formalization, experiments and a discussion to interpret the obtained results.

scholarly journals Goal-driven active learning

2021 ◽  
Vol 35 (2) ◽  
Author(s):  
Nicolas Bougie ◽  
Ryutaro Ichise
Keyword(s):  
Decision Making ◽  
Reinforcement Learning ◽  
Learning Process ◽  
Real World ◽  
Imitation Learning ◽  
Learning Approaches ◽  
Wide Range ◽  
Fixed Set ◽  
Complex Decision Making ◽  
Complex Decision

AbstractDeep reinforcement learning methods have achieved significant successes in complex decision-making problems. In fact, they traditionally rely on well-designed extrinsic rewards, which limits their applicability to many real-world tasks where rewards are naturally sparse. While cloning behaviors provided by an expert is a promising approach to the exploration problem, learning from a fixed set of demonstrations may be impracticable due to lack of state coverage or distribution mismatch—when the learner’s goal deviates from the demonstrated behaviors. Besides, we are interested in learning how to reach a wide range of goals from the same set of demonstrations. In this work we propose a novel goal-conditioned method that leverages very small sets of goal-driven demonstrations to massively accelerate the learning process. Crucially, we introduce the concept of active goal-driven demonstrations to query the demonstrator only in hard-to-learn and uncertain regions of the state space. We further present a strategy for prioritizing sampling of goals where the disagreement between the expert and the policy is maximized. We evaluate our method on a variety of benchmark environments from the Mujoco domain. Experimental results show that our method outperforms prior imitation learning approaches in most of the tasks in terms of exploration efficiency and average scores.

Yarning as protected space: relational accountability in research

2021 ◽  
pp. 117718012098615
Author(s):  
Stuart Barlo ◽  
William (Bill) Edgar Boyd ◽  
Margaret Hughes ◽  
Shawn Wilson ◽  
Alessandro Pelizzon
Keyword(s):  
Research Methodology ◽  
Personal Experience ◽  
Indigenous Research ◽  
Indigenous Australian ◽  
Australian Context ◽  
Data History ◽  
Research Relationships ◽  
Relational Accountability ◽  
Relational Process ◽  
The Way

In this article, we open up Yarning as a fundamentally relational methodology. We discuss key relationships involved in Indigenous research, including with participants, Country, Ancestors, data, history, and Knowledge. We argue that the principles and protocols associated with the deepest layers of yarning in an Indigenous Australian context create a protected space which supports the researcher to develop and maintain accountability in each of these research relationships. Protection and relational accountability in turn contribute to research which is trustworthy and has integrity. Woven throughout the article are excerpts of a yarn in which the first author reflects on his personal experience of this research methodology. We hope this device serves to demonstrate the way yarning as a relational process of communication helps to bring out deeper reflection and analysis and invoke accountability in all of our research relationships.

scholarly journals How to train your robot with deep reinforcement learning: lessons we have learned

2021 ◽  
pp. 027836492098785
Author(s):  
Julian Ibarz ◽  
Jie Tan ◽  
Chelsea Finn ◽  
Mrinal Kalakrishnan ◽  
Peter Pastor ◽  
...  
Keyword(s):  
Machine Learning ◽  
Reinforcement Learning ◽  
Case Studies ◽  
Real World ◽  
Review Article ◽  
The Real ◽  
Complex Skills ◽  
Real World Learning ◽  
Level Sensor ◽  
Embodied Agent

Deep reinforcement learning (RL) has emerged as a promising approach for autonomously acquiring complex behaviors from low-level sensor observations. Although a large portion of deep RL research has focused on applications in video games and simulated control, which does not connect with the constraints of learning in real environments, deep RL has also demonstrated promise in enabling physical robots to learn complex skills in the real world. At the same time, real-world robotics provides an appealing domain for evaluating such algorithms, as it connects directly to how humans learn: as an embodied agent in the real world. Learning to perceive and move in the real world presents numerous challenges, some of which are easier to address than others, and some of which are often not considered in RL research that focuses only on simulated domains. In this review article, we present a number of case studies involving robotic deep RL. Building off of these case studies, we discuss commonly perceived challenges in deep RL and how they have been addressed in these works. We also provide an overview of other outstanding challenges, many of which are unique to the real-world robotics setting and are not often the focus of mainstream RL research. Our goal is to provide a resource both for roboticists and machine learning researchers who are interested in furthering the progress of deep RL in the real world.

Sign in / Sign up

Export Citation Format

Share Document