scholarly journals Understanding the Relationship between Interactions and Outcomes in Human-in-the-Loop Machine Learning

2021 ◽  
Author(s):  
Yuchen Cui ◽  
Pallavi Koppol ◽  
Henny Admoni ◽  
Scott Niekum ◽  
Reid Simmons ◽  
...  
Keyword(s):  
Machine Learning ◽  
Human Performance ◽  
Autonomous Agents ◽  
Training Data ◽  
Open Problems ◽  
Human In The Loop ◽  
Efficiency And Effectiveness ◽  
Interaction Types ◽  
The Relationship ◽  
And Training

Human-in-the-loop Machine Learning (HIL-ML) is a widely adopted paradigm for instilling human knowledge in autonomous agents. Many design choices influence the efficiency and effectiveness of such interactive learning processes, particularly the interaction type through which the human teacher may provide feedback. While different interaction types (demonstrations, preferences, etc.) have been proposed and evaluated in the HIL-ML literature, there has been little discussion of how these compare or how they should be selected to best address a particular learning problem. In this survey, we propose an organizing principle for HIL-ML that provides a way to analyze the effects of interaction types on human performance and training data. We also identify open problems in understanding the effects of interaction types.

scholarly journals On the utility of dreaming: A general model for how learning in artificial agents can benefit from data hallucination

2020 ◽  
pp. 105971231989648 ◽  
Author(s):  
David Windridge ◽  
Henrik Svensson ◽  
Serge Thill
Keyword(s):  
Machine Learning ◽  
Simulated Data ◽  
Training Data ◽  
Successful Implementation ◽  
Artificial Agents ◽  
Learning Context ◽  
Training Set ◽  
Convergence Point ◽  
And Training ◽  
General Method

We consider the benefits of dream mechanisms – that is, the ability to simulate new experiences based on past ones – in a machine learning context. Specifically, we are interested in learning for artificial agents that act in the world, and operationalize “dreaming” as a mechanism by which such an agent can use its own model of the learning environment to generate new hypotheses and training data. We first show that it is not necessarily a given that such a data-hallucination process is useful, since it can easily lead to a training set dominated by spurious imagined data until an ill-defined convergence point is reached. We then analyse a notably successful implementation of a machine learning-based dreaming mechanism by Ha and Schmidhuber (Ha, D., & Schmidhuber, J. (2018). World models. arXiv e-prints, arXiv:1803.10122). On that basis, we then develop a general framework by which an agent can generate simulated data to learn from in a manner that is beneficial to the agent. This, we argue, then forms a general method for an operationalized dream-like mechanism. We finish by demonstrating the general conditions under which such mechanisms can be useful in machine learning, wherein the implicit simulator inference and extrapolation involved in dreaming act without reinforcing inference error even when inference is incomplete.

scholarly journals DeepEthogram, a machine learning pipeline for supervised behavior classification from raw pixels

eLife ◽  
2021 ◽  
Vol 10 ◽  
Author(s):  
James P Bohnslav ◽  
Nivanthika K Wimalasena ◽  
Kelsey J Clausing ◽  
Yu Y Dai ◽  
David A Yarmolinsky ◽  
...  
Keyword(s):  
Machine Learning ◽  
Human Performance ◽  
Video Recording ◽  
Gene Mutations ◽  
General Purpose ◽  
Training Data ◽  
Supervised Machine Learning ◽  
Video Frame ◽  
Neural Function ◽  
End User Programming

Videos of animal behavior are used to quantify researcher-defined behaviors-of-interest to study neural function, gene mutations, and pharmacological therapies. Behaviors-of-interest are often scored manually, which is time-consuming, limited to few behaviors, and variable across researchers. We created DeepEthogram: software that uses supervised machine learning to convert raw video pixels into an ethogram, the behaviors-of-interest present in each video frame. DeepEthogram is designed to be general-purpose and applicable across species, behaviors, and video-recording hardware. It uses convolutional neural networks to compute motion, extract features from motion and images, and classify features into behaviors. Behaviors are classified with above 90% accuracy on single frames in videos of mice and flies, matching expert-level human performance. DeepEthogram accurately predicts rare behaviors, requires little training data, and generalizes across subjects. A graphical interface allows beginning-to-end analysis without end-user programming. DeepEthogram's rapid, automatic, and reproducible labeling of researcher-defined behaviors-of-interest may accelerate and enhance supervised behavior analysis.

scholarly journals Efficient and effective OCR engine training

2019 ◽  
Vol 23 (1) ◽  
pp. 73-88
Author(s):  
Christian Clausner ◽  
Apostolos Antonacopoulos ◽  
Stefan Pletschacher
Keyword(s):  
Evaluation System ◽  
Training Data ◽  
Text Recognition ◽  
Training Approach ◽  
Efficiency And Effectiveness ◽  
Analysis System ◽  
The Impact ◽  
Data Requirements ◽  
And Training

Abstract We present an efficient and effective approach to train OCR engines using the Aletheia document analysis system. All components required for training are seamlessly integrated into Aletheia: training data preparation, the OCR engine’s training processes themselves, text recognition, and quantitative evaluation of the trained engine. Such a comprehensive training and evaluation system, guided through a GUI, allows for iterative incremental training to achieve best results. The widely used Tesseract OCR engine is used as a case study to demonstrate the efficiency and effectiveness of the proposed approach. Experimental results are presented validating the training approach with two different historical datasets, representative of recent significant digitisation projects. The impact of different training strategies and training data requirements is presented in detail.

scholarly journals Predicting On-axis Rotorcraft Dynamic Responses Using Machine Learning Techniques

2020 ◽  
Vol 65 (3) ◽  
pp. 1-12
Author(s):  
Ryan D. Jackson ◽  
Michael Jump ◽  
Peter L. Green
Keyword(s):  
Machine Learning ◽  
Real Time ◽  
Computational Cost ◽  
Response Term ◽  
Dynamic Responses ◽  
Training Data ◽  
Machine Learning Techniques ◽  
Human In The Loop ◽  
Roll Rate ◽  
Gp Model

Physical-law-based models are widely utilized in the aerospace industry. One such use is to provide flight dynamics models for use in flight simulators. For human-in-the-loop use, such simulators must run in real-time. Owing to the complex physics of rotorcraft flight, to meet this real-time requirement, simplifications to the underlying physics sometimes have to be applied to the model, leading to errors in the model's predictions of the real vehicle's response. This study investigated whether a machine-learning technique could be employed to provide rotorcraft dynamic response predictions. Machine learning was facilitated using a Gaussian process (GP) nonlinear autoregressive model, which predicted the on-axis pitch rate, roll rate, yaw rate, and heave responses of a Bo105 rotorcraft. A variational sparse GP model was then developed to reduce the computational cost of implementing the approach on large datasets. It was found that both of the GP models were able to provide accurate on-axis response predictions, particularly when the model input contained all four control inceptors and one lagged on-axis response term. The predictions made showed improvement compared to a corresponding physics-based model. The reduction of training data to one-third (rotational axes) or one-half (heave axis) resulted in only minor degradation of the sparse GP model predictions.

scholarly journals On the relationship between research parasites and fairness in machine learning: challenges and opportunities

GigaScience ◽  
2021 ◽  
Vol 10 (12) ◽  
Author(s):  
Nicolás Nieto ◽  
Agostina Larrazabal ◽  
Victoria Peterson ◽  
Diego H Milone ◽  
Enzo Ferrante
Keyword(s):  
Machine Learning ◽  
Data Model ◽  
Learning Systems ◽  
Training Data ◽  
Model Construction ◽  
Daily Lives ◽  
The Past ◽  
Learning Challenges ◽  
Challenges And Opportunities ◽  
The Relationship

Abstract Machine learning systems influence our daily lives in many different ways. Hence, it is crucial to ensure that the decisions and recommendations made by these systems are fair, equitable, and free of unintended biases. Over the past few years, the field of fairness in machine learning has grown rapidly, investigating how, when, and why these models capture, and even potentiate, biases that are deeply rooted not only in the training data but also in our society. In this Commentary, we discuss challenges and opportunities for rigorous posterior analyses of publicly available data to build fair and equitable machine learning systems, focusing on the importance of training data, model construction, and diversity in the team of developers. The thoughts presented here have grown out of the work we did, which resulted in our winning the annual Research Parasite Award that GigaSciencesponsors.

Machine Learning for Agents and Multi-Agent Systems

2011 ◽  
pp. 1-26
Author(s):  
Daniel Kudenko ◽  
Dimitar Kazakov ◽  
Eduardo Alonso
Keyword(s):  
Machine Learning ◽  
Autonomous Agents ◽  
Machine Learning Techniques ◽  
Multi Agent Systems ◽  
Learning Agents ◽  
Agent Systems ◽  
Learning Techniques ◽  
Comprehensive Survey ◽  
Multi Agent ◽  
The Relationship

In order to be truly autonomous, agents need the ability to learn from and adapt to the environment and other agents. This chapter introduces key concepts of machine learning and how they apply to agent and multi-agent systems. Rather than present a comprehensive survey, we discuss a number of issues that we believe are important in the design of learning agents and multi-agent systems. Specifically, we focus on the challenges involved in adapting (originally disembodied) machine learning techniques to situated agents, the relationship between learning and communication, learning to collaborate and compete, learning of roles, evolution and natural selection, and distributed learning. In the second part of the chapter, we focus on some practicalities and present two case studies.

Machine Learning for Agents and Multi-Agent Systems

2011 ◽  
pp. 403-420
Author(s):  
Daniel Kudenko ◽  
Dimitar Kazakov ◽  
Eduardo Alonso
Keyword(s):  
Machine Learning ◽  
Autonomous Agents ◽  
Machine Learning Techniques ◽  
Multi Agent Systems ◽  
Learning Agents ◽  
Agent Systems ◽  
Learning Techniques ◽  
Comprehensive Survey ◽  
Multi Agent ◽  
The Relationship

In order to be truly autonomous, agents need the ability to learn from and adapt to the environment and other agents. This chapter introduces key concepts of machine learning and how they apply to agent and multi-agent systems. Rather than present a comprehensive survey, we discuss a number of issues that we believe are important in the design of learning agents and multi-agent systems. Specifically, we focus on the challenges involved in adapting (originally disembodied) machine learning techniques to situated agents, the relationship between learning and communication, learning to collaborate and compete, learning of roles, evolution and natural selection, and distributed learning. In the second part of the chapter, we focus on some practicalities and present two case studies.

scholarly journals A Survey of Game Theoretic Approaches for Adversarial Machine Learning in Cybersecurity Tasks

AI Magazine ◽  
2019 ◽  
Vol 40 (2) ◽  
pp. 31-43 ◽  
Author(s):  
Prithviraj Dasgupta ◽  
Joseph Collins
Keyword(s):  
Machine Learning ◽  
Learning Algorithm ◽  
Training Data ◽  
Machine Learning Techniques ◽  
Critical Systems ◽  
Open Problems ◽  
Learning Techniques ◽  
Supervised Learning Algorithms ◽  
Game Theoretic ◽  
Using Data

Machine learning techniques are used extensively for automating various cybersecurity tasks. Most of these techniques use supervised learning algorithms that rely on training the algorithm to classify incoming data into categories, using data encountered in the relevant domain. A critical vulnerability of these algorithms is that they are susceptible to adversarial attacks by which a malicious entity called an adversary deliberately alters the training data to misguide the learning algorithm into making classification errors. Adversarial attacks could render the learning algorithm unsuitable for use and leave critical systems vulnerable to cybersecurity attacks. This article provides a detailed survey of the stateof-the-art techniques that are used to make a machine learning algorithm robust against adversarial attacks by using the computational framework of game theory. We also discuss open problems and challenges and possible directions for further research that would make deep machine learning–based systems more robust and reliable for cybersecurity tasks.

Special Issue on Machine Learning for Robotics and Swarm Systems

2019 ◽  
Vol 31 (4) ◽  
pp. 519-519
Author(s):  
Masahito Yamamoto ◽  
Takashi Kawakami ◽  
Keitaro Naruse
Keyword(s):  
Machine Learning ◽  
Autonomous Agents ◽  
Autonomous Robots ◽  
Machine Learning Techniques ◽  
Multi Agent Systems ◽  
Special Issue ◽  
Learning Techniques ◽  
Machine Learning Applications ◽  
Multi Agent ◽  
The Relationship

In recent years, machine-learning applications have been rapidly expanding in the fields of robotics and swarm systems, including multi-agent systems. Swarm systems were developed in the field of robotics as a kind of distributed autonomous robotic systems, imbibing the concepts of the emergent methodology for extremely redundant systems. They typically consist of homogeneous autonomous robots, which resemble living animals that build swarms. Machine-learning techniques such as deep learning have played a remarkable role in controlling robotic behaviors in the real world or multi-agents in the simulation environment. In this special issue, we highlight five interesting papers that cover topics ranging from the analysis of the relationship between the congestion among autonomous robots and the task performances, to the decision making process among multiple autonomous agents. We thank the authors and reviewers of the papers and hope that this special issue encourages readers to explore recent topics and future studies in machine-learning applications for robotics and swarm systems.

Sign in / Sign up

Export Citation Format

Share Document