scholarly journals Language to Action: Towards Interactive Task Learning with Physical Agents

2018 ◽  
Author(s):  
Joyce Y. Chai ◽  
Qiaozi Gao ◽  
Lanbo She ◽  
Shaohua Yang ◽  
Sari Saba-Sadiya ◽  
...  
Keyword(s):  
Perception And Action ◽  
Human Learning ◽  
Physical Causality ◽  
Task Learning ◽  
Challenges And Opportunities ◽  
Language Communication ◽  
Interactive Task ◽  
New Generation ◽  
Cognitive Robots ◽  
Natural Language Communication

Language communication plays an important role in human learning and knowledge acquisition. With the emergence of a new generation of cognitive robots, empowering these robots to learn directly from human partners becomes increasingly important. This paper gives a brief introduction to interactive task learning where humans can teach physical agents new tasks through natural language communication and action demonstration. It discusses research challenges and opportunities in language and communication grounding that are critical in this process. It further highlights the importance of commonsense knowledge, particularly the very basic physical causality knowledge, in grounding language to perception and action.

scholarly journals Interactive task learning via embodied corrective feedback

2020 ◽  
Vol 34 (2) ◽  
Author(s):  
Mattias Appelgren ◽  
Alex Lascarides
Keyword(s):  
Learning Process ◽  
Corrective Feedback ◽  
Domain Model ◽  
Stanford University ◽  
Cambridge University ◽  
Task Learning ◽  
Discourse Coherence ◽  
Linguistic Evidence ◽  
Interactive Task

Abstract This paper addresses a task in Interactive Task Learning (Laird et al. IEEE Intell Syst 32:6–21, 2017). The agent must learn to build towers which are constrained by rules, and whenever the agent performs an action which violates a rule the teacher provides verbal corrective feedback: e.g. “No, red blocks should be on blue blocks”. The agent must learn to build rule compliant towers from these corrections and the context in which they were given. The agent is not only ignorant of the rules at the start of the learning process, but it also has a deficient domain model, which lacks the concepts in which the rules are expressed. Therefore an agent that takes advantage of the linguistic evidence must learn the denotations of neologisms and adapt its conceptualisation of the planning domain to incorporate those denotations. We show that by incorporating constraints on interpretation that are imposed by discourse coherence into the models for learning (Hobbs in On the coherence and structure of discourse, Stanford University, Stanford, 1985; Asher et al. in Logics of conversation, Cambridge University Press, Cambridge, 2003), an agent which utilizes linguistic evidence outperforms a strong baseline which does not.

scholarly journals Teaching semantics and skills for human-robot collaboration

2019 ◽  
Vol 10 (1) ◽  
pp. 318-329 ◽  
Author(s):  
Alexandre Angleraud ◽  
Quentin Houbre ◽  
Roel Pieters
Keyword(s):  
Natural Language ◽  
Web Application ◽  
Robotic Arm ◽  
Task Execution ◽  
Learning Skills ◽  
Action Semantics ◽  
Language Communication ◽  
The Cost ◽  
Natural Language Communication ◽  
Human Robot Collaboration

AbstractRecent advances in robotics allow for collaboration between humans and machines in performing tasks at home or in industrial settings without harming the life of the user. While humans can easily adapt to each other and work in team, it is not as trivial for robots. In their case, interaction skills typically come at the cost of extensive programming and teaching. Besides, understanding the semantics of a task is necessary to work efficiently and react to changes in the task execution process. As a result, in order to achieve seamless collaboration, appropriate reasoning, learning skills and interaction capabilities are needed. For us humans, a cornerstone of our communication is language that we use to teach, coordinate and communicate. In this paper we thus propose a system allowing (i) to teach new action semantics based on the already available knowledge and (ii) to use natural language communication to resolve ambiguities that could arise while giving commands to the robot. Reasoning then allows new skills to be performed either autonomously or in collaboration with a human. Teaching occurs through a web application and motions are learned with physical demonstration of the robotic arm. We demonstrate the utility of our system in two scenarios and reflect upon the challenges that it introduces.

Sign in / Sign up

Export Citation Format

Share Document