Multi-modal referring expressions in human-human task descriptions and their implications for human-robot interaction

2016 ◽  
Vol 17 (2) ◽  
pp. 180-210
Author(s):  
Stephanie Gross ◽  
Brigitte Krenn ◽  
Matthias Scheutz
Keyword(s):  
Language Processing ◽  
Eye Gaze ◽  
Human Robot Interaction ◽  
Referring Expressions ◽  
Visual Channel ◽  
Student Interactions ◽  
Seamless Integration ◽  
Teacher Student ◽  
Processing Module ◽  
Task Descriptions

Abstract Human instructors often refer to objects and actions involved in a task description using both linguistic and non-linguistic means of communication. Hence, for robots to engage in natural human-robot interactions, we need to better understand the various relevant aspects of human multi-modal task descriptions. We analyse reference resolution to objects in a data collection comprising two object manipulation tasks (22 teacher student interactions in Task 1 and 16 in Task 2) and find that 78.76% of all referring expressions to the objects relevant in Task 1 are verbally underspecified and 88.64% of all referring expressions are verbally underspecified in Task 2. The data strongly suggests that a language processing module for robots must be genuinely multi-modal, allowing for seamless integration of information transmitted in the verbal and the visual channel, whereby tracking the speaker’s eye gaze and gestures as well as object recognition are necessary preconditions.

Teacher Identification of Behavior Problems among Junior High School Students: A Preliminary Study

1981 ◽  
Vol 6 (4) ◽  
pp. 219-222 ◽  
Author(s):  
Lyman W. Boomer ◽  
Tom R. King
Keyword(s):  
High School ◽  
High School Students ◽  
Behavior Problems ◽  
Junior High School ◽  
Junior High ◽  
Junior High School Students ◽  
School Students ◽  
Student Interactions ◽  
Teacher Student ◽  
Scanning Behavior

Factors associated with teacher identification of behavior problems among junior high school students were investigated. Teacher-student interactions were compared to examine the differences between students identified as emotionally disturbed and non-identified students. Results indicated there were significant differences between interaction profiles. These were in the areas of student attention-to-task and student scanning behavior while the teacher was instructing.

Reciprocity in the Teacher-Pupil and Peer Verbal Interactions of Learning Disabled, Behavior-Disordered, and Regular Education Students

1984 ◽  
Vol 7 (1) ◽  
pp. 30-38 ◽  
Author(s):  
Steven R. Moore ◽  
Richard L. Simpson
Keyword(s):  
Regular Education ◽  
Learning Disabled ◽  
Education Students ◽  
Student Interactions ◽  
Verbal Interactions ◽  
Teacher Aide ◽  
Target Behaviors ◽  
Observation Instrument ◽  
Teacher Student ◽  
Behavior Disordered

The purpose of this study was to examine the reciprocal interactions of learning disabled (LD), behavior-disordered (BD), and regular education students. The interactions of 15 students from each diagnostic group (LD, BD, regular education) and their peers, teachers, and classroom aides were observed using a behavior observation instrument designed to monitor (a) frequency of 14 target behaviors, (b) direction of the interaction (i.e., given to or received from), and (c) status of the party involved in the interaction (i.e., peer, teacher, aide). A correlational analysis indicated that negative peer-student interactions were reciprocal. In contrast, neither positive or negative teacher-student interactions nor positive peer-student interactions were reciprocal. First-order conditional probabilities (i.e., the probability of a statement being followed by a selected response) showed that BD, LD, and regular students responded to others in a similar manner. Likewise, the teachers of the three groups were similar in their responses to students. In all groups, positive, negative, and neutral statements were most likely to be followed either by an absence of a response or by a neutral response.

The Verbmobil prototype system – a software engineering perspective

1999 ◽  
Vol 5 (1) ◽  
pp. 95-112 ◽  
Author(s):  
THOMAS BUB ◽  
JOHANNES SCHWINN
Keyword(s):  
Software Engineering ◽  
Computational Linguistics ◽  
Language Processing ◽  
System Integration ◽  
Evolutionary Process ◽  
Prototype System ◽  
Functional Architecture ◽  
Seamless Integration ◽  
Extensible Software ◽  
Transfer Problems

Verbmobil represents a new generation of speech-to-speech translation systems in which spontaneously spoken language, speaker independence and adaptability as well as the combination of deep and shallow approaches to the analysis and transfer problems are the main features. The project brought together researchers from the fields of signal processing, computational linguistics and artificial intelligence. Verbmobil goes beyond the state-of-the-art in each of these areas, but its main achievement is the seamless integration of them. The first project phase (1993–1996) has been followed up by the second project phase (1997–2000), which aims at applying the results to further languages and at integrating innovative telecooperation techniques. Quite apart from the speech and language processing issues, the size and complexity of the project represent an extreme challenge on the areas of project management and software engineering:[bull ] 50 researchers from 29 organizations at different sites in different countries are involved in the software development process,[bull ] to reuse existing software, hardware, knowledge and experience, only a few technical restrictions could be given to the partners.In this article we describe the Verbmobil prototype system from a software-engineering perspective. We discuss:[bull ] the modularized functional architecture,[bull ] the flexible and extensible software architecture which reflects that functional architecture,[bull ] the evolutionary process of system integration,[bull ] the communication-based organizational structure of the project,[bull ] the evaluation of the system operational by the end of the first project phase.

AI and libraries: trends and projections

2021 ◽  
Vol ahead-of-print (ahead-of-print) ◽  
Author(s):  
Adetoun A. Oyelude
Keyword(s):  
Language Processing ◽  
Service Provision ◽  
Design Methodology ◽  
Human Robot Interaction ◽  
Human Intelligence ◽  
Library Service ◽  
Robot Interaction ◽  
Content Type ◽  
New Horizons ◽  
Advanced Technologies

Purpose This paper aims to focus on the trends and projection for future use of artificial intelligence (AI) in libraries. AI technologies is the latest among the technologies being used in libraries. The technology has systems that have natural language processing, machine learning and pattern recognition capabilities that make service provision easier for libraries. Design/methodology/approach Systematic literature review is done, exploring blogs and wikis, to collect information on the ways in which AI is used and can be futuristically used in libraries. Findings This paper found that uses of AI in libraries entailed enhanced services such as content indexing, document matching, content mapping content summarization and many others. AI possibilities were also found to include improving the technology of gripping, localizing and human–robot interaction and also having artificial superintelligence, the hypothetical AI that surpasses human intelligence and abilities. Originality/value It is concluded that advanced technologies that AI are, will help librarians to open up new horizons and solve challenges that crop up in library service delivery.

scholarly journals Deep Learning Based BCI Control of a Robotic Service Assistant Using Intelligent Goal Formulation

2018 ◽  
Author(s):  
D. Kuhner ◽  
L.D.J. Fiederer ◽  
J. Aldinger ◽  
F. Burget ◽  
M. Völker ◽  
...  
Keyword(s):  
Deep Learning ◽  
User Study ◽  
Human Robot Interaction ◽  
Service Robots ◽  
Referring Expressions ◽  
Non Invasive ◽  
Neuronal Signal ◽  
High Level ◽  
User Friendly ◽  
Robotic Tasks

AbstractAs autonomous service robots become more affordable and thus available for the general public, there is a growing need for user-friendly interfaces to control these systems. Control interfaces typically get more complicated with increasing complexity of the robotic tasks and the environment. Traditional control modalities as touch, speech or gesture commands are not necessarily suited for all users. While non-expert users can make the effort to familiarize themselves with a robotic system, paralyzed users may not be capable of controlling such systems even though they need robotic assistance most. In this paper, we present a novel framework, that allows these users to interact with a robotic service assistant in a closed-loop fashion, using only thoughts. The system is composed of several interacting components: non-invasive neuronal signal recording and co-adaptive deep learning which form the brain-computer interface (BCI), high-level task planning based on referring expressions, navigation and manipulation planning as well as environmental perception. We extensively evaluate the BCI in various tasks, determine the performance of the goal formulation user interface and investigate its intuitiveness in a user study. Furthermore, we demonstrate the applicability and robustness of the system in real world scenarios, considering fetch-and-carry tasks and tasks involving human-robot interaction. As our results show, the system is capable of adapting to frequent changes in the environment and reliably accomplishes given tasks within a reasonable amount of time. Combined with high-level planning using referring expressions and autonomous robotic systems, interesting new perspectives open up for non-invasive BCI-based human-robot interactions.

Sign in / Sign up

Export Citation Format

Share Document