A Methodology to Design and Evaluate Agents

2018 ◽  
pp. 63-87
Author(s):  
Silvia Tamayo-Moreno ◽  
Diana Pérez-Marín
Keyword(s):  
Natural Language ◽  
Infinite Number ◽  
Computer Applications ◽  
Automatic Evaluation ◽  
Conversational Agents ◽  
Immediate Feedback ◽  
Level Of Difficulty ◽  
Syntactical Analysis

Pedagogic conversational agents are computer applications that interact with the students in natural language. They usually focus the dialogue on a certain topic under study. In this chapter, the authors propose the possibility of designing and evaluating agents according to a generic methodology. In particular, the methodology called MEDIE was applied to a study language agent. The main benefit of the study language agent called Lingu is that it is able to generate an infinite number of sentences, and it automatically generates the morphological and syntactical analysis from a given grammar. That way, students can practice with all the sentences they need, receive immediate feedback with automatic evaluation at their own rhythm, and the level of difficulty can be adapted to their particular competence of analysis. To be able to extend Lingu to different schools and needs, MEDIE has been applied as described in this chapter.

A Procedure to Engage Children in the Morphological and Syntax Analysis of Pedagogic Conversational Agent-Generated Sentences to Study Language

2015 ◽  
Vol 5 (2) ◽  
pp. 23-42
Author(s):  
Diana Pérez Marín
Keyword(s):  
Computer Applications ◽  
Computer Assisted ◽  
Conversational Agent ◽  
User Centered Design ◽  
Conversational Agents ◽  
Immediate Feedback ◽  
Syntax Analysis ◽  
Computer Teaching ◽  
Different Levels ◽  
Syntactical Analysis

Pedagogic Conversational Agents are computer applications that interact with the students in natural language. They usually focus the dialogue on a certain topic under study. In this paper, the authors propose the possibility of children studying morphology and syntax by using a Pedagogic Conversational Agent. The main benefit is that the agent is able to generate an infinite number of sentences and, it automatically generates the morphological and syntactical analysis from a given grammar. That way, students can practise with all the sentences they need, receive immediate feedback with automatic evaluation, at their own rhythm and, the level of difficulty can be adapted to their particular competence of analysis. Given the originality of this new computer assisted learning initiative, the authors have devised a procedure to engage the students in the dialogue with the agent to carry out the morphological and syntax analysis at five different levels of difficulty, and test the validity of the approach with a limited number of users according to the principles of User-Centered Design. The results gathered provide evidence of the goodness of the procedure and, they encourage us to keep working on this promising field of using pedagogic agents as computer teaching language assistants.

A Procedure to Create a Pedagogic Conversational Agent in Secondary Physics and Chemistry Education

2013 ◽  
Vol 9 (4) ◽  
pp. 94-112 ◽  
Author(s):  
Diana Pérez-Marín ◽  
Antonio Boza
Keyword(s):  
Natural Language ◽  
Chemistry Education ◽  
Teaching Method ◽  
Computer Applications ◽  
Conversational Agent ◽  
Conversational Agents ◽  
Innovative Teaching ◽  
Language Support ◽  
Teachers And Students ◽  
Innovative Teaching Method

Pedagogic Conversational Agents are computer applications that can interact with students in natural language. They have been used with satisfactory results on the instruction of several domains. The authors believe that they could also be useful for the instruction of Secondary Physics and Chemistry Education. Therefore, in this paper, the authors present a procedure to create an agent for that domain. First, teachers have to introduce the exercises with their correct answers. Secondly, students will be presented the exercises, and if the students know the answer, and if it is correct, more difficult exercises will be presented. Otherwise, step-by-step natural language support will be provided to guide the student towards the solution. It is the authors’ hypothesis that this innovative teaching method will be satisfactory and useful for teachers and students, and that by following the procedure more computer programmers can be encouraged to develop agents for other domains to be used by teachers and students at class.

How to Create a Pedagogic Conversational Agent for Teaching Computer Science

2019 ◽  
pp. 114-134
Author(s):  
José Miguel Ocaña ◽  
Elizabeth K. Morales-Urrutia ◽  
Diana Pérez-Marín ◽  
Silvia Tamayo-Moreno
Keyword(s):  
Natural Language ◽  
Computer Science ◽  
Primary Education ◽  
Computational Thinking ◽  
Computer Applications ◽  
Conversational Agent ◽  
Conversational Agents ◽  
Teacher Team

Pedagogic conversational agents are computer applications that can interact with students in natural language. They have been used with satisfactory results on the instruction of several domains. The authors believe that they could also be useful for the instruction of computer science programming. Therefore, in this chapter, the MEDIE methodology is described to explain how to create an agent to teach programming to primary education children and develop their computational thinking. The main steps are to communicate with the teacher team, to validate the interface, and to validate the functionality, practical sessions, and evaluation. The first two steps are covered in this chapter.

scholarly journals Comparison of disclosure/concealment of medical information given to conversational agents or to physicians

2021 ◽  
Vol 27 (1) ◽  
pp. 146045822199486
Author(s):  
Nicholas RJ Frick ◽  
Felix Brünker ◽  
Björn Ross ◽  
Stefan Stieglitz
Keyword(s):  
Natural Language ◽  
Medical Information ◽  
Frame Of Reference ◽  
Future Research ◽  
Negative Consequences ◽  
Conversational Agents ◽  
Online Questionnaire ◽  
Computer Based

Within the anamnesis, medical information is frequently withheld, incomplete, or incorrect, potentially causing negative consequences for the patient. The use of conversational agents (CAs), computer-based systems using natural language to interact with humans, may mitigate this problem. The present research examines whether CAs differ from physicians in their ability to elicit truthful disclosure and discourage concealment of medical information. We conducted an online questionnaire with German participants ( N = 148) to assess their willingness to reveal medical information. The results indicate that patients would rather disclose medical information to a physician than to a CA; there was no difference in the tendency to conceal information. This research offers a frame of reference for future research on applying CAs during the anamnesis to support physicians. From a practical view, physicians might gain better understanding of how the use of CAs can facilitate the anamnesis.

scholarly journals Human Annotated Dialogues Dataset for Natural Conversational Agents

2020 ◽  
Vol 10 (3) ◽  
pp. 762
Author(s):  
Erinc Merdivan ◽  
Deepika Singh ◽  
Sten Hanke ◽  
Johannes Kropf ◽  
Andreas Holzinger ◽  
...  
Keyword(s):  
Natural Language ◽  
Detailed Analysis ◽  
Human Perception ◽  
Natural Language Understanding ◽  
Industrial Applications ◽  
Benchmark Dataset ◽  
Conversational Agents ◽  
High Quality ◽  
Language Understanding ◽  
Major Drawback

Conversational agents are gaining huge popularity in industrial applications such as digital assistants, chatbots, and particularly systems for natural language understanding (NLU). However, a major drawback is the unavailability of a common metric to evaluate the replies against human judgement for conversational agents. In this paper, we develop a benchmark dataset with human annotations and diverse replies that can be used to develop such metric for conversational agents. The paper introduces a high-quality human annotated movie dialogue dataset, HUMOD, that is developed from the Cornell movie dialogues dataset. This new dataset comprises 28,500 human responses from 9500 multi-turn dialogue history-reply pairs. Human responses include: (i) ratings of the dialogue reply in relevance to the dialogue history; and (ii) unique dialogue replies for each dialogue history from the users. Such unique dialogue replies enable researchers in evaluating their models against six unique human responses for each given history. Detailed analysis on how dialogues are structured and human perception on dialogue score in comparison with existing models are also presented.

Using Conceptual Models to Implement Natural Language Pedagogic Agent-Student Conversations

2013 ◽  
Vol 3 (2) ◽  
pp. 29-47 ◽  
Author(s):  
Diana Pérez-Marín ◽  
Carlos Caballero
Keyword(s):  
Natural Language ◽  
Conceptual Models ◽  
Interactive Learning Environments ◽  
Software Systems ◽  
Virtual Agents ◽  
Learning Needs ◽  
Conversational Agents ◽  
Agent Based ◽  
Intelligent Virtual Agents ◽  
Mixed Initiative

The last several decades have seen a growing trend in incorporating pedagogic conversational agents in interactive learning environments. Software systems have increasingly integrated intelligent virtual agents that can interact with students in natural language to fulfill specific tasks such as reviewing content or providing tutor training. The use of an agent-based approach in education has shown many benefits. However, certain design and development issues are still unresolved. This article focuses on the potentials of employing conceptual models to generate agent-student dialog and introduces a new mixed-initiative general domain agent called JARO. The authors report on the procedure for creating the initial conceptual model and discuss its use in guiding agent-student conversations adapted to students' individual learning needs. The stages of implementation of the model as well as the model's viability tested in a proof-of-concept experiment are addressed.

scholarly journals Modelling a Conversational Agent with Complex Emotional Intelligence

2020 ◽  
Vol 34 (10) ◽  
pp. 13710-13711
Author(s):  
Billal Belainine ◽  
Fatiha Sadat ◽  
Hakim Lounis
Keyword(s):  
Emotional Intelligence ◽  
Natural Language ◽  
Natural Language Understanding ◽  
Conversational Agent ◽  
Conversational Agents ◽  
Language Understanding ◽  
Leading Role ◽  
Complex Emotions

Chatbots or conversational agents have enjoyed great popularity in recent years. They surprisingly perform sensitive tasks in modern societies. However, despite the fact that they offer help, support, and fellowship, there is a task that is not yet mastered: dealing with complex emotions and simulating human sensations. This research aims to design an architecture for an emotional conversation agent for long-text conversations (multi-turns). This agent is intended to work in areas where the analysis of users feelings plays a leading role. This work refers to natural language understanding and response generation.

Sign in / Sign up

Export Citation Format

Share Document