scholarly journals Benchmarking Natural Language Understanding Services for Building Conversational Agents

2021 ◽  
pp. 165-183
Author(s):  
Xingkun Liu ◽  
Arash Eshghi ◽  
Pawel Swietojanski ◽  
Verena Rieser
Keyword(s):  
Natural Language ◽  
Natural Language Understanding ◽  
Conversational Agents ◽  
Language Understanding

scholarly journals Human Annotated Dialogues Dataset for Natural Conversational Agents

2020 ◽  
Vol 10 (3) ◽  
pp. 762
Author(s):  
Erinc Merdivan ◽  
Deepika Singh ◽  
Sten Hanke ◽  
Johannes Kropf ◽  
Andreas Holzinger ◽  
...  
Keyword(s):  
Natural Language ◽  
Detailed Analysis ◽  
Human Perception ◽  
Natural Language Understanding ◽  
Industrial Applications ◽  
Benchmark Dataset ◽  
Conversational Agents ◽  
High Quality ◽  
Language Understanding ◽  
Major Drawback

Conversational agents are gaining huge popularity in industrial applications such as digital assistants, chatbots, and particularly systems for natural language understanding (NLU). However, a major drawback is the unavailability of a common metric to evaluate the replies against human judgement for conversational agents. In this paper, we develop a benchmark dataset with human annotations and diverse replies that can be used to develop such metric for conversational agents. The paper introduces a high-quality human annotated movie dialogue dataset, HUMOD, that is developed from the Cornell movie dialogues dataset. This new dataset comprises 28,500 human responses from 9500 multi-turn dialogue history-reply pairs. Human responses include: (i) ratings of the dialogue reply in relevance to the dialogue history; and (ii) unique dialogue replies for each dialogue history from the users. Such unique dialogue replies enable researchers in evaluating their models against six unique human responses for each given history. Detailed analysis on how dialogues are structured and human perception on dialogue score in comparison with existing models are also presented.

scholarly journals Modelling a Conversational Agent with Complex Emotional Intelligence

2020 ◽  
Vol 34 (10) ◽  
pp. 13710-13711
Author(s):  
Billal Belainine ◽  
Fatiha Sadat ◽  
Hakim Lounis
Keyword(s):  
Emotional Intelligence ◽  
Natural Language ◽  
Natural Language Understanding ◽  
Conversational Agent ◽  
Conversational Agents ◽  
Language Understanding ◽  
Leading Role ◽  
Complex Emotions

Chatbots or conversational agents have enjoyed great popularity in recent years. They surprisingly perform sensitive tasks in modern societies. However, despite the fact that they offer help, support, and fellowship, there is a task that is not yet mastered: dealing with complex emotions and simulating human sensations. This research aims to design an architecture for an emotional conversation agent for long-text conversations (multi-turns). This agent is intended to work in areas where the analysis of users feelings plays a leading role. This work refers to natural language understanding and response generation.

scholarly journals Alexa Prize — State of the Art in Conversational AI

AI Magazine ◽  
2018 ◽  
Vol 39 (3) ◽  
pp. 40-55 ◽  
Author(s):  
Chandra Khatri ◽  
Anu Venkatesh ◽  
Behnam Hedayatnia ◽  
Raefer Gabriel ◽  
Ashwin Ram ◽  
...  
Keyword(s):  
Natural Language ◽  
State Of The Art ◽  
Natural Language Understanding ◽  
Academic Community ◽  
The State ◽  
First Year ◽  
Grand Challenge ◽  
Conversational Agents ◽  
Language Understanding ◽  
Second Year

To advance the state of the art in conversational AI, Amazon launched the Alexa Prize, a 2.5 million dollar competition that challenges university teams to build conversational agents, or "socialbots", that can converse coherently and engagingly with humans on popular topics for 20 minutes. The Alexa Prize offers the academic community a unique opportunity to perform research at scale with real conversational data obtained by interacting with millions of Alexa users, along with user-provided ratings and feedback, over several months. This enables teams to effectively iterate, improve and evaluate their socialbots throughout the competition. Sixteen teams were selected for the inaugural competition last year. To build their socialbots, the students combined state-of-the-art techniques with their own novel strategies in the areas of Natural Language Understanding and Conversational AI. This article reports on the research conducted over the 2017-2018 year. While the 20 minute grand challenge was not achieved in the first year, the competition produced several conversational agents that advanced the state of the art, are interesting for everyday users to interact with, and help form a baseline for the second year of the competition.

The Distinction between Linguistic and Conceptual Semantics in Medical Terminology and its Implication for NLP-Based Knowledge Acquisition

1998 ◽  
Vol 37 (04/05) ◽  
pp. 327-333 ◽  
Author(s):  
F. Buekens ◽  
G. De Moor ◽  
A. Waagmeester ◽  
W. Ceusters
Keyword(s):  
Natural Language ◽  
Knowledge Acquisition ◽  
Natural Language Understanding ◽  
Knowledge Bases ◽  
Linguistic Knowledge ◽  
Medical Terminology ◽  
Language Understanding ◽  
Conceptual Semantics

AbstractNatural language understanding systems have to exploit various kinds of knowledge in order to represent the meaning behind texts. Getting this knowledge in place is often such a huge enterprise that it is tempting to look for systems that can discover such knowledge automatically. We describe how the distinction between conceptual and linguistic semantics may assist in reaching this objective, provided that distinguishing between them is not done too rigorously. We present several examples to support this view and argue that in a multilingual environment, linguistic ontologies should be designed as interfaces between domain conceptualizations and linguistic knowledge bases.

Automated Coding of Patient Discharge Summaries Using Conceptual Graphs

1995 ◽  
Vol 34 (04) ◽  
pp. 345-351 ◽  
Author(s):  
A. Burgun ◽  
L. P. Seka ◽  
D. Delamarre ◽  
P. Le Beux
Keyword(s):  
Natural Language ◽  
Patient Discharge ◽  
Natural Language Understanding ◽  
Assessment Procedure ◽  
Coding System ◽  
Language Understanding ◽  
Processing Scheme ◽  
Automated Coding ◽  
Conceptual Graph ◽  
Discharge Summaries

Abstract:In medicine, as in other domains, indexing and classification is a natural human task which is used for information retrieval and representation. In the medical field, encoding of patient discharge summaries is still a manual time-consuming task. This paper describes an automated coding system of patient discharge summaries from the field of coronary diseases into the ICD-9-CM classification. The system is developed in the context of the European AIM MENELAS project, a natural-language understanding system which uses the conceptual-graph formalism. Indexing is performed by using a two-step processing scheme; a first recognition stage is implemented by a matching procedure and a secondary selection stage is made according to the coding priorities. We show the general features of the necessary translation of the classification terms in the conceptual-graph model, and for the coding rules compliance. An advantage of the system is to provide an objective evaluation and assessment procedure for natural-language understanding.

Sign in / Sign up

Export Citation Format

Share Document