Extrinsic Versus Intrinsic Evaluation of Natural Language Generation for Spoken Dialogue Systems and Social Robotics

In the increasingly popular and diverse research area of social robotics, the primary goal is to develop robot agents that exhibit socially intelligent behaviour while interacting in a face-to-face context with human partners. An important aspect of face-to-face social conversation is fluent, flexible linguistic interaction; face-to-face dialogue is both the basic form of human communication and the richest and most flexible, combining unrestricted verbal expression with meaningful non-verbal acts such as gestures and facial displays, along with instantaneous, continuous collaboration between the speaker and the listener. In practice, however, most developers of social robots tend not to use the full possibilities of the unrestricted verbal expression afforded by face-to-face conversation; instead, they generally tend to employ relatively simplistic processes for choosing the words for their robots to say. This contrasts with the work carried out Natural Language Generation (NLG), the field of computational linguistics devoted to the automated production of high-quality linguistic content; while this research area is also an active one, in general most effort in NLG is focused on producing high-quality written text. This article summarizes the state of the art in the two individual research areas of social robotics and natural language generation. It then discusses the reasons why so few current social robots make use of more sophisticated generation techniques. Finally, an approach is proposed to bringing some aspects of NLG into social robotics, concentrating on techniques and tools that are most appropriate to the needs of socially interactive robots. This article is part of the theme issue ‘From social brains to social robots: applying neurocognitive insights to human–robot interaction’.

Download Full-text

Natural Language Generation as Incremental Planning Under Uncertainty: Adaptive Information Presentation for Statistical Dialogue Systems

IEEE/ACM Transactions on Audio Speech and Language Processing ◽

10.1109/tasl.2014.2315271 ◽

2014 ◽

Vol 22 (5) ◽

pp. 979-994 ◽

Cited By ~ 10

Author(s):

Verena Rieser ◽

Oliver Lemon ◽

Simon Keizer

Keyword(s):

Natural Language ◽

Natural Language Generation ◽

Dialogue Systems ◽

Planning Under Uncertainty ◽

Information Presentation ◽

Language Generation ◽

Incremental Planning

Download Full-text

Reinforcement adaptation of an attention-based neural natural language generator for spoken dialogue systems

Dialogue & Discourse ◽

10.5087/dad.2019.101 ◽

2019 ◽

Vol 10 (1) ◽

pp. 1-19

Author(s):

Matthieu Riou ◽

Bassam Jabaian ◽

Stéphane Huet ◽

Fabrice Lefèvre

Keyword(s):

Natural Language ◽

Short Term Memory ◽

Network Models ◽

Dialogue Systems ◽

Neural Network Models ◽

Spoken Dialogue Systems ◽

Spoken Dialogue ◽

Automatic Transcription ◽

Long Short Term Memory ◽

Expected Payoffs

Following some recent propositions to handle natural language generation in spoken dialogue systems with long short-term memory recurrent neural network models~\citep{Wen2016a} we first investigate a variant thereof with the objective of a better integration of the attention subnetwork. Then our next objective is to propose and evaluate a framework to adapt the NLG module online through direct interactions with the users. When doing so the basic way is to ask the user to utter an alternative sentence to express a particular dialogue act. But then the system has to decide between using an automatic transcription or to ask for a manual transcription. To do so a reinforcement learning approach based on an adversarial bandit scheme is retained. We show that by defining appropriately the rewards as a linear combination of expected payoffs and costs of acquiring the new data provided by the user, a system design can balance between improving the system's performance towards a better match with the user's preferences and the burden associated with it. Then the actual benefits of this system is assessed with a human evaluation, showing that the addition of more diverse utterances allows to produce sentences more satisfying for the user.

Download Full-text