Real user evaluation of spoken dialogue systems using Amazon Mechanical Turk

Author(s):  
F. Jurčíček ◽  
S. Keizer ◽  
Milica Gašić ◽  
François Mairesse ◽  
B. Thomson ◽  
...  
2014 ◽  
Author(s):  
Ioannis Klasinas ◽  
Elias Iosif ◽  
Katerina Louka ◽  
Alexandros Potamianos

2014 ◽  
Vol 21 (1) ◽  
pp. 46-51 ◽  
Author(s):  
Pierre Lison ◽  
Raveesh Meena

2006 ◽  
Vol 32 (3) ◽  
pp. 417-438 ◽  
Author(s):  
Diane Litman ◽  
Julia Hirschberg ◽  
Marc Swerts

This article focuses on the analysis and prediction of corrections, defined as turns where a user tries to correct a prior error made by a spoken dialogue system. We describe our labeling procedure of various corrections types and statistical analyses of their features in a corpus collected from a train information spoken dialogue system. We then present results of machine-learning experiments designed to identify user corrections of speech recognition errors. We investigate the predictive power of features automatically computable from the prosody of the turn, the speech recognition process, experimental conditions, and the dialogue history. Our best-performing features reduce classification error from baselines of 25.70–28.99% to 15.72%.


Sign in / Sign up

Export Citation Format

Share Document