Feedback-Driven Refinement of Mandarin Speech Recognition Result Based on Lattice Modification and Rescoring

2020 ◽  
pp. 1237-1247
Author(s):  
Xiangdong Wang ◽  
Yang Yang ◽  
Hong Liu ◽  
Yueliang Qian ◽  
Duan Jia
Keyword(s):  
Speech Recognition ◽  
Real World ◽  
Computational Cost ◽  
User Feedback ◽  
Recognition Result ◽  
Novel Method ◽  
Mandarin Speech Recognition ◽  
Recognition Errors ◽  
Dynamic Updating ◽  
Manual Correction

In real world applications of speech recognition, recognition errors are inevitable, and manual correction is necessary. This paper presents an approach for the refinement of Mandarin speech recognition result by exploiting user feedback. An interface incorporating character-based candidate lists and feedback-driven updating of the candidate lists is introduced. For dynamic updating of candidate lists, a novel method based on lattice modification and rescoring is proposed. By adding words with similar pronunciations to the candidates next to the corrected character into the lattice and then performing rescoring on the modified lattice, the proposed method can improve the accuracy of the candidate lists even if the correct characters are not in the original lattice, with much lower computational cost than that of the speech re-recognition methods. Experimental results show that the proposed method can reduce 24.03% of user inputs and improve average candidate rank by 25.31%.

Feedback-Driven Refinement of Mandarin Speech Recognition Result based on Lattice Modification and Rescoring

2017 ◽  
Vol 9 (2) ◽  
pp. 55-64
Author(s):  
Xiangdong Wang ◽  
Yang Yang ◽  
Hong Liu ◽  
Yueliang Qian ◽  
Duan Jia
Keyword(s):  
Speech Recognition ◽  
Real World ◽  
Computational Cost ◽  
Recognition Result ◽  
Real World Applications ◽  
Novel Method ◽  
Mandarin Speech Recognition ◽  
Recognition Errors ◽  
Dynamic Updating ◽  
Manual Correction

In real world applications of speech recognition, recognition errors are inevitable, and manual correction is necessary. This paper presents an approach for the refinement of Mandarin speech recognition result by exploiting user feedback. An interface incorporating character-based candidate lists and feedback-driven updating of the candidate lists is introduced. For dynamic updating of candidate lists, a novel method based on lattice modification and rescoring is proposed. By adding words with similar pronunciations to the candidates next to the corrected character into the lattice and then performing rescoring on the modified lattice, the proposed method can improve the accuracy of the candidate lists even if the correct characters are not in the original lattice, with much lower computational cost than that of the speech re-recognition methods. Experimental results show that the proposed method can reduce 24.03% of user inputs and improve average candidate rank by 25.31%.

scholarly journals Dynamic Acoustic Unit Augmentation with BPE-Dropout for Low-Resource End-to-End Speech Recognition

Sensors ◽  
2021 ◽  
Vol 21 (9) ◽  
pp. 3063
Author(s):  
Aleksandr Laptev ◽  
Andrei Andrusenko ◽  
Ivan Podluzhny ◽  
Anton Mitrofanov ◽  
Ivan Medennikov ◽  
...  
Keyword(s):  
Speech Recognition ◽  
Error Rate ◽  
Rapid Development ◽  
Computational Cost ◽  
Vocabulary Size ◽  
Word Error Rate ◽  
Low Resource ◽  
Steady Improvement ◽  
End To End ◽  
Asr System

With the rapid development of speech assistants, adapting server-intended automatic speech recognition (ASR) solutions to a direct device has become crucial. For on-device speech recognition tasks, researchers and industry prefer end-to-end ASR systems as they can be made resource-efficient while maintaining a higher quality compared to hybrid systems. However, building end-to-end models requires a significant amount of speech data. Personalization, which is mainly handling out-of-vocabulary (OOV) words, is another challenging task associated with speech assistants. In this work, we consider building an effective end-to-end ASR system in low-resource setups with a high OOV rate, embodied in Babel Turkish and Babel Georgian tasks. We propose a method of dynamic acoustic unit augmentation based on the Byte Pair Encoding with dropout (BPE-dropout) technique. The method non-deterministically tokenizes utterances to extend the token’s contexts and to regularize their distribution for the model’s recognition of unseen words. It also reduces the need for optimal subword vocabulary size search. The technique provides a steady improvement in regular and personalized (OOV-oriented) speech recognition tasks (at least 6% relative word error rate (WER) and 25% relative F-score) at no additional computational cost. Owing to the BPE-dropout use, our monolingual Turkish Conformer has achieved a competitive result with 22.2% character error rate (CER) and 38.9% WER, which is close to the best published multilingual system.

scholarly journals Characterizing and Predicting Corrections in Spoken Dialogue Systems

2006 ◽  
Vol 32 (3) ◽  
pp. 417-438 ◽  
Author(s):  
Diane Litman ◽  
Julia Hirschberg ◽  
Marc Swerts
Keyword(s):  
Speech Recognition ◽  
Predictive Power ◽  
Classification Error ◽  
Dialogue Systems ◽  
Dialogue System ◽  
Spoken Dialogue Systems ◽  
Experimental Conditions ◽  
Spoken Dialogue ◽  
Spoken Dialogue System ◽  
Recognition Errors

This article focuses on the analysis and prediction of corrections, defined as turns where a user tries to correct a prior error made by a spoken dialogue system. We describe our labeling procedure of various corrections types and statistical analyses of their features in a corpus collected from a train information spoken dialogue system. We then present results of machine-learning experiments designed to identify user corrections of speech recognition errors. We investigate the predictive power of features automatically computable from the prosody of the turn, the speech recognition process, experimental conditions, and the dialogue history. Our best-performing features reduce classification error from baselines of 25.70–28.99% to 15.72%.

Sign in / Sign up

Export Citation Format

Share Document