Background:
In India, thousands of languages or dialects are in use. Most Indian dialects are low asset dialects.
A well-performing Automatic Speech Recognition (ASR) system for Indian languages is unavailable due to a lack of
resources. Hindi is one of them as large vocabulary Hindi speech datasets are not freely available. We have only a few hours
of transcribed Hindi speech dataset. There is a lot of time and money involved in creating a well-transcribed speech dataset.
Thus, developing a real-time ASR system with a few hours of the training dataset is the most challenging task. The different
techniques like data augmentation, semi-supervised training, multilingual architecture, and transfer learning, have been
reported in the past to tackle the fewer speech data issues. In this paper, we examine the effect of multilingual acoustic
modeling in ASR systems for the Hindi language.
Objective:
This article’s objective is to develop a high accuracy Hindi ASR system with a reasonable computational load
and high accuracy using a few hours of training data.
Method:
To achieve this goal we used Multilingual training with Time Delay Neural Network- Bidirectional Long Short
Term Memory (TDNN-BLSTM) acoustic modeling. Multilingual acoustic modeling has significantly improved the ASR
system's performance for low and limited resource languages. The common practice is to train the acoustic model by
merging data from similar languages. In this work, we use three Indian languages, namely Hindi, Marathi, and Bengali.
Hindi with 2.5 hours of training data and Marathi with 5.5 hours of training data and Bengali with 28.5 hours of transcribed
data, was used in this work to train the proposed model.
Results:
The Kaldi toolkit was used to perform all the experiments. The paper is investigated over three main points. First,
we present the monolingual ASR system using various Neural Network (NN) based acoustic models. Second, we show that
Recurrent Neural Network (RNN) language modeling helps to improve the ASR performance further. Finally, we show that
a multilingual ASR system significantly reduces the Word Error Rate (WER) (absolute 2% WER reduction for Hindi and
3% for the Marathi language). In all the three languages, the proposed TDNN-BLSTM-A multilingual acoustic models help
to get the lowest WER.
Conclusion:
The multilingual hybrid TDNN-BLSTM-A architecture shows a 13.67% relative improvement over the
monolingual Hindi ASR system. The best WER of 8.65% was recorded for Hindi ASR. For Marathi and Bengali, the
proposed TDNN-BLSTM-A acoustic model reports the best WER of 30.40% and 10.85%.
