English as a second language is widely used in countries such as Malaysia and Indonesia, and it is common for English words to appear in Malay and Indonesian sentences. Malay and Indonesian have high homology and relatively few electronic language resources. We combine the corpus datasets of these two similar languages to design and implement a HMM–DNN-based cross-lingual speech synthesis system for Malay (including English words) and Indonesian (including English words). The methods used include: sharing synthesis units between Malay, Indonesian, and English, designing unified context attributes and question set in the process of acoustic model training, speaker-adaptive training with speech corpus of these three languages, and synthesizing speech using speaker-dependent Malay and Indonesian acoustic models. Experimental results show that the speech synthesis quality of the system is better than the traditional Hidden Markov model-based cross-lingual speech synthesis system.
