scholarly journals Pronunciation augmentation for Mandarin-English code-switching speech recognition

2021 ◽  
Vol 2021 (1) ◽  
Author(s):  
Yanhua Long ◽  
Shuang Wei ◽  
Jie Lian ◽  
Yijie Li
Keyword(s):  
Speech Recognition ◽  
Code Switching ◽  
Training Data ◽  
Acoustic Data ◽  
Pronunciation Variation ◽  
Learning Framework ◽  
Switching Property ◽  
Recognition Systems ◽  
Language Universal ◽  
Heavy Training

AbstractCode-switching (CS) refers to the phenomenon of using more than one language in an utterance, and it presents great challenge to automatic speech recognition (ASR) due to the code-switching property in one utterance, the pronunciation variation phenomenon of the embedding language words and the heavy training data sparse problem. This paper focuses on the Mandarin-English CS ASR task. We aim at dealing with the pronunciation variation and alleviating the sparse problem of code-switches by using pronunciation augmentation methods. An English-to-Mandarin mix-language phone mapping approach is first proposed to obtain a language-universal CS lexicon. Based on this lexicon, an acoustic data-driven lexicon learning framework is further proposed to learn new pronunciations to cover the accents, mis-pronunciations, or pronunciation variations of those embedding English words. Experiments are performed on real CS ASR tasks. Effectiveness of the proposed methods are examined on all of the conventional, hybrid, and the recent end-to-end speech recognition systems. Experimental results show that both the learned phone mapping and augmented pronunciations can significantly improve the performance of code-switching speech recognition.

scholarly journals Phonetic Variation Modeling and a Language Model Adaptation for Korean English Code-Switching Speech Recognition

2021 ◽  
Vol 11 (6) ◽  
pp. 2866
Author(s):  
Damheo Lee ◽  
Donghyun Kim ◽  
Seung Yun ◽  
Sanghun Kim
Keyword(s):  
Speech Recognition ◽  
Language Model ◽  
Reduction Rate ◽  
Code Switching ◽  
Training Data ◽  
Target Domain ◽  
Phonetic Variation ◽  
Language Model Adaptation ◽  
Imbalanced Training Data ◽  
Lm Adaptation

In this paper, we propose a new method for code-switching (CS) automatic speech recognition (ASR) in Korean. First, the phonetic variations in English pronunciation spoken by Korean speakers should be considered. Thus, we tried to find a unified pronunciation model based on phonetic knowledge and deep learning. Second, we extracted the CS sentences semantically similar to the target domain and then applied the language model (LM) adaptation to solve the biased modeling toward Korean due to the imbalanced training data. In this experiment, training data were AI Hub (1033 h) in Korean and Librispeech (960 h) in English. As a result, when compared to the baseline, the proposed method improved the error reduction rate (ERR) by up to 11.6% with phonetic variant modeling and by 17.3% when semantically similar sentences were applied to the LM adaptation. If we considered only English words, the word correction rate improved up to 24.2% compared to that of the baseline. The proposed method seems to be very effective in CS speech recognition.

scholarly journals Investigating the effects of gender, dialect, and training size on the performance of Arabic speech recognition

2020 ◽  
Vol 54 (4) ◽  
pp. 975-998
Author(s):  
Eiman Alsharhan ◽  
Allan Ramsay
Keyword(s):  
Speech Recognition ◽  
Arabic Language ◽  
Learning Curves ◽  
Training Data ◽  
Language Resources ◽  
Acoustic Models ◽  
Pronunciation Variation ◽  
Register Variation ◽  
Fine Grained ◽  
Dialectal Arabic

Abstract Research in Arabic automatic speech recognition (ASR) is constrained by datasets of limited size, and of highly variable content and quality. Arabic-language resources vary in the attributes that affect language resources in other languages (noise, channel, speaker, genre), but also vary significantly in the dialect and level of formality of the spoken Arabic they capture. Many languages suffer similar levels of cross-dialect and cross-register acoustic variability, but these effects have been under-studied. This paper is an experimental analysis of the interaction between classical ASR corpus-compensation methods (feature selection, data selection, gender-dependent acoustic models) and the dialect-dependent/register-dependent variation among Arabic ASR corpora. The first interaction studied in this paper is that between acoustic recording quality and discrete pronunciation variation. Discrete pronunciation variation can be compensated by using grapheme-based instead of phone-based acoustic models, and by filtering out speakers with insufficient training data; the latter technique also helps to compensate for poor recording quality, which is further compensated by eliminating delta-delta acoustic features. All three techniques, together, reduce Word Error Rate (WER) by between 3.24% and 5.35%. The second aspect of dialect and register variation to be considered is variation in the fine-grained acoustic pronunciations of each phoneme in the language. Experimental results prove that gender and dialect are the principal components of variation in speech, therefore, building gender and dialect-specific models leads to substantial decreases in WER. In order to further explore the degree of acoustic differences between phone models required for each of the dialects of Arabic, cross-dialect experiments are conducted to measure how far apart Arabic dialects are acoustically in order to make a better decision about the minimal number of recognition systems needed to cover all dialectal Arabic. Finally, the research addresses an important question: how much training data is needed for building efficient speaker-independent ASR systems? This includes developing some learning curves to find out how large must the training set be to achieve acceptable performance.

Acoustic data augmentation for Mandarin-English code-switching speech recognition

2020 ◽  
Vol 161 ◽  
pp. 107175 ◽  
Author(s):  
Yanhua Long ◽  
Yijie Li ◽  
Qiaozheng Zhang ◽  
Shuang Wei ◽  
Hong Ye ◽  
...  
Keyword(s):  
Speech Recognition ◽  
Data Augmentation ◽  
Code Switching ◽  
Acoustic Data

scholarly journals Investigations on speech recognition systems for low-resource dialectal Arabic-English code-switching speech

2021 ◽  
pp. 101278
Author(s):  
Injy Hamed ◽  
Pavel Denisov ◽  
Chia-Yu Li ◽  
Mohamed Elmahdy ◽  
Slim Abdennadher ◽  
...  
Keyword(s):  
Speech Recognition ◽  
Code Switching ◽  
Low Resource ◽  
Recognition Systems ◽  
Dialectal Arabic

Exploring E2E speech recognition systems for new languages

2018 ◽  
Author(s):  
Conrad Bernath ◽  
Aitor Alvarez ◽  
Haritz Arzelus ◽  
Carlos David Martínez
Keyword(s):  
Speech Recognition ◽  
Recognition Systems

Improving Transformer-Based Speech Recognition Systems with Compressed Structure and Speech Attributes Augmentation

2019 ◽  
Author(s):  
Sheng Li ◽  
Dabre Raj ◽  
Xugang Lu ◽  
Peng Shen ◽  
Tatsuya Kawahara ◽  
...  
Keyword(s):  
Speech Recognition ◽  
Recognition Systems

scholarly journals SCOUR: a stepwise machine learning framework for predicting metabolite-dependent regulatory interactions

2021 ◽  
Vol 22 (1) ◽  
Author(s):  
Justin Y. Lee ◽  
Britney Nguyen ◽  
Carlos Orosco ◽  
Mark P. Styczynski
Keyword(s):  
Machine Learning ◽  
Metabolic Networks ◽  
Sampling Frequency ◽  
Low Noise ◽  
Training Data ◽  
High Noise ◽  
Regulatory Interactions ◽  
Learning Framework ◽  
Metabolic Systems ◽  
Noise Data

Abstract Background The topology of metabolic networks is both well-studied and remarkably well-conserved across many species. The regulation of these networks, however, is much more poorly characterized, though it is known to be divergent across organisms—two characteristics that make it difficult to model metabolic networks accurately. While many computational methods have been built to unravel transcriptional regulation, there have been few approaches developed for systems-scale analysis and study of metabolic regulation. Here, we present a stepwise machine learning framework that applies established algorithms to identify regulatory interactions in metabolic systems based on metabolic data: stepwise classification of unknown regulation, or SCOUR. Results We evaluated our framework on both noiseless and noisy data, using several models of varying sizes and topologies to show that our approach is generalizable. We found that, when testing on data under the most realistic conditions (low sampling frequency and high noise), SCOUR could identify reaction fluxes controlled only by the concentration of a single metabolite (its primary substrate) with high accuracy. The positive predictive value (PPV) for identifying reactions controlled by the concentration of two metabolites ranged from 32 to 88% for noiseless data, 9.2 to 49% for either low sampling frequency/low noise or high sampling frequency/high noise data, and 6.6–27% for low sampling frequency/high noise data, with results typically sufficiently high for lab validation to be a practical endeavor. While the PPVs for reactions controlled by three metabolites were lower, they were still in most cases significantly better than random classification. Conclusions SCOUR uses a novel approach to synthetically generate the training data needed to identify regulators of reaction fluxes in a given metabolic system, enabling metabolomics and fluxomics data to be leveraged for regulatory structure inference. By identifying and triaging the most likely candidate regulatory interactions, SCOUR can drastically reduce the amount of time needed to identify and experimentally validate metabolic regulatory interactions. As high-throughput experimental methods for testing these interactions are further developed, SCOUR will provide critical impact in the development of predictive metabolic models in new organisms and pathways.

scholarly journals Human-robot-interaction using cloud-based speech recognition systems

Procedia CIRP ◽  
2021 ◽  
Vol 97 ◽  
pp. 130-135
Author(s):  
Christian Deuerlein ◽  
Moritz Langer ◽  
Julian Seßner ◽  
Peter Heß ◽  
Jörg Franke
Keyword(s):  
Speech Recognition ◽  
Human Robot Interaction ◽  
Robot Interaction ◽  
Recognition Systems

MULFE: Multi-Label Learning via Label-Specific Feature Space Ensemble

2021 ◽  
Vol 16 (1) ◽  
pp. 1-24
Author(s):  
Yaojin Lin ◽  
Qinghua Hu ◽  
Jinghua Liu ◽  
Xingquan Zhu ◽  
Xindong Wu
Keyword(s):  
Empirical Studies ◽  
Feature Space ◽  
Training Data ◽  
Data Sets ◽  
Learning Framework ◽  
Feature Spaces ◽  
Public Data ◽  
Margin Distribution ◽  
Label Correlations ◽  
Label Correlation

In multi-label learning, label correlations commonly exist in the data. Such correlation not only provides useful information, but also imposes significant challenges for multi-label learning. Recently, label-specific feature embedding has been proposed to explore label-specific features from the training data, and uses feature highly customized to the multi-label set for learning. While such feature embedding methods have demonstrated good performance, the creation of the feature embedding space is only based on a single label, without considering label correlations in the data. In this article, we propose to combine multiple label-specific feature spaces, using label correlation, for multi-label learning. The proposed algorithm, mu lti- l abel-specific f eature space e nsemble (MULFE), takes consideration label-specific features, label correlation, and weighted ensemble principle to form a learning framework. By conducting clustering analysis on each label’s negative and positive instances, MULFE first creates features customized to each label. After that, MULFE utilizes the label correlation to optimize the margin distribution of the base classifiers which are induced by the related label-specific feature spaces. By combining multiple label-specific features, label correlation based weighting, and ensemble learning, MULFE achieves maximum margin multi-label classification goal through the underlying optimization framework. Empirical studies on 10 public data sets manifest the effectiveness of MULFE.

Sign in / Sign up

Export Citation Format

Share Document