scholarly journals Multi-Path and Group-Loss-Based Network for Speech Emotion Recognition in Multi-Domain Datasets

Sensors ◽  
2021 ◽  
Vol 21 (5) ◽  
pp. 1579 ◽  
Author(s):  
Kyoung Ju Noh ◽  
Chi Yoon Jeong ◽  
Jiyoun Lim ◽  
Seungeun Chung ◽  
Gague Kim ◽  
...  
Keyword(s):  
Emotion Recognition ◽  
Short Term Memory ◽  
Domain Adaptation ◽  
Classification Model ◽  
Speech Emotion Recognition ◽  
Target Domain ◽  
Model Generalization ◽  
Speech Database ◽  
Emotion Labels ◽  
Temporal Feature

Speech emotion recognition (SER) is a natural method of recognizing individual emotions in everyday life. To distribute SER models to real-world applications, some key challenges must be overcome, such as the lack of datasets tagged with emotion labels and the weak generalization of the SER model for an unseen target domain. This study proposes a multi-path and group-loss-based network (MPGLN) for SER to support multi-domain adaptation. The proposed model includes a bidirectional long short-term memory-based temporal feature generator and a transferred feature extractor from the pre-trained VGG-like audio classification model (VGGish), and it learns simultaneously based on multiple losses according to the association of emotion labels in the discrete and dimensional models. For the evaluation of the MPGLN SER as applied to multi-cultural domain datasets, the Korean Emotional Speech Database (KESD), including KESDy18 and KESDy19, is constructed, and the English-speaking Interactive Emotional Dyadic Motion Capture database (IEMOCAP) is used. The evaluation of multi-domain adaptation and domain generalization showed 3.7% and 3.5% improvements, respectively, of the F1 score when comparing the performance of MPGLN SER with a baseline SER model that uses a temporal feature generator. We show that the MPGLN SER efficiently supports multi-domain adaptation and reinforces model generalization.

scholarly journals Pre-trained Deep Convolution Neural Network Model With Attention for Speech Emotion Recognition

2021 ◽  
Vol 12 ◽  
Author(s):  
Hua Zhang ◽  
Ruoyun Gou ◽  
Jili Shang ◽  
Fangyao Shen ◽  
Yifan Wu ◽  
...  
Keyword(s):  
Neural Network ◽  
Emotion Recognition ◽  
Short Term Memory ◽  
Convolution Neural Network ◽  
Classification Model ◽  
Speech Emotion Recognition ◽  
Deep Convolution Neural Network ◽  
Long Short Term Memory ◽  
High Level ◽  
Better Than

Speech emotion recognition (SER) is a difficult and challenging task because of the affective variances between different speakers. The performances of SER are extremely reliant on the extracted features from speech signals. To establish an effective features extracting and classification model is still a challenging task. In this paper, we propose a new method for SER based on Deep Convolution Neural Network (DCNN) and Bidirectional Long Short-Term Memory with Attention (BLSTMwA) model (DCNN-BLSTMwA). We first preprocess the speech samples by data enhancement and datasets balancing. Secondly, we extract three-channel of log Mel-spectrograms (static, delta, and delta-delta) as DCNN input. Then the DCNN model pre-trained on ImageNet dataset is applied to generate the segment-level features. We stack these features of a sentence into utterance-level features. Next, we adopt BLSTM to learn the high-level emotional features for temporal summarization, followed by an attention layer which can focus on emotionally relevant features. Finally, the learned high-level emotional features are fed into the Deep Neural Network (DNN) to predict the final emotion. Experiments on EMO-DB and IEMOCAP database obtain the unweighted average recall (UAR) of 87.86 and 68.50%, respectively, which are better than most popular SER methods and demonstrate the effectiveness of our propose method.

scholarly journals Universum Autoencoder-Based Domain Adaptation for Speech Emotion Recognition

2017 ◽  
Vol 24 (4) ◽  
pp. 500-504 ◽  
Author(s):  
Jun Deng ◽  
Xinzhou Xu ◽  
Zixing Zhang ◽  
Sascha Fruhholz ◽  
Bjorn Schuller
Keyword(s):  
Emotion Recognition ◽  
Domain Adaptation ◽  
Speech Emotion Recognition

scholarly journals Speech Emotion Recognition System

2021 ◽  
pp. 156-159
Author(s):  
Sourabh Suke ◽  
Ganesh Regulwar ◽  
Nikesh Aote ◽  
Pratik Chaudhari ◽  
Rajat Ghatode ◽  
...  
Keyword(s):  
Emotion Recognition ◽  
Automobile Industry ◽  
Emotional State ◽  
Recognition System ◽  
Classification Model ◽  
General Idea ◽  
Speech Emotion Recognition ◽  
Support Vector ◽  
Emotional Speech ◽  
Acoustic Features

This project describes "VoiEmo- A Speech Emotion Recognizer", a system for recognizing the emotional state of an individual from his/her speech. For example, one's speech becomes loud and fast, with a higher and wider range in pitch, when in a state of fear, anger, or joy whereas human voice is generally slow and low pitched in sadness and tiredness. We have particularly developed a classification model speech emotion detection based on Convolutional neural networks (CNNs), Support Vector Machine (SVM), Multilayer Perceptron (MLP) Classification which make predictions considering the acoustic features of speech signal such as Mel Frequency Cepstral Coefficient (MFCC). Our models have been trained to recognize seven common emotions (neutral, calm, happy, sad, angry, fearful, disgust, surprise). For training and testing the model, we have used relevant data from the Ryerson Audio-Visual Database of Emotional Speech and Song (RAVDESS) dataset and the Toronto Emotional Speech Set (TESS) Dataset. The system is advantageous as it can provide a general idea about the emotional state of the individual based on the acoustic features of the speech irrespective of the language the speaker speaks in, moreover, it also saves time and effort. Speech emotion recognition systems have their applications in various fields like in call centers and BPOs, criminal investigation, psychiatric therapy, the automobile industry, etc.

scholarly journals Gender Classification for Emotional Speech using GMFCC and Deep LSTM

2019 ◽  
Vol 9 (2) ◽  
pp. 3923-3928
Keyword(s):  
Emotion Recognition ◽  
Short Term Memory ◽  
Identification Accuracy ◽  
Speech Emotion Recognition ◽  
Gender Classification ◽  
Feature Extraction Method ◽  
Long Short Term Memory ◽  
Pass Through ◽  
Mel Frequency Cepstral Coefficient

We have come to the point that one of the important aspects of the process speech emotion recognition is the gender classification. The correct classification of gender will improve the performance of Speech Emotion Recognition (SER) system towards its robustness. Here, we are specifically referring to Gammatone Mel Frequency Cepstral Coefficient (GMFCC) as a feature extraction method that extracts features from IITKGPSESHC dataset, which is very crucial in deciding either male or female in gender classification. The well known classifier “Deep Long Short Term Memory (Deep LSTM)” is itself an important kind of Recurrent Neural Network (RNN) that handles the longrange dependencies more efficiently than the RNNs. The GMFCC feature has to pass through the Deep LSTM and get average percent gender identification accuracy of 98.3%.

scholarly journals RNN-based Dimensional Speech Emotion Recognition

2020 ◽  
Author(s):  
Bagus Tris Atmaja
Keyword(s):  
Emotion Recognition ◽  
Short Term Memory ◽  
Mean Squared Error ◽  
Absolute Error ◽  
Recognition System ◽  
Speech Emotion Recognition ◽  
Percentage Error ◽  
Concordance Correlation ◽  
Acoustic Feature ◽  
Dense System

◆ A speech emotion recognition system based on recurrent neural networks is developed using long short-term memory networks.◆ Two of acoustic feature sets are evaluated: 31 Features (3 time-domain features, 5 frequency-domain features, 13 MFCCs, 5 F0s, and 5 Harmonics) and eGeMaps feature set (23 features).◆ To evaluate the performance, some metrics are used i.e. mean squared error (MSE), mean absolute percentage error (MAPE), mean absolute error (MAE) and concordance correlation coefficient (CCC). Among those metrics, CCC is main focus as it is used by other researchers.◆ The developed system used multi-task learning to maximize arousal, valence, and dominance at the same time using CCC loss (1 - CCC). The result shows using LSTM networks improve the CCC score compared to baseline dense system. The best CCC score isobtained on arousal followed by dominance and valence.

Sign in / Sign up

Export Citation Format

Share Document