scholarly journals Adaptive Data Boosting Technique for Robust Personalized Speech Emotion in Emotionally-Imbalanced Small-Sample Environments

Sensors ◽  
2018 ◽  
Vol 18 (11) ◽  
pp. 3744 ◽  
Author(s):  
Jaehun Bang ◽  
Taeho Hur ◽  
Dohyeong Kim ◽  
Thien Huynh-The ◽  
Jongwon Lee ◽  
...  
Keyword(s):  
Emotion Recognition ◽  
Data Augmentation ◽  
Imbalanced Data ◽  
Sampling Technique ◽  
Cold Start ◽  
Training Model ◽  
Small Sample ◽  
Speech Emotion Recognition ◽  
Target User ◽  
Cold Start Problem

Personalized emotion recognition provides an individual training model for each target user in order to mitigate the accuracy problem when using general training models collected from multiple users. Existing personalized speech emotion recognition research has a cold-start problem that requires a large amount of emotionally-balanced data samples from the target user when creating the personalized training model. Such research is difficult to apply in real environments due to the difficulty of collecting numerous target user speech data with emotionally-balanced label samples. Therefore, we propose the Robust Personalized Emotion Recognition Framework with the Adaptive Data Boosting Algorithm to solve the cold-start problem. The proposed framework incrementally provides a customized training model for the target user by reinforcing the dataset by combining the acquired target user speech with speech from other users, followed by applying SMOTE (Synthetic Minority Over-sampling Technique)-based data augmentation. The proposed method proved to be adaptive across a small number of target user datasets and emotionally-imbalanced data environments through iterative experiments using the IEMOCAP (Interactive Emotional Dyadic Motion Capture) database.

scholarly journals Speech Emotion Recognition Based on Selective Interpolation Synthetic Minority Over-Sampling Technique in Small Sample Environment

Sensors ◽  
2020 ◽  
Vol 20 (8) ◽  
pp. 2297
Author(s):  
Zhen-Tao Liu ◽  
Bao-Han Wu ◽  
Dan-Yun Li ◽  
Peng Xiao ◽  
Jun-Wei Mao
Keyword(s):  
Emotion Recognition ◽  
Feature Selection Method ◽  
Sampling Technique ◽  
Small Sample ◽  
Speech Emotion Recognition ◽  
Gradient Boosting ◽  
Data Imbalance ◽  
The Arts ◽  
The Impact ◽  
Sample Environment

Speech emotion recognition often encounters the problems of data imbalance and redundant features in different application scenarios. Researchers usually design different recognition models for different sample conditions. In this study, a speech emotion recognition model for a small sample environment is proposed. A data imbalance processing method based on selective interpolation synthetic minority over-sampling technique (SISMOTE) is proposed to reduce the impact of sample imbalance on emotion recognition results. In addition, feature selection method based on variance analysis and gradient boosting decision tree (GBDT) is introduced, which can exclude the redundant features that possess poor emotional representation. Results of experiments of speech emotion recognition on three databases (i.e., CASIA, Emo-DB, SAVEE) show that our method obtains average recognition accuracy of 90.28% (CASIA), 75.00% (SAVEE) and 85.82% (Emo-DB) for speaker-dependent speech emotion recognition which is superior to some state-of-the-arts works.

Speech Emotion Recognition Using 2D-CNN with Data Augmentation

2021 ◽  
Author(s):  
Auliya Mujaddidurrahman ◽  
Ferda Ernawan ◽  
Adi Wibowo ◽  
Eko Adi Sarwoko ◽  
Aris Sugiharto ◽  
...  
Keyword(s):  
Emotion Recognition ◽  
Data Augmentation ◽  
Speech Emotion Recognition

Speech Emotion Recognition on Small Sample Learning by Hybrid WGAN-LSTM Networks

2021 ◽  
Author(s):  
Cunwei Sun ◽  
Luping Ji ◽  
Hailing Zhong
Keyword(s):  
Emotion Recognition ◽  
Language Processing ◽  
Short Term Memory ◽  
Small Sample ◽  
New Method ◽  
Small Samples ◽  
Speech Emotion Recognition ◽  
Generative Adversarial Network ◽  
Adversarial Network ◽  
In Series

The speech emotion recognition based on the deep networks on small samples is often a very challenging problem in natural language processing. The massive parameters of a deep network are much difficult to be trained reliably on small-quantity speech samples. Aiming at this problem, we propose a new method through the systematical cooperation of Generative Adversarial Network (GAN) and Long Short Term Memory (LSTM). In this method, it utilizes the adversarial training of GAN’s generator and discriminator on speech spectrogram images to implement sufficient sample augmentation. A six-layer convolution neural network (CNN), followed in series by a two-layer LSTM, is designed to extract features from speech spectrograms. For accelerating the training of networks, the parameters of discriminator are transferred to our feature extractor. By the sample augmentation, a well-trained feature extraction network and an efficient classifier could be achieved. The tests and comparisons on two publicly available datasets, i.e., EMO-DB and IEMOCAP, show that our new method is effective, and it is often superior to some state-of-the-art methods.

scholarly journals Speech Emotion Recognition using Data Augmentation Method by Cycle-Generative Adversarial Networks

2021 ◽  
Author(s):  
Arash Shilandari ◽  
Hossein Marvi ◽  
Hossein Khosravi
Keyword(s):  
Neural Network ◽  
Emotion Recognition ◽  
Speech Processing ◽  
Data Augmentation ◽  
Generative Adversarial Networks ◽  
Speech Emotion Recognition ◽  
Support Vector ◽  
Generative Adversarial Network ◽  
Adversarial Network ◽  
Adversarial Networks

Nowadays, and with the mechanization of life, speech processing has become so crucial for the interaction between humans and machines. Deep neural networks require a database with enough data for training. The more features are extracted from the speech signal, the more samples are needed to train these networks. Adequate training of these networks can be ensured when there is access to sufficient and varied data in each class. If there is not enough data; it is possible to use data augmentation methods to obtain a database with enough samples. One of the obstacles to developing speech emotion recognition systems is the Data sparsity problem in each class for neural network training. The current study has focused on making a cycle generative adversarial network for data augmentation in a system for speech emotion recognition. For each of the five emotions employed, an adversarial generating network is designed to generate data that is very similar to the main data in that class, as well as differentiate the emotions of the other classes. These networks are taught in an adversarial way to produce feature vectors like each class in the space of the main feature, and then they add to the training sets existing in the database to train the classifier network. Instead of using the common cross-entropy error to train generative adversarial networks and to remove the vanishing gradient problem, Wasserstein Divergence has been used to produce high-quality artificial samples. The suggested network has been tested to be applied for speech emotion recognition using EMODB as training, testing, and evaluating sets, and the quality of artificial data evaluated using two Support Vector Machine (SVM) and Deep Neural Network (DNN) classifiers. Moreover, it has been revealed that extracting and reproducing high-level features from acoustic features, speech emotion recognition with separating five primary emotions has been done with acceptable accuracy.

scholarly journals CNN+LSTM Architecture for Speech Emotion Recognition with Data Augmentation

2018 ◽  
Author(s):  
Caroline Etienne ◽  
Guillaume Fidanza ◽  
Andrei Petrovskii ◽  
Laurence Devillers ◽  
Benoit Schmauch
Keyword(s):  
Emotion Recognition ◽  
Data Augmentation ◽  
Speech Emotion Recognition
Sign in / Sign up

Export Citation Format

Share Document