Correction to: Deep learning approaches for speech emotion recognition: state of the art and research challenges

Feature extraction and emotional classification are significant roles in speech emotion recognition. It is hard to extract and select the optimal features, researchers can not be sure what the features should be. With deep learning approaches, features could be extracted by using hierarchical abstraction layers, but it requires high computational resources and a large number of data. In this article, we choose static, differential, and acceleration coefficients of log Mel-spectrogram as inputs for the deep learning model. To avoid performance degradation, we also add a skip connection with dilated convolution network integration. All representatives are fed into a self-attention mechanism with bidirectional recurrent neural networks to learn long term global features and exploit context for each time step. Finally, we investigate contrastive center loss with softmax loss as loss function to improve the accuracy of emotion recognition. For validating robustness and effectiveness, we tested the proposed method on the Emo-DB and ERC2019 datasets. Experimental results show that the performance of the proposed method is strongly comparable with the existing state-of-the-art methods on the Emo-DB and ERC2019 with 88% and 67%, respectively. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium provided the original work is properly cited.

Download Full-text

Deep Learning Techniques for Speech Emotion Recognition, from Databases to Models

Sensors ◽

10.3390/s21041249 ◽

2021 ◽

Vol 21 (4) ◽

pp. 1249

Author(s):

Babak Joze Abbaschian ◽

Daniel Sierra-Sosa ◽

Adel Elmaghraby

Keyword(s):

Deep Learning ◽

Emotion Recognition ◽

Machine Learning Techniques ◽

Speech Emotion Recognition ◽

Learning Approaches ◽

Human Computer Interactions ◽

Learning Techniques ◽

Conventional Machine ◽

Feasible Solutions ◽

Network Approaches

The advancements in neural networks and the on-demand need for accurate and near real-time Speech Emotion Recognition (SER) in human–computer interactions make it mandatory to compare available methods and databases in SER to achieve feasible solutions and a firmer understanding of this open-ended problem. The current study reviews deep learning approaches for SER with available datasets, followed by conventional machine learning techniques for speech emotion recognition. Ultimately, we present a multi-aspect comparison between practical neural network approaches in speech emotion recognition. The goal of this study is to provide a survey of the field of discrete speech emotion recognition.

Download Full-text

Deep Emotion Recognition in Dynamic Data using Facial, Speech and Textual Cues: A Survey

10.36227/techrxiv.15184302.v1 ◽

2021 ◽

Author(s):

Tao Zhang ◽

Zhenhua Tan

Keyword(s):

Deep Learning ◽

Emotion Recognition ◽

State Of The Art ◽

The State ◽

Research Progress ◽

Speech Emotion Recognition ◽

Expression Recognition ◽

Learning Techniques ◽

Textual Cues ◽

Definition Of

With the development of social media and human-computer interaction, video has become one of the most common data formats. As a research hotspot, emotion recognition system is essential to serve people by perceiving people’s emotional state in videos. In recent years, a large number of studies focus on tackling the issue of emotion recognition based on three most common modalities in videos, that is, face, speech and text. The focus of this paper is to sort out the relevant studies of emotion recognition using facial, speech and textual cues due to the lack of review papers concentrating on the three modalities. On the other hand, because of the effective leverage of deep learning techniques to learn latent representation for emotion recognition, this paper focuses on the emotion recognition method based on deep learning techniques. In this paper, we firstly introduce widely accepted emotion models for the purpose of interpreting the definition of emotion. Then we introduce the state-of-the-art for emotion recognition based on unimodality including facial expression recognition, speech emotion recognition and textual emotion recognition. For multimodal emotion recognition, we summarize the feature-level and decision-level fusion methods in detail. In addition, the description of relevant benchmark datasets, the definition of metrics and the performance of the state-of-the-art in recent years are also outlined for the convenience of readers to find out the current research progress. Ultimately, we explore some potential research challenges and opportunities to give researchers reference for the enrichment of emotion recognition-related researches.

Download Full-text

Deep Learning Approaches for Speech Emotion Recognition

Algorithms for Intelligent Systems - Deep Learning-Based Approaches for Sentiment Analysis ◽

10.1007/978-981-15-1216-2_10 ◽

2020 ◽

pp. 259-289

Author(s):

Anjali Bhavan ◽

Mohit Sharma ◽

Mehak Piplani ◽

Pankaj Chauhan ◽

Hitkul ◽

...

Keyword(s):

Deep Learning ◽

Emotion Recognition ◽

Speech Emotion Recognition ◽

Learning Approaches

Download Full-text

Deep Emotion Recognition in Dynamic Data using Facial, Speech and Textual Cues: A Survey

10.36227/techrxiv.15184302 ◽

2021 ◽

Author(s):

Tao Zhang ◽

Zhenhua Tan

Keyword(s):

Deep Learning ◽

Emotion Recognition ◽

State Of The Art ◽

The State ◽

Research Progress ◽

Speech Emotion Recognition ◽

Expression Recognition ◽

Learning Techniques ◽

Textual Cues ◽

Definition Of

With the development of social media and human-computer interaction, video has become one of the most common data formats. As a research hotspot, emotion recognition system is essential to serve people by perceiving people’s emotional state in videos. In recent years, a large number of studies focus on tackling the issue of emotion recognition based on three most common modalities in videos, that is, face, speech and text. The focus of this paper is to sort out the relevant studies of emotion recognition using facial, speech and textual cues due to the lack of review papers concentrating on the three modalities. On the other hand, because of the effective leverage of deep learning techniques to learn latent representation for emotion recognition, this paper focuses on the emotion recognition method based on deep learning techniques. In this paper, we firstly introduce widely accepted emotion models for the purpose of interpreting the definition of emotion. Then we introduce the state-of-the-art for emotion recognition based on unimodality including facial expression recognition, speech emotion recognition and textual emotion recognition. For multimodal emotion recognition, we summarize the feature-level and decision-level fusion methods in detail. In addition, the description of relevant benchmark datasets, the definition of metrics and the performance of the state-of-the-art in recent years are also outlined for the convenience of readers to find out the current research progress. Ultimately, we explore some potential research challenges and opportunities to give researchers reference for the enrichment of emotion recognition-related researches.

Download Full-text

Utterance Level Feature Aggregation with Deep Metric Learning for Speech Emotion Recognition

Sensors ◽

10.3390/s21124233 ◽

2021 ◽

Vol 21 (12) ◽

pp. 4233

Author(s):

Bogdan Mocanu ◽

Ruxandra Tapu ◽

Titus Zaharia

Keyword(s):

Emotion Recognition ◽

Loss Function ◽

State Of The Art ◽

Disease Diagnosis ◽

Data Representation ◽

Speech Emotion Recognition ◽

Audio Features ◽

Global Accuracy ◽

Space Data ◽

Art Techniques

Emotion is a form of high-level paralinguistic information that is intrinsically conveyed by human speech. Automatic speech emotion recognition is an essential challenge for various applications; including mental disease diagnosis; audio surveillance; human behavior understanding; e-learning and human–machine/robot interaction. In this paper, we introduce a novel speech emotion recognition method, based on the Squeeze and Excitation ResNet (SE-ResNet) model and fed with spectrogram inputs. In order to overcome the limitations of the state-of-the-art techniques, which fail in providing a robust feature representation at the utterance level, the CNN architecture is extended with a trainable discriminative GhostVLAD clustering layer that aggregates the audio features into compact, single-utterance vector representation. In addition, an end-to-end neural embedding approach is introduced, based on an emotionally constrained triplet loss function. The loss function integrates the relations between the various emotional patterns and thus improves the latent space data representation. The proposed methodology achieves 83.35% and 64.92% global accuracy rates on the RAVDESS and CREMA-D publicly available datasets, respectively. When compared with the results provided by human observers, the gains in global accuracy scores are superior to 24%. Finally, the objective comparative evaluation with state-of-the-art techniques demonstrates accuracy gains of more than 3%.

Download Full-text