Time-Frequency Deep Representation Learning for Speech Emotion Recognition Integrating Self-attention

2019 ◽  
pp. 681-689
Author(s):  
Jiaxing Liu ◽  
Zhilei Liu ◽  
Longbiao Wang ◽  
Lili Guo ◽  
Jianwu Dang
Keyword(s):  
Emotion Recognition ◽  
Representation Learning ◽  
Speech Emotion Recognition ◽  
Time Frequency

Adaptive Domain-Aware Representation Learning for Speech Emotion Recognition

2020 ◽  
Author(s):  
Weiquan Fan ◽  
Xiangmin Xu ◽  
Xiaofen Xing ◽  
Dongyan Huang
Keyword(s):  
Emotion Recognition ◽  
Representation Learning ◽  
Speech Emotion Recognition

scholarly journals Survey of Deep Representation Learning for Speech Emotion Recognition

2021 ◽  
Author(s):  
Siddique Latif ◽  
Rajib Rana ◽  
Sara Khalifa ◽  
Raja Jurdak ◽  
Junaid Qadir ◽  
...  
Keyword(s):  
Emotion Recognition ◽  
General Setting ◽  
Representation Learning ◽  
Data Driven ◽  
Speech Emotion Recognition ◽  
Feature Engineering ◽  
Acoustic Features ◽  
Learning Techniques ◽  
Comprehensive Survey ◽  
Hierarchical Representations

<div>Traditionally, speech emotion recognition (SER) research has relied on manually handcrafted acoustic features using feature engineering. However, the design of handcrafted features for complex SER tasks requires significant manual effort, which impedes generalisability and slows the pace of innovation. This has motivated the adoption of representation learning techniques that can automatically learn an intermediate representation of the input signal without any manual feature engineering. Representation learning has led to improved SER performance and enabled rapid innovation. Its effectiveness has further increased with advances in deep learning (DL), which has facilitated deep representation learning where hierarchical representations are automatically learned in a data-driven manner. This paper presents the first comprehensive survey on the important topic of deep representation learning for SER. We highlight various techniques, related challenges and identify important future areas of research. Our survey bridges the gap in the literature since existing surveys either focus on SER with hand-engineered features or representation learning in the general setting without focusing on SER.</div>

scholarly journals Deep Multimodal Emotion Recognition on Human Speech: A Review

2021 ◽  
Vol 11 (17) ◽  
pp. 7962
Author(s):  
Panagiotis Koromilas ◽  
Theodoros Giannakopoulos
Keyword(s):  
Emotion Recognition ◽  
Visual Information ◽  
Multimodal Interaction ◽  
Representation Learning ◽  
Basic Feature ◽  
Feature Representation ◽  
Speech Emotion Recognition ◽  
Temporal Dimension ◽  
Multimodal Interactions ◽  
Depth Analysis

This work reviews the state of the art in multimodal speech emotion recognition methodologies, focusing on audio, text and visual information. We provide a new, descriptive categorization of methods, based on the way they handle the inter-modality and intra-modality dynamics in the temporal dimension: (i) non-temporal architectures (NTA), which do not significantly model the temporal dimension in both unimodal and multimodal interaction; (ii) pseudo-temporal architectures (PTA), which also assume an oversimplification of the temporal dimension, although in one of the unimodal or multimodal interactions; and (iii) temporal architectures (TA), which try to capture both unimodal and cross-modal temporal dependencies. In addition, we review the basic feature representation methods for each modality, and we present aggregated evaluation results on the reported methodologies. Finally, we conclude this work with an in-depth analysis of the future challenges related to validation procedures, representation learning and method robustness.

scholarly journals Survey of Deep Representation Learning for Speech Emotion Recognition

2021 ◽  
Author(s):  
Siddique Latif ◽  
Rajib Rana ◽  
Sara Khalifa ◽  
Raja Jurdak ◽  
Junaid Qadir ◽  
...  
Keyword(s):  
Emotion Recognition ◽  
General Setting ◽  
Representation Learning ◽  
Data Driven ◽  
Speech Emotion Recognition ◽  
Feature Engineering ◽  
Acoustic Features ◽  
Learning Techniques ◽  
Comprehensive Survey ◽  
Hierarchical Representations

<div>Traditionally, speech emotion recognition (SER) research has relied on manually handcrafted acoustic features using feature engineering. However, the design of handcrafted features for complex SER tasks requires significant manual effort, which impedes generalisability and slows the pace of innovation. This has motivated the adoption of representation learning techniques that can automatically learn an intermediate representation of the input signal without any manual feature engineering. Representation learning has led to improved SER performance and enabled rapid innovation. Its effectiveness has further increased with advances in deep learning (DL), which has facilitated deep representation learning where hierarchical representations are automatically learned in a data-driven manner. This paper presents the first comprehensive survey on the important topic of deep representation learning for SER. We highlight various techniques, related challenges and identify important future areas of research. Our survey bridges the gap in the literature since existing surveys either focus on SER with hand-engineered features or representation learning in the general setting without focusing on SER.</div>

scholarly journals Towards Discriminative Representation Learning for Speech Emotion Recognition

2019 ◽  
Author(s):  
Runnan Li ◽  
Zhiyong Wu ◽  
Jia Jia ◽  
Yaohua Bu ◽  
Sheng Zhao ◽  
...  
Keyword(s):  
Emotion Recognition ◽  
Short Term Memory ◽  
Representation Learning ◽  
Speech Emotion Recognition ◽  
User Intention ◽  
Global Context ◽  
Interaction Database ◽  
Benchmark Database ◽  
Realistic Interaction ◽  
Speech Interaction

In intelligent speech interaction, automatic speech emotion recognition (SER) plays an important role in understanding user intention. While sentimental speech has different speaker characteristics but similar acoustic attributes, one vital challenge in SER is how to learn robust and discriminative representations for emotion inferring. In this paper, inspired by human emotion perception, we propose a novel representation learning component (RLC) for SER system, which is constructed with Multi-head Self-attention and Global Context-aware Attention Long Short-Term Memory Recurrent Neutral Network (GCA-LSTM). With the ability of Multi-head Self-attention mechanism in modeling the element-wise correlative dependencies, RLC can exploit the common patterns of sentimental speech features to enhance emotion-salient information importing in representation learning. By employing GCA-LSTM, RLC can selectively focus on emotion-salient factors with the consideration of entire utterance context, and gradually produce discriminative representation for emotion inferring. Experiments on public emotional benchmark database IEMOCAP and a tremendous realistic interaction database demonstrate the outperformance of the proposed SER framework, with 6.6% to 26.7% relative improvement on unweighted accuracy compared to state-of-the-art techniques.

Sign in / Sign up

Export Citation Format

Share Document