scholarly journals Multi-Source Co-adaptation for EEG-Based Emotion Recognition by Mining Correlation Information

2021 ◽  
Vol 15 ◽  
Author(s):  
Jianwen Tao ◽  
Yufang Dan
Keyword(s):  
Emotion Recognition ◽  
Experimental Evidence ◽  
Domain Adaptation ◽  
Multiple Sources ◽  
Individual Subject ◽  
Correlation Information ◽  
Metric Function ◽  
Eeg Features

Since each individual subject may present completely different encephalogram (EEG) patterns with respect to other subjects, existing subject-independent emotion classifiers trained on data sampled from cross-subjects or cross-dataset generally fail to achieve sound accuracy. In this scenario, the domain adaptation technique could be employed to address this problem, which has recently got extensive attention due to its effectiveness on cross-distribution learning. Focusing on cross-subject or cross-dataset automated emotion recognition with EEG features, we propose in this article a robust multi-source co-adaptation framework by mining diverse correlation information (MACI) among domains and features with l2,1−norm as well as correlation metric regularization. Specifically, by minimizing the statistical and semantic distribution differences between source and target domains, multiple subject-invariant classifiers can be learned together in a joint framework, which can make MACI use relevant knowledge from multiple sources by exploiting the developed correlation metric function. Comprehensive experimental evidence on DEAP and SEED datasets verifies the better performance of MACI in EEG-based emotion recognition.

scholarly journals Multi-Path and Group-Loss-Based Network for Speech Emotion Recognition in Multi-Domain Datasets

Sensors ◽  
2021 ◽  
Vol 21 (5) ◽  
pp. 1579 ◽  
Author(s):  
Kyoung Ju Noh ◽  
Chi Yoon Jeong ◽  
Jiyoun Lim ◽  
Seungeun Chung ◽  
Gague Kim ◽  
...  
Keyword(s):  
Emotion Recognition ◽  
Short Term Memory ◽  
Domain Adaptation ◽  
Classification Model ◽  
Speech Emotion Recognition ◽  
Target Domain ◽  
Model Generalization ◽  
Speech Database ◽  
Emotion Labels ◽  
Temporal Feature

Speech emotion recognition (SER) is a natural method of recognizing individual emotions in everyday life. To distribute SER models to real-world applications, some key challenges must be overcome, such as the lack of datasets tagged with emotion labels and the weak generalization of the SER model for an unseen target domain. This study proposes a multi-path and group-loss-based network (MPGLN) for SER to support multi-domain adaptation. The proposed model includes a bidirectional long short-term memory-based temporal feature generator and a transferred feature extractor from the pre-trained VGG-like audio classification model (VGGish), and it learns simultaneously based on multiple losses according to the association of emotion labels in the discrete and dimensional models. For the evaluation of the MPGLN SER as applied to multi-cultural domain datasets, the Korean Emotional Speech Database (KESD), including KESDy18 and KESDy19, is constructed, and the English-speaking Interactive Emotional Dyadic Motion Capture database (IEMOCAP) is used. The evaluation of multi-domain adaptation and domain generalization showed 3.7% and 3.5% improvements, respectively, of the F1 score when comparing the performance of MPGLN SER with a baseline SER model that uses a temporal feature generator. We show that the MPGLN SER efficiently supports multi-domain adaptation and reinforces model generalization.

scholarly journals Universum Autoencoder-Based Domain Adaptation for Speech Emotion Recognition

2017 ◽  
Vol 24 (4) ◽  
pp. 500-504 ◽  
Author(s):  
Jun Deng ◽  
Xinzhou Xu ◽  
Zixing Zhang ◽  
Sascha Fruhholz ◽  
Bjorn Schuller
Keyword(s):  
Emotion Recognition ◽  
Domain Adaptation ◽  
Speech Emotion Recognition

scholarly journals Acted vs. Improvised: Domain Adaptation for Elicitation Approaches in Audio-Visual Emotion Recognition

2021 ◽  
Author(s):  
Haoqi Li ◽  
Yelin Kim ◽  
Cheng-Hao Kuo ◽  
Shrikanth S. Narayanan
Keyword(s):  
Emotion Recognition ◽  
Domain Adaptation

scholarly journals Multi-Source Distilling Domain Adaptation

2020 ◽  
Vol 34 (07) ◽  
pp. 12975-12983
Author(s):  
Sicheng Zhao ◽  
Guangzhi Wang ◽  
Shanghang Zhang ◽  
Yang Gu ◽  
Yaxian Li ◽  
...  
Keyword(s):  
Domain Adaptation ◽  
Feature Space ◽  
Wasserstein Distance ◽  
Training Data ◽  
Source Distribution ◽  
Single Source ◽  
Multiple Sources ◽  
Target Domain ◽  
Target Feature ◽  
Training Samples

Deep neural networks suffer from performance decay when there is domain shift between the labeled source domain and unlabeled target domain, which motivates the research on domain adaptation (DA). Conventional DA methods usually assume that the labeled data is sampled from a single source distribution. However, in practice, labeled data may be collected from multiple sources, while naive application of the single-source DA algorithms may lead to suboptimal solutions. In this paper, we propose a novel multi-source distilling domain adaptation (MDDA) network, which not only considers the different distances among multiple sources and the target, but also investigates the different similarities of the source samples to the target ones. Specifically, the proposed MDDA includes four stages: (1) pre-train the source classifiers separately using the training data from each source; (2) adversarially map the target into the feature space of each source respectively by minimizing the empirical Wasserstein distance between source and target; (3) select the source training samples that are closer to the target to fine-tune the source classifiers; and (4) classify each encoded target feature by corresponding source classifier, and aggregate different predictions using respective domain weight, which corresponds to the discrepancy between each source and target. Extensive experiments are conducted on public DA benchmarks, and the results demonstrate that the proposed MDDA significantly outperforms the state-of-the-art approaches. Our source code is released at: https://github.com/daoyuan98/MDDA.

Emotion recognition based on 3D fuzzy visual and EEG features in movie clips

2014 ◽  
Vol 144 ◽  
pp. 560-568 ◽  
Author(s):  
Giyoung Lee ◽  
Mingu Kwon ◽  
Swathi Kavuri Sri ◽  
Minho Lee
Keyword(s):  
Emotion Recognition ◽  
Eeg Features

scholarly journals Towards emotion recognition for virtual environments: an evaluation of eeg features on benchmark dataset

2017 ◽  
Vol 21 (6) ◽  
pp. 1003-1013 ◽  
Author(s):  
M. L. R. Menezes ◽  
A. Samara ◽  
L. Galway ◽  
A. Sant’Anna ◽  
A. Verikas ◽  
...  
Keyword(s):  
Emotion Recognition ◽  
Virtual Environments ◽  
Benchmark Dataset ◽  
Eeg Features

scholarly journals Cross-Individual Affective Detection Using EEG Signals with Audio-Visual Embedding

2021 ◽  
Author(s):  
Zhen Liang ◽  
Xihao Zhang ◽  
Rushuang Zhou ◽  
Li Zhang ◽  
Linling Li ◽  
...  
Keyword(s):  
Emotion Recognition ◽  
Temporal Dynamics ◽  
Recognition Performance ◽  
Critical Issue ◽  
Feature Representation ◽  
Fusion Model ◽  
Generative Adversarial Network ◽  
Adversarial Network ◽  
Brain Data ◽  
Eeg Features

How to effectively and efficiently extract valid and reliable features from high-dimensional electroencephalography (EEG), particularly how to fuse the spatial and temporal dynamic brain information into a better feature representation, is a critical issue in brain data analysis. Most current EEG studies work in a task driven manner and explore the valid EEG features with a supervised model, which would be limited by the given labels to a great extent. In this paper, we propose a practical hybrid unsupervised deep convolutional recurrent generative adversarial network based EEG feature characterization and fusion model, which is termed as EEGFuseNet. EEGFuseNet is trained in an unsupervised manner, and deep EEG features covering both spatial and temporal dynamics are automatically characterized. Comparing to the existing features, the characterized deep EEG features could be considered to be more generic and independent of any specific EEG task. The performance of the extracted deep and low-dimensional features by EEGFuseNet is carefully evaluated in an unsupervised emotion recognition application based on three public emotion databases. The results demonstrate the proposed EEGFuseNet is a robust and reliable model, which is easy to train and performs efficiently in the representation and fusion of dynamic EEG features. In particular, EEGFuseNet is established as an optimal unsupervised fusion model with promising cross-subject emotion recognition performance. It proves EEGFuseNet is capable of characterizing and fusing deep features that imply comparative cortical dynamic significance corresponding to the changing of different emotion states, and also demonstrates the possibility of realizing EEG based cross-subject emotion recognition in a pure unsupervised manner.

Sign in / Sign up

Export Citation Format

Share Document