scholarly journals Interpretable Cross-Subject EEG-Based Emotion Recognition Using Channel-Wise Features

Sensors ◽  
2020 ◽  
Vol 20 (23) ◽  
pp. 6719
Author(s):  
Longbin Jin ◽  
Eun Yi Kim
Keyword(s):  
Emotion Recognition ◽  
Short Term Memory ◽  
Symmetric Matrix ◽  
Pearson Correlation ◽  
Brain Regions ◽  
Temporal Features ◽  
Eeg Data ◽  
Emotional Model ◽  
Subject Specific ◽  
Valence And Arousal

Electroencephalogram (EEG)-based emotion recognition is receiving significant attention in research on brain-computer interfaces (BCI) and health care. To recognize cross-subject emotion from EEG data accurately, a technique capable of finding an effective representation robust to the subject-specific variability associated with EEG data collection processes is necessary. In this paper, a new method to predict cross-subject emotion using time-series analysis and spatial correlation is proposed. To represent the spatial connectivity between brain regions, a channel-wise feature is proposed, which can effectively handle the correlation between all channels. The channel-wise feature is defined by a symmetric matrix, the elements of which are calculated by the Pearson correlation coefficient between two-pair channels capable of complementarily handling subject-specific variability. The channel-wise features are then fed to two-layer stacked long short-term memory (LSTM), which can extract temporal features and learn an emotional model. Extensive experiments on two publicly available datasets, the Dataset for Emotion Analysis using Physiological Signals (DEAP) and the SJTU (Shanghai Jiao Tong University) Emotion EEG Dataset (SEED), demonstrate the effectiveness of the combined use of channel-wise features and LSTM. Experimental results achieve state-of-the-art classification rates of 98.93% and 99.10% during the two-class classification of valence and arousal in DEAP, respectively, with an accuracy of 99.63% during three-class classification in SEED.

scholarly journals A EEG-based emotion recognition model with rhythm and time characteristics

2019 ◽  
Vol 6 (1) ◽  
Author(s):  
Jianzhuo Yan ◽  
Shangbin Chen ◽  
Sinuo Deng
Keyword(s):  
Emotion Recognition ◽  
Time Scales ◽  
Time Scale ◽  
Short Term Memory ◽  
Recognition Model ◽  
Memory Network ◽  
Long Short Term Memory ◽  
Valence And Arousal ◽  
Memory Characteristics

Abstract As an advanced function of the human brain, emotion has a significant influence on human studies, works, and other aspects of life. Artificial Intelligence has played an important role in recognizing human emotion correctly. EEG-based emotion recognition (ER), one application of Brain Computer Interface (BCI), is becoming more popular in recent years. However, due to the ambiguity of human emotions and the complexity of EEG signals, the EEG-ER system which can recognize emotions with high accuracy is not easy to achieve. Based on the time scale, this paper chooses the recurrent neural network as the breakthrough point of the screening model. According to the rhythmic characteristics and temporal memory characteristics of EEG, this research proposes a Rhythmic Time EEG Emotion Recognition Model (RT-ERM) based on the valence and arousal of Long–Short-Term Memory Network (LSTM). By applying this model, the classification results of different rhythms and time scales are different. The optimal rhythm and time scale of the RT-ERM model are obtained through the results of the classification accuracy of different rhythms and different time scales. Then, the classification of emotional EEG is carried out by the best time scales corresponding to different rhythms. Finally, by comparing with other existing emotional EEG classification methods, it is found that the rhythm and time scale of the model can contribute to the accuracy of RT-ERM.

Analyzing Emotional Oscillatory Brain Network for Valence and Arousal-Based Emotion Recognition Using EEG Data

2019 ◽  
Vol 18 (04) ◽  
pp. 1359-1378
Author(s):  
Jianzhuo Yan ◽  
Hongzhi Kuai ◽  
Jianhui Chen ◽  
Ning Zhong
Keyword(s):  
Emotion Recognition ◽  
Affective Computing ◽  
Affective State ◽  
Feature Selection Method ◽  
Emotional Valence ◽  
Brain Network ◽  
Previous Method ◽  
Emotional States ◽  
Eeg Data ◽  
Valence And Arousal

Emotion recognition is a highly noteworthy and challenging work in both cognitive science and affective computing. Currently, neurobiology studies have revealed the partially synchronous oscillating phenomenon within brain, which needs to be analyzed from oscillatory synchronization. This combination of oscillations and synchronism is worthy of further exploration to achieve inspiring learning of the emotion recognition models. In this paper, we propose a novel approach of valence and arousal-based emotion recognition using EEG data. First, we construct the emotional oscillatory brain network (EOBN) inspired by the partially synchronous oscillating phenomenon for emotional valence and arousal. And then, a coefficient of variation and Welch’s [Formula: see text]-test based feature selection method is used to identify the core pattern (cEOBN) within EOBN for different emotional dimensions. Finally, an emotional recognition model (ERM) is built by combining cEOBN-inspired information obtained in the above process and different classifiers. The proposed approach can combine oscillation and synchronization characteristics of multi-channel EEG signals for recognizing different emotional states under the valence and arousal dimensions. The cEOBN-based inspired information can effectively reduce the dimensionality of the data. The experimental results show that the previous method can be used to detect affective state at a reasonable level of accuracy.

scholarly journals Spatial-Temporal 3D Residual Correlation Network for Urban Traffic Status Prediction

Symmetry ◽  
2021 ◽  
Vol 14 (1) ◽  
pp. 33
Author(s):  
Yin-Xin Bao ◽  
Quan Shi ◽  
Qin-Qin Shen ◽  
Yang Cao
Keyword(s):  
Neural Network ◽  
Convolutional Neural Network ◽  
Short Term Memory ◽  
Pearson Correlation ◽  
Urban Traffic ◽  
Correlation Network ◽  
Convolutional Network ◽  
Residual Correlation ◽  
Temporal Characteristics ◽  
Temporal Features

Accurate traffic status prediction is of great importance to improve the security and reliability of the intelligent transportation system. However, urban traffic status prediction is a very challenging task due to the tight symmetry among the Human–Vehicle–Environment (HVE). The recently proposed spatial–temporal 3D convolutional neural network (ST-3DNet) effectively extracts both spatial and temporal characteristics in HVE, but ignores the essential long-term temporal characteristics and the symmetry of historical data. Therefore, a novel spatial–temporal 3D residual correlation network (ST-3DRCN) is proposed for urban traffic status prediction in this paper. The ST-3DRCN firstly introduces the Pearson correlation coefficient method to extract a high correlation between traffic data. Then, a dynamic spatial feature extraction component is constructed by using 3D convolution combined with residual units to capture dynamic spatial features. After that, based on the idea of long short-term memory (LSTM), a novel architectural unit is proposed to extract dynamic temporal features. Finally, the spatial and temporal features are fused to obtain the final prediction results. Experiments have been performed using two datasets from Chengdu, China (TaxiCD) and California, USA (PEMS-BAY). Taking the root mean square error (RMSE) as the evaluation index, the prediction accuracy of ST-3DRCN on TaxiCD dataset is 21.4%, 21.3%, 11.7%, 10.8%, 4.7%, 3.6% and 2.3% higher than LSTM, convolutional neural network (CNN), 3D-CNN, spatial–temporal residual network (ST-ResNet), spatial–temporal graph convolutional network (ST-GCN), dynamic global-local spatial–temporal network (DGLSTNet), and ST-3DNet, respectively.

scholarly journals The correlation between fractional amplitude of low-frequency fluctuation-based resting-state functional magnetic resonance imaging and facial emotion recognition ability in patients with first-episode schizophrenia

2021 ◽  
Author(s):  
Qijie Kuang ◽  
Yi Liu ◽  
Sumiao Zhou ◽  
Taiyong Bi ◽  
Lin Mi ◽  
...  
Keyword(s):  
Emotion Recognition ◽  
Resting State ◽  
Meta Analysis ◽  
Pearson Correlation ◽  
Low Frequency ◽  
Brain Regions ◽  
Facial Emotion ◽  
First Episode ◽  
The Right ◽  
First Episode Schizophrenia

Abstract Our aim was to analyse the correlation between the fractional amplitude of low-frequency fluctuation (fALFF) and facial emotion recognition (FER) ability in patients with first-episode schizophrenia (FSZ). A total of 28 patients with FSZ and 33 healthy controls (HCs) completed visual search tasks for FER ability. Regions of interest (ROIs) related to facial emotion were obtained from a previous meta-analysis. Pearson correlation analysis was performed to understand the correlation between fALFF and FER ability. Our results indicated that the patients performed worse than the HCs in the accuracy performances of happy FER and fearful FER. The previous meta-analysis results showed that the brain regions related to FER included the bilateral amygdala (AMY)/hippocampus (HIP), right fusiform gyrus (FFG), and right supplementary motor area (SMA). Pearson correlation showed that the fALFF of the right FFG was associated with high-load fearful FER accuracy (r = -0.43, p = 0.022). Multiple regression analysis showed that the fALFF of the right FFG was an independent contributor to fearful FER accuracy. Our study indicates that FER ability is correlated with resting-state intrinsic activity in brain regions related to facial emotion, which may provide a reference for the study of FER in schizophrenia.

scholarly journals Automatic subject-specific spatiotemporal feature selection for subject-independent affective BCI

PLoS ONE ◽  
2021 ◽  
Vol 16 (8) ◽  
pp. e0253383
Author(s):  
Badar Almarri ◽  
Sanguthevar Rajasekaran ◽  
Chun-Hsi Huang
Keyword(s):  
Time Series ◽  
Feature Selection ◽  
Emotion Recognition ◽  
Learning Algorithm ◽  
Feature Space ◽  
Support Vector ◽  
Affective Stimuli ◽  
Subject Specific ◽  
Spatially Distributed ◽  
Valence And Arousal

The dimensionality of the spatially distributed channels and the temporal resolution of electroencephalogram (EEG) based brain-computer interfaces (BCI) undermine emotion recognition models. Thus, prior to modeling such data, as the final stage of the learning pipeline, adequate preprocessing, transforming, and extracting temporal (i.e., time-series signals) and spatial (i.e., electrode channels) features are essential phases to recognize underlying human emotions. Conventionally, inter-subject variations are dealt with by avoiding the sources of variation (e.g., outliers) or turning the problem into a subject-deponent. We address this issue by preserving and learning from individual particularities in response to affective stimuli. This paper investigates and proposes a subject-independent emotion recognition framework that mitigates the subject-to-subject variability in such systems. Using an unsupervised feature selection algorithm, we reduce the feature space that is extracted from time-series signals. For the spatial features, we propose a subject-specific unsupervised learning algorithm that learns from inter-channel co-activation online. We tested this framework on real EEG benchmarks, namely DEAP, MAHNOB-HCI, and DREAMER. We train and test the selection outcomes using nested cross-validation and a support vector machine (SVM). We compared our results with the state-of-the-art subject-independent algorithms. Our results show an enhanced performance by accurately classifying human affection (i.e., based on valence and arousal) by 16%–27% compared to other studies. This work not only outperforms other subject-independent studies reported in the literature but also proposes an online analysis solution to affection recognition.

scholarly journals Hierarchical Spatiotemporal Electroencephalogram Feature Learning and Emotion Recognition With Attention-Based Antagonism Neural Network

2021 ◽  
Vol 15 ◽  
Author(s):  
Pengwei Zhang ◽  
Chongdan Min ◽  
Kangjia Zhang ◽  
Wen Xue ◽  
Jingxia Chen
Keyword(s):  
Neural Network ◽  
Emotion Recognition ◽  
Short Term Memory ◽  
Brain Area ◽  
Brain Regions ◽  
Dynamic Responses ◽  
Classification Model ◽  
Emotion Classification ◽  
Global Brain ◽  
Eeg Features

Inspired by the neuroscience research results that the human brain can produce dynamic responses to different emotions, a new electroencephalogram (EEG)-based human emotion classification model was proposed, named R2G-ST-BiLSTM, which uses a hierarchical neural network model to learn more discriminative spatiotemporal EEG features from local to global brain regions. First, the bidirectional long- and short-term memory (BiLSTM) network is used to obtain the internal spatial relationship of EEG signals on different channels within and between regions of the brain. Considering the different effects of various cerebral regions on emotions, the regional attention mechanism is introduced in the R2G-ST-BiLSTM model to determine the weight of different brain regions, which could enhance or weaken the contribution of each brain area to emotion recognition. Then a hierarchical BiLSTM network is again used to learn the spatiotemporal EEG features from regional to global brain areas, which are then input into an emotion classifier. Especially, we introduce a domain discriminator to work together with the classifier to reduce the domain offset between the training and testing data. Finally, we make experiments on the EEG data of the DEAP and SEED datasets to test and compare the performance of the models. It is proven that our method achieves higher accuracy than those of the state-of-the-art methods. Our method provides a good way to develop affective brain–computer interface applications.

scholarly journals The Study of a Five-Dimensional Emotional Model for Facial Emotion Recognition

2020 ◽  
Vol 2020 ◽  
pp. 1-10
Author(s):  
Hanzhong Zhang ◽  
Jibin Yin ◽  
Xiangliang Zhang
Keyword(s):  
Emotion Recognition ◽  
Pearson Correlation ◽  
General Concept ◽  
Basic Emotions ◽  
Proposed Model ◽  
Emotional Model ◽  
Emotion Model ◽  
Pearson Correlation Analysis ◽  
The Relationship ◽  
Better Than

Based on basic emotion theory and the PAD emotion model that can describe continuous emotion changes, we first propose a more general concept of a five-dimensional emotion model to better meet the needs in the area of emotion recognition. We determined the relationship between its dimensions and basic emotions and used a Pearson correlation analysis, multilayer perceptron, and other methods to compare and verify it with volunteer human identifiers. The results demonstrated that the five-dimensional emotion model was better than human identification in the field of emotion recognition. We also compared it with the PAD emotion model. The results demonstrated that the five-dimensional emotion model performed better. Finally, using the proposed model, we designed a technology prototype of a mood adaptive interface to demonstrate its potential application.

scholarly journals Deep Learning Method for Selecting Effective Models and Feature Groups in Emotion Recognition Using an Asian Multimodal Database

Electronics ◽  
2020 ◽  
Vol 9 (12) ◽  
pp. 1988
Author(s):  
Jun-Ho Maeng ◽  
Dong-Hyun Kang ◽  
Deok-Hwan Kim
Keyword(s):  
Deep Learning ◽  
Emotion Recognition ◽  
Short Term Memory ◽  
Asian Population ◽  
Emotional Awareness ◽  
Model Parameters ◽  
Brain Lateralization ◽  
Time Frequency ◽  
Valence And Arousal ◽  
Effective Models

Emotional awareness is vital for advanced interactions between humans and computer systems. This paper introduces a new multimodal dataset called MERTI-Apps based on Asian physiological signals and proposes a genetic algorithm (GA)—long short-term memory (LSTM) deep learning model to derive the active feature groups for emotion recognition. This study developed an annotation labeling program for observers to tag the emotions of subjects by their arousal and valence during dataset creation. In the learning phase, a GA was used to select effective LSTM model parameters and determine the active feature group from 37 features and 25 brain lateralization features extracted from the electroencephalogram (EEG) time, frequency, and time–frequency domains. The proposed model achieved a root-mean-square error (RMSE) of 0.0156 in terms of the valence regression performance in the MAHNOB-HCI dataset, and RMSE performances of 0.0579 and 0.0287 in terms of valence and arousal regression performance, and 65.7% and 88.3% in terms of valence and arousal accuracy in the in-house MERTI-Apps dataset, which uses Asian-population-specific 12-channel EEG data and adds an additional brain lateralization (BL) feature. The results revealed 91.3% and 94.8% accuracy in the valence and arousal domain in the DEAP dataset owing to the effective model selection of a GA.

scholarly journals Decoding the Neural Signatures of Valence and Arousal From Portable EEG Headset

2021 ◽  
Author(s):  
Nikhil Garg ◽  
Rohit Garg ◽  
Parrivesh NS ◽  
Apoorv Anand ◽  
V.A.S. Abhinav ◽  
...  
Keyword(s):  
Machine Learning ◽  
Feature Extraction ◽  
Feature Selection ◽  
Emotion Recognition ◽  
Machine Learning Techniques ◽  
Emotion Classification ◽  
Learning Techniques ◽  
Eeg Data ◽  
Valence And Arousal ◽  
Mean Square Errors

This paper focuses on classifying emotions on the valence-arousal plane using various feature extraction, feature selection and machine learning techniques. Emotion classification using EEG data and machine learning techniques has been on the rise in the recent past. We evaluate different feature extraction techniques, feature selection techniques and propose the optimal set of features and electrodes for emotion recognition. The images from the OASIS image dataset were used for eliciting the Valence and Arousal emotions, and the EEG data was recorded using the Emotiv Epoc X mobile EEG headset. The analysis is additionally carried out on publicly available datasets: DEAP and DREAMER. We propose a novel feature ranking technique and incremental learning approach to analyze the dependence of performance on the number of participants. Leave one out cross-validation was carried out to identify subject bias in emotion elicitation patterns. The importance of different electrode locations was calculated, which could be used for designing a headset for emotion recognition. Our study achieved root mean square errors of less than 0.75 on DREAMER, 1.76 on DEAP, and 2.39 on our dataset.

scholarly journals CorrNet: Fine-Grained Emotion Recognition for Video Watching Using Wearable Physiological Sensors

Sensors ◽  
2020 ◽  
Vol 21 (1) ◽  
pp. 52
Author(s):  
Tianyi Zhang ◽  
Abdallah El Ali ◽  
Chen Wang ◽  
Alan Hanjalic ◽  
Pablo Cesar
Keyword(s):  
Emotion Recognition ◽  
Electrodermal Activity ◽  
Short Form ◽  
Recognition Performance ◽  
Binary Classification ◽  
Wearable Sensors ◽  
Recognition Algorithm ◽  
Fine Grained ◽  
Physiological Sensors ◽  
Valence And Arousal

Recognizing user emotions while they watch short-form videos anytime and anywhere is essential for facilitating video content customization and personalization. However, most works either classify a single emotion per video stimuli, or are restricted to static, desktop environments. To address this, we propose a correlation-based emotion recognition algorithm (CorrNet) to recognize the valence and arousal (V-A) of each instance (fine-grained segment of signals) using only wearable, physiological signals (e.g., electrodermal activity, heart rate). CorrNet takes advantage of features both inside each instance (intra-modality features) and between different instances for the same video stimuli (correlation-based features). We first test our approach on an indoor-desktop affect dataset (CASE), and thereafter on an outdoor-mobile affect dataset (MERCA) which we collected using a smart wristband and wearable eyetracker. Results show that for subject-independent binary classification (high-low), CorrNet yields promising recognition accuracies: 76.37% and 74.03% for V-A on CASE, and 70.29% and 68.15% for V-A on MERCA. Our findings show: (1) instance segment lengths between 1–4 s result in highest recognition accuracies (2) accuracies between laboratory-grade and wearable sensors are comparable, even under low sampling rates (≤64 Hz) (3) large amounts of neutral V-A labels, an artifact of continuous affect annotation, result in varied recognition performance.

Sign in / Sign up

Export Citation Format

Share Document