Video‐triggered EEG‐emotion public databases and current methods: A survey

Emotions, formed in the process of perceiving external environment, directly affect human daily life, such as social interaction, work efficiency, physical wellness, and mental health. In recent decades, emotion recognition has become a promising research direction with significant application values. Taking the advantages of electroencephalogram (EEG) signals (i.e., high time resolution) and video‐based external emotion evoking (i.e., rich media information), video‐triggered emotion recognition with EEG signals has been proven as a useful tool to conduct emotion‐related studies in a laboratory environment, which provides constructive technical supports for establishing real‐time emotion interaction systems. In this paper, we will focus on video‐triggered EEG‐based emotion recognition and present a systematical introduction of the current available video‐triggered EEG‐based emotion databases with the corresponding analysis methods. First, current video‐triggered EEG databases for emotion recognition (e.g., DEAP, MAHNOB‐HCI, SEED series databases) will be presented with full details. Then, the commonly used EEG feature extraction, feature selection, and modeling methods in video‐triggered EEG‐based emotion recognition will be systematically summarized and a brief review of current situation about video‐triggered EEG‐based emotion studies will be provided. Finally, the limitations and possible prospects of the existing video‐triggered EEG‐emotion databases will be fully discussed.

Download Full-text

EEG-Based Estimation on the Reduction of Negative Emotions for Illustrated Surgical Images

Sensors ◽

10.3390/s20247103 ◽

2020 ◽

Vol 20 (24) ◽

pp. 7103

Author(s):

Heekyung Yang ◽

Jongdae Han ◽

Kyungha Min

Keyword(s):

Deep Learning ◽

Emotion Recognition ◽

Emotional Reaction ◽

Emotional Reactions ◽

Fatty Tissue ◽

Eeg Signals ◽

Recognition Model ◽

The Difference ◽

Electroencephalogram Eeg ◽

Professional People

Electroencephalogram (EEG) biosignals are widely used to measure human emotional reactions. The recent progress of deep learning-based classification models has improved the accuracy of emotion recognition in EEG signals. We apply a deep learning-based emotion recognition model from EEG biosignals to prove that illustrated surgical images reduce the negative emotional reactions that the photographic surgical images generate. The strong negative emotional reactions caused by surgical images, which show the internal structure of the human body (including blood, flesh, muscle, fatty tissue, and bone) act as an obstacle in explaining the images to patients or communicating with the images with non-professional people. We claim that the negative emotional reactions generated by illustrated surgical images are less severe than those caused by raw surgical images. To demonstrate the difference in emotional reaction, we produce several illustrated surgical images from photographs and measure the emotional reactions they engender using EEG biosignals; a deep learning-based emotion recognition model is applied to extract emotional reactions. Through this experiment, we show that the negative emotional reactions associated with photographic surgical images are much higher than those caused by illustrated versions of identical images. We further execute a self-assessed user survey to prove that the emotions recognized from EEG signals effectively represent user-annotated emotions.

Download Full-text

Matching pursuit algorithm for enhancing EEG signal quality and increasing the accuracy and efficiency of emotion recognition

Biomedical Engineering / Biomedizinische Technik ◽

10.1515/bmt-2019-0327 ◽

2020 ◽

Vol 65 (4) ◽

pp. 393-404

Author(s):

Ali Momennezhad

Keyword(s):

Emotion Recognition ◽

Matching Pursuit ◽

Recognition Algorithm ◽

Eeg Signals ◽

User Friendliness ◽

Signal Collection ◽

Average Accuracy ◽

Matching Pursuit Algorithm ◽

User Friendly ◽

Electroencephalogram Eeg

AbstractIn this paper, we suggest an efficient, accurate and user-friendly brain-computer interface (BCI) system for recognizing and distinguishing different emotion states. For this, we used a multimodal dataset entitled “MAHOB-HCI” which can be freely reached through an email request. This research is based on electroencephalogram (EEG) signals carrying emotions and excludes other physiological features, as it finds EEG signals more reliable to extract deep and true emotions compared to other physiological features. EEG signals comprise low information and signal-to-noise ratios (SNRs); so it is a huge challenge for proposing a robust and dependable emotion recognition algorithm. For this, we utilized a new method, based on the matching pursuit (MP) algorithm, to resolve this imperfection. We applied the MP algorithm for increasing the quality and SNRs of the original signals. In order to have a signal of high quality, we created a new dictionary including 5-scale Gabor atoms with 5000 atoms. For feature extraction, we used a 9-scale wavelet algorithm. A 32-electrode configuration was used for signal collection, but we used just eight electrodes out of that; therefore, our method is highly user-friendly and convenient for users. In order to evaluate the results, we compared our algorithm with other similar works. In average accuracy, the suggested algorithm is superior to the same algorithm without applying MP by 2.8% and in terms of f-score by 0.03. In comparison with corresponding works, the accuracy and f-score of the proposed algorithm are better by 10.15% and 0.1, respectively. So as it is seen, our method has improved past works in terms of accuracy, f-score and user-friendliness despite using just eight electrodes.

Download Full-text

Personality first in emotion: a deep neural network based on electroencephalogram channel attention for cross-subject emotion recognition

Royal Society Open Science ◽

10.1098/rsos.201976 ◽

2021 ◽

Vol 8 (8) ◽

pp. 201976

Author(s):

Zhihang Tian ◽

Dongmin Huang ◽

Sijin Zhou ◽

Zhidan Zhao ◽

Dazhi Jiang

Keyword(s):

Emotion Recognition ◽

Recognition Accuracy ◽

Personality Factors ◽

Eeg Signals ◽

Recognition Experiment ◽

Feature Information ◽

Micro Level ◽

Spatio Temporal ◽

Valence And Arousal ◽

Electroencephalogram Eeg

In recent years, more and more researchers have focused on emotion recognition methods based on electroencephalogram (EEG) signals. However, most studies only consider the spatio-temporal characteristics of EEG and the modelling based on this feature, without considering personality factors, let alone studying the potential correlation between different subjects. Considering the particularity of emotions, different individuals may have different subjective responses to the same physical stimulus. Therefore, emotion recognition methods based on EEG signals should tend to be personalized. This paper models the personalized EEG emotion recognition from the macro and micro levels. At the macro level, we use personality characteristics to classify the individuals’ personalities from the perspective of ‘birds of a feather flock together’. At the micro level, we employ deep learning models to extract the spatio-temporal feature information of EEG. To evaluate the effectiveness of our method, we conduct an EEG emotion recognition experiment on the ASCERTAIN dataset. Our experimental results demonstrate that the recognition accuracy of our proposed method is 72.4% and 75.9% on valence and arousal, respectively, which is 10.2% and 9.1% higher than that of no consideration of personalization.

Download Full-text

Recognition of emotional states using EEG signals based on time-frequency analysis and SVM classifier

International Journal of Electrical and Computer Engineering (IJECE) ◽

10.11591/ijece.v9i2.pp1012-1020 ◽

2019 ◽

Vol 9 (2) ◽

pp. 1012 ◽

Cited By ~ 4

Author(s):

Fabian Parsia George ◽

Istiaque Mannafee Shaikat ◽

Prommy Sultana Ferdawoos Hossain ◽

Mohammad Zavid Parvez ◽

Jia Uddin

Keyword(s):

Emotion Recognition ◽

Age Groups ◽

Svm Classifier ◽

Emotional States ◽

Eeg Signals ◽

Time Frequency ◽

Gender And Age ◽

Box And Whisker Plot ◽

Traditional Approaches ◽

Electroencephalogram Eeg

The recognition of emotions is a vast significance and a high developing field of research in the recent years. The applications of emotion recognition have left an exceptional mark in various fields including education and research. Traditional approaches used facial expressions or voice intonation to detect emotions, however, facial gestures and spoken language can lead to biased and ambiguous results. This is why, researchers have started to use electroencephalogram (EEG) technique which is well defined method for emotion recognition. Some approaches used standard and pre-defined methods of the signal processing area and some worked with either fewer channels or fewer subjects to record EEG signals for their research. This paper proposed an emotion detection method based on time-frequency domain statistical features. Box-and-whisker plot is used to select the optimal features, which are later feed to SVM classifier for training and testing the DEAP dataset, where 32 participants with different gender and age groups are considered. The experimental results show that the proposed method exhibits 92.36% accuracy for our tested dataset. In addition, the proposed method outperforms than the state-of-art methods by exhibiting higher accuracy.

Download Full-text

EEG-Based Emotion Recognition by Convolutional Neural Network with Multi-Scale Kernels

Sensors ◽

10.3390/s21155092 ◽

2021 ◽

Vol 21 (15) ◽

pp. 5092

Author(s):

Tran-Dac-Thinh Phan ◽

Soo-Hyung Kim ◽

Hyung-Jeong Yang ◽

Guee-Sang Lee

Keyword(s):

Emotion Recognition ◽

Binary Classification ◽

Eeg Signals ◽

Frequency Bands ◽

Multi Scale ◽

Eeg Data ◽

Crucial Information ◽

The Time Domain ◽

Electroencephalogram Eeg ◽

Selection Of

Besides facial or gesture-based emotion recognition, Electroencephalogram (EEG) data have been drawing attention thanks to their capability in countering the effect of deceptive external expressions of humans, like faces or speeches. Emotion recognition based on EEG signals heavily relies on the features and their delineation, which requires the selection of feature categories converted from the raw signals and types of expressions that could display the intrinsic properties of an individual signal or a group of them. Moreover, the correlation or interaction among channels and frequency bands also contain crucial information for emotional state prediction, and it is commonly disregarded in conventional approaches. Therefore, in our method, the correlation between 32 channels and frequency bands were put into use to enhance the emotion prediction performance. The extracted features chosen from the time domain were arranged into feature-homogeneous matrices, with their positions following the corresponding electrodes placed on the scalp. Based on this 3D representation of EEG signals, the model must have the ability to learn the local and global patterns that describe the short and long-range relations of EEG channels, along with the embedded features. To deal with this problem, we proposed the 2D CNN with different kernel-size of convolutional layers assembled into a convolution block, combining features that were distributed in small and large regions. Ten-fold cross validation was conducted on the DEAP dataset to prove the effectiveness of our approach. We achieved the average accuracies of 98.27% and 98.36% for arousal and valence binary classification, respectively.

Download Full-text

Human emotion recognition through short time Electroencephalogram (EEG) signals using Fast Fourier Transform (FFT)

2013 IEEE 9th International Colloquium on Signal Processing and its Applications ◽

10.1109/cspa.2013.6530058 ◽

2013 ◽

Cited By ~ 32

Author(s):

M. Murugappan ◽

S. Murugappan

Keyword(s):

Fourier Transform ◽

Fast Fourier Transform ◽

Emotion Recognition ◽

Eeg Signals ◽

Human Emotion ◽

Short Time ◽

Electroencephalogram Eeg

Download Full-text

The challenges of emotion recognition methods based on electroencephalogram signals: a literature review

International Journal of Electrical and Computer Engineering (IJECE) ◽

10.11591/ijece.v12i2.pp1508-1519 ◽

2022 ◽

Vol 12 (2) ◽

pp. 1508

Author(s):

I Made Agus Wirawan ◽

Retantyo Wardoyo ◽

Danang Lelono

Keyword(s):

Emotion Recognition ◽

Eeg Signals ◽

Data Set ◽

Eeg Data ◽

Network Method ◽

Reduction Methods ◽

Research Questions ◽

Participant Characteristics ◽

Diverse Data ◽

Electroencephalogram Eeg

Electroencephalogram (EEG) signals in recognizing emotions have several advantages. Still, the success of this study, however, is strongly influenced by: i) the distribution of the data used, ii) consider of differences in participant characteristics, and iii) consider the characteristics of the EEG signals. In response to these issues, this study will examine three important points that affect the success of emotion recognition packaged in several research questions: i) What factors need to be considered to generate and distribute EEG data?, ii) How can EEG signals be generated with consideration of differences in participant characteristics?, and iii) How do EEG signals with characteristics exist among its features for emotion recognition? The results, therefore, indicate some important challenges to be studied further in EEG signals-based emotion recognition research. These include i) determine robust methods for imbalanced EEG signals data, ii) determine the appropriate smoothing method to eliminate disturbances on the baseline signals, iii) determine the best baseline reduction methods to reduce the differences in the characteristics of the participants on the EEG signals, iv) determine the robust architecture of the capsule network method to overcome the loss of knowledge information and apply it in more diverse data set.

Download Full-text

Electroencephalogram Emotion Recognition Based on 3D Feature Fusion and Convolutional Autoencoder

Frontiers in Computational Neuroscience ◽

10.3389/fncom.2021.743426 ◽

2021 ◽

Vol 15 ◽

Author(s):

Yanling An ◽

Shaohai Hu ◽

Xiaoying Duan ◽

Ling Zhao ◽

Caiyun Xie ◽

...

Keyword(s):

Emotion Recognition ◽

Spatial Information ◽

Feature Fusion ◽

Recognition Accuracy ◽

Recognition Algorithm ◽

Eeg Signals ◽

Key Technologies ◽

Convolutional Autoencoder ◽

Valence And Arousal ◽

Electroencephalogram Eeg

As one of the key technologies of emotion computing, emotion recognition has received great attention. Electroencephalogram (EEG) signals are spontaneous and difficult to camouflage, so they are used for emotion recognition in academic and industrial circles. In order to overcome the disadvantage that traditional machine learning based emotion recognition technology relies too much on a manual feature extraction, we propose an EEG emotion recognition algorithm based on 3D feature fusion and convolutional autoencoder (CAE). First, the differential entropy (DE) features of different frequency bands of EEG signals are fused to construct the 3D features of EEG signals, which retain the spatial information between channels. Then, the constructed 3D features are input into the CAE constructed in this paper for emotion recognition. In this paper, many experiments are carried out on the open DEAP dataset, and the recognition accuracy of valence and arousal dimensions are 89.49 and 90.76%, respectively. Therefore, the proposed method is suitable for emotion recognition tasks.

Download Full-text