A Novel Spatio-Temporal Field for Emotion Recognition Based on EEG Signals

In recent years, more and more researchers have focused on emotion recognition methods based on electroencephalogram (EEG) signals. However, most studies only consider the spatio-temporal characteristics of EEG and the modelling based on this feature, without considering personality factors, let alone studying the potential correlation between different subjects. Considering the particularity of emotions, different individuals may have different subjective responses to the same physical stimulus. Therefore, emotion recognition methods based on EEG signals should tend to be personalized. This paper models the personalized EEG emotion recognition from the macro and micro levels. At the macro level, we use personality characteristics to classify the individuals’ personalities from the perspective of ‘birds of a feather flock together’. At the micro level, we employ deep learning models to extract the spatio-temporal feature information of EEG. To evaluate the effectiveness of our method, we conduct an EEG emotion recognition experiment on the ASCERTAIN dataset. Our experimental results demonstrate that the recognition accuracy of our proposed method is 72.4% and 75.9% on valence and arousal, respectively, which is 10.2% and 9.1% higher than that of no consideration of personalization.

Download Full-text

Spatio-Temporal Representation of an Electoencephalogram for Emotion Recognition Using a Three-Dimensional Convolutional Neural Network

Sensors ◽

10.3390/s20123491 ◽

2020 ◽

Vol 20 (12) ◽

pp. 3491 ◽

Cited By ~ 2

Author(s):

Jungchan Cho ◽

Hyoseok Hwang

Keyword(s):

Neural Network ◽

Emotion Recognition ◽

Three Dimensional ◽

Series Data ◽

Electrode Position ◽

Eeg Signals ◽

Remarkable Feature ◽

Efficient Representation ◽

Spatio Temporal

Emotion recognition plays an important role in the field of human–computer interaction (HCI). An electroencephalogram (EEG) is widely used to estimate human emotion owing to its convenience and mobility. Deep neural network (DNN) approaches using an EEG for emotion recognition have recently shown remarkable improvement in terms of their recognition accuracy. However, most studies in this field still require a separate process for extracting handcrafted features despite the ability of a DNN to extract meaningful features by itself. In this paper, we propose a novel method for recognizing an emotion based on the use of three-dimensional convolutional neural networks (3D CNNs), with an efficient representation of the spatio-temporal representations of EEG signals. First, we spatially reconstruct raw EEG signals represented as stacks of one-dimensional (1D) time series data to two-dimensional (2D) EEG frames according to the original electrode position. We then represent a 3D EEG stream by concatenating the 2D EEG frames to the time axis. These 3D reconstructions of the raw EEG signals can be efficiently combined with 3D CNNs, which have shown a remarkable feature representation from spatio-temporal data. Herein, we demonstrate the accuracy of the emotional classification of the proposed method through extensive experiments on the DEAP (a Dataset for Emotion Analysis using EEG, Physiological, and video signals) dataset. Experimental results show that the proposed method achieves a classification accuracy of 99.11%, 99.74%, and 99.73% in the binary classification of valence and arousal, and, in four-class classification, respectively. We investigate the spatio-temporal effectiveness of the proposed method by comparing it to several types of input methods with 2D/3D CNN. We then verify the best performing shape of both the kernel and input data experimentally. We verify that an efficient representation of an EEG and a network that fully takes advantage of the data characteristics can outperform methods that apply handcrafted features.

Download Full-text

Stochastic simulation of the spatio-temporal field of the average daily heat index in Southern Russia

Climate Research ◽

10.3354/cr01623 ◽

2020 ◽

Vol 82 ◽

pp. 149-160

Author(s):

N Kargapolova

Keyword(s):

Heat Waves ◽

Numerical Models ◽

Heat Index ◽

Real Field ◽

Gaussian Field ◽

The Real ◽

Southern Russia ◽

Weather Stations ◽

Spatio Temporal ◽

Temporal Field

Numerical models of the heat index time series and spatio-temporal fields can be used for a variety of purposes, from the study of the dynamics of heat waves to projections of the influence of future climate on humans. To conduct these studies one must have efficient numerical models that successfully reproduce key features of the real weather processes. In this study, 2 numerical stochastic models of the spatio-temporal non-Gaussian field of the average daily heat index (ADHI) are considered. The field is simulated on an irregular grid determined by the location of weather stations. The first model is based on the method of the inverse distribution function. The second model is constructed using the normalization method. Real data collected at weather stations located in southern Russia are used to both determine the input parameters and to verify the proposed models. It is shown that the first model reproduces the properties of the real field of the ADHI more precisely compared to the second one, but the numerical implementation of the first model is significantly more time consuming. In the future, it is intended to transform the models presented to a numerical model of the conditional spatio-temporal field of the ADHI defined on a dense spatio-temporal grid and to use the model constructed for the stochastic forecasting of the heat index.

Download Full-text

Continuous emotion recognition during music listening using EEG signals: A fuzzy parallel cascades model

Applied Soft Computing ◽

10.1016/j.asoc.2020.107028 ◽

2020 ◽

pp. 107028

Author(s):

Fatemeh Hasanzadeh ◽

Mohsen Annabestani ◽

Sahar Moghimi

Keyword(s):

Emotion Recognition ◽

Music Listening ◽

Eeg Signals

Download Full-text

Integrated structured light architectures

Scientific Reports ◽

10.1038/s41598-020-80502-y ◽

2021 ◽

Vol 11 (1) ◽

Author(s):

Randy Lemons ◽

Wei Liu ◽

Josef C. Frisch ◽

Alan Fry ◽

Joseph Robinson ◽

...

Keyword(s):

Degrees Of Freedom ◽

Temporal Distribution ◽

Multiple Degrees Of Freedom ◽

Beam Combination ◽

Angular Momenta ◽

Field Control ◽

Topological States ◽

Spatio Temporal ◽

Temporal Field ◽

Structural Versatility

AbstractThe structural versatility of light underpins an outstanding collection of optical phenomena where both geometrical and topological states of light can dictate how matter will respond or display. Light possesses multiple degrees of freedom such as amplitude, and linear, spin angular, and orbital angular momenta, but the ability to adaptively engineer the spatio-temporal distribution of all these characteristics is primarily curtailed by technologies used to impose any desired structure to light. We demonstrate a laser architecture based on coherent beam combination offering integrated spatio-temporal field control and programmability, thereby presenting unique opportunities for generating light by design to exploit its topology.

Download Full-text

Multi-modal Fusion Using Spatio-temporal and Static Features for Group Emotion Recognition

Proceedings of the 2020 International Conference on Multimodal Interaction ◽

10.1145/3382507.3417971 ◽

2020 ◽

Author(s):

Mo Sun ◽

Jian Li ◽

Hui Feng ◽

Wei Gou ◽

Haifeng Shen ◽

...

Keyword(s):

Emotion Recognition ◽

Spatio Temporal ◽

Group Emotion

Download Full-text

Affective Computing on Machine Learning-Based Emotion Recognition Using a Self-Made EEG Device

Sensors ◽

10.3390/s21155135 ◽

2021 ◽

Vol 21 (15) ◽

pp. 5135

Author(s):

Ngoc-Dau Mai ◽

Boon-Giin Lee ◽

Wan-Young Chung

Keyword(s):

Machine Learning ◽

Emotion Recognition ◽

Temporal Lobe ◽

Affective Computing ◽

Electrode Placement ◽

Entropy Measure ◽

Support Vector ◽

Emotion Classification ◽

Eeg Signals ◽

Computing Method

In this research, we develop an affective computing method based on machine learning for emotion recognition using a wireless protocol and a wearable electroencephalography (EEG) custom-designed device. The system collects EEG signals using an eight-electrode placement on the scalp; two of these electrodes were placed in the frontal lobe, and the other six electrodes were placed in the temporal lobe. We performed experiments on eight subjects while they watched emotive videos. Six entropy measures were employed for extracting suitable features from the EEG signals. Next, we evaluated our proposed models using three popular classifiers: a support vector machine (SVM), multi-layer perceptron (MLP), and one-dimensional convolutional neural network (1D-CNN) for emotion classification; both subject-dependent and subject-independent strategies were used. Our experiment results showed that the highest average accuracies achieved in the subject-dependent and subject-independent cases were 85.81% and 78.52%, respectively; these accuracies were achieved using a combination of the sample entropy measure and 1D-CNN. Moreover, our study investigates the T8 position (above the right ear) in the temporal lobe as the most critical channel among the proposed measurement positions for emotion classification through electrode selection. Our results prove the feasibility and efficiency of our proposed EEG-based affective computing method for emotion recognition in real-world applications.

Download Full-text

Emotion recognition using time–frequency ridges of EEG signals based on multivariate synchrosqueezing transform

Biomedical Engineering / Biomedizinische Technik ◽

10.1515/bmt-2020-0295 ◽

2021 ◽

Vol 0 (0) ◽

Author(s):

Ahmet Mert ◽

Hasan Huseyin Celik

Keyword(s):

Emotion Recognition ◽

Feature Vector ◽

Dimensional Space ◽

Computational Cost ◽

Feature Space ◽

Maximum Energy ◽

Machine Learning Algorithms ◽

Eeg Signals ◽

Time Frequency ◽

Synchrosqueezing Transform

Abstract The feasibility of using time–frequency (TF) ridges estimation is investigated on multi-channel electroencephalogram (EEG) signals for emotional recognition. Without decreasing accuracy rate of the valence/arousal recognition, the informative component extraction with low computational cost will be examined using multivariate ridge estimation. The advanced TF representation technique called multivariate synchrosqueezing transform (MSST) is used to obtain well-localized components of multi-channel EEG signals. Maximum-energy components in the 2D TF distribution are determined using TF-ridges estimation to extract instantaneous frequency and instantaneous amplitude, respectively. The statistical values of the estimated ridges are used as a feature vector to the inputs of machine learning algorithms. Thus, component information in multi-channel EEG signals can be captured and compressed into low dimensional space for emotion recognition. Mean and variance values of the five maximum-energy ridges in the MSST based TF distribution are adopted as feature vector. Properties of five TF-ridges in frequency and energy plane (e.g., mean frequency, frequency deviation, mean energy, and energy deviation over time) are computed to obtain 20-dimensional feature space. The proposed method is performed on the DEAP emotional EEG recordings for benchmarking, and the recognition rates are yielded up to 71.55, and 70.02% for high/low arousal, and high/low valence, respectively.

Download Full-text