Extracting the Auditory Attention in a Dual-Speaker Scenario From EEG Using a Joint CNN-LSTM Model

Human brain performs remarkably well in segregating a particular speaker from interfering ones in a multispeaker scenario. We can quantitatively evaluate the segregation capability by modeling a relationship between the speech signals present in an auditory scene, and the listener's cortical signals measured using electroencephalography (EEG). This has opened up avenues to integrate neuro-feedback into hearing aids where the device can infer user's attention and enhance the attended speaker. Commonly used algorithms to infer the auditory attention are based on linear systems theory where cues such as speech envelopes are mapped on to the EEG signals. Here, we present a joint convolutional neural network (CNN)—long short-term memory (LSTM) model to infer the auditory attention. Our joint CNN-LSTM model takes the EEG signals and the spectrogram of the multiple speakers as inputs and classifies the attention to one of the speakers. We evaluated the reliability of our network using three different datasets comprising of 61 subjects, where each subject undertook a dual-speaker experiment. The three datasets analyzed corresponded to speech stimuli presented in three different languages namely German, Danish, and Dutch. Using the proposed joint CNN-LSTM model, we obtained a median decoding accuracy of 77.2% at a trial duration of 3 s. Furthermore, we evaluated the amount of sparsity that the model can tolerate by means of magnitude pruning and found a tolerance of up to 50% sparsity without substantial loss of decoding accuracy.

Download Full-text

Classification of Noisy Epileptic EEG Signals Using Fortified Long Short-term Memory Network

Proceedings of the 2020 12th International Conference on Bioinformatics and Biomedical Technology ◽

10.1145/3405758.3405782 ◽

2020 ◽

Author(s):

Jarno Vrolijk ◽

Maryam Alimardani

Keyword(s):

Short Term Memory ◽

Eeg Signals ◽

Short Term ◽

Term Memory ◽

Memory Network ◽

Long Short Term Memory

Download Full-text

A Long Short-Term Memory deep learning network for the prediction of epileptic seizures using EEG signals

Computers in Biology and Medicine ◽

10.1016/j.compbiomed.2018.05.019 ◽

2018 ◽

Vol 99 ◽

pp. 24-37 ◽

Cited By ~ 93

Author(s):

Κostas Μ. Tsiouris ◽

Vasileios C. Pezoulas ◽

Michalis Zervakis ◽

Spiros Konitsiotis ◽

Dimitrios D. Koutsouris ◽

...

Keyword(s):

Deep Learning ◽

Short Term Memory ◽

Epileptic Seizures ◽

Eeg Signals ◽

Short Term ◽

Term Memory ◽

Learning Network ◽

Long Short Term Memory ◽

Deep Learning Network

Download Full-text

Confusion detection using cognitive ability tests

The International FLAIRS Conference Proceedings ◽

10.32473/flairs.v34i1.128474 ◽

2021 ◽

Vol 34 (1) ◽

Author(s):

Caroline Dakoure ◽

Mohamed Sahbi Benlamine ◽

Claude Frasson

Keyword(s):

Cognitive Ability ◽

Short Term Memory ◽

Nearest Neighbors ◽

Support Vector ◽

Eeg Signals ◽

Short Term ◽

K Nearest Neighbors ◽

Cognitive Ability Tests ◽

Ability Tests ◽

Long Short Term Memory

It is of great importance to detect users’ confusion in a variety of situations such as orientation, reasoning, learning, and memorization. Confusion affects our ability to make decisions and can lower our cognitive ability. This study examines whether a confusion recognition model based on EEG features, recorded on cognitive ability tests, can be used to detect three levels (low, medium, high) of confusion. This study also addresses the extraction of additional features relevant to classification. We compare the performance of the K-nearest neighbors (KNN), support vector memory (SVM), and long short-term memory (LSTM) models. Results suggest that confusion can be efficiently recognized with EEG signals (78.6% accuracy in detecting a confused/unconfused state and 68.0% accuracy in predicting the level of confusion). Implications for educational situations are discussed.

Download Full-text

CONVOLUTIONAL LONG-SHORT TERM MEMORY NETWORKS MODEL FOR LONG DURATION EEG SIGNAL CLASSIFICATION

Journal of Mechanics in Medicine and Biology ◽

10.1142/s0219519419400050 ◽

2019 ◽

Vol 19 (01) ◽

pp. 1940005 ◽

Cited By ~ 3

Author(s):

ULAS BARAN BALOGLU ◽

ÖZAL YILDIRIM

Keyword(s):

Deep Learning ◽

Short Term Memory ◽

Learning Algorithms ◽

Eeg Signals ◽

Short Term ◽

Term Memory ◽

Long Duration ◽

Proposed Model ◽

Clinical Scenarios ◽

Long Short Term Memory

Background and objective: Deep learning structures have recently achieved remarkable success in the field of machine learning. Convolutional neural networks (CNN) in image processing and long-short term memory (LSTM) in the time-series analysis are commonly used deep learning algorithms. Healthcare applications of deep learning algorithms provide important contributions for computer-aided diagnosis research. In this study, convolutional long-short term memory (CLSTM) network was used for automatic classification of EEG signals and automatic seizure detection. Methods: A new nine-layer deep network model consisting of convolutional and LSTM layers was designed. The signals processed in the convolutional layers were given as an input to the LSTM network whose outputs were processed in densely connected neural network layers. The EEG data is appropriate for a model having 1-D convolution layers. A bidirectional model was employed in the LSTM layer. Results: Bonn University EEG database with five different datasets was used for experimental studies. In this database, each dataset contains 23.6[Formula: see text]s duration 100 single channel EEG segments which consist of 4097 dimensional samples (173.61[Formula: see text]Hz). Eight two-class and three three-class clinical scenarios were examined. When the experimental results were evaluated, it was seen that the proposed model had high accuracy on both binary and ternary classification tasks. Conclusions: The proposed end-to-end learning structure showed a good performance without using any hand-crafted feature extraction or shallow classifiers to detect the seizures. The model does not require filtering, and also automatically learns to filter the input as well. As a result, the proposed model can process long duration EEG signals without applying segmentation, and can detect epileptic seizures automatically by using the correlation of ictal and interictal signals of raw data.

Download Full-text