Using Wearable ECG/PPG Sensors for Driver Drowsiness Detection Based on Distinguishable Pattern of Recurrence Plots

This paper aims to investigate the robust and distinguishable pattern of heart rate variability (HRV) signals, acquired from wearable electrocardiogram (ECG) or photoplethysmogram (PPG) sensors, for driver drowsiness detection. As wearable sensors are so vulnerable to slight movement, they often produce more noise in signals. Thus, from noisy HRV signals, we need to find good traits that differentiate well between drowsy and awake states. To this end, we explored three types of recurrence plots (RPs) generated from the R–R intervals (RRIs) of heartbeats: Bin-RP, Cont-RP, and ReLU-RP. Here Bin-RP is a binary recurrence plot, Cont-RP is a continuous recurrence plot, and ReLU-RP is a thresholded recurrence plot obtained by filtering Cont-RP with a modified rectified linear unit (ReLU) function. By utilizing each of these RPs as input features to a convolutional neural network (CNN), we examined their usefulness for drowsy/awake classification. For experiments, we collected RRIs at drowsy and awake conditions with an ECG sensor of the Polar H7 strap and a PPG sensor of the Microsoft (MS) band 2 in a virtual driving environment. The results showed that ReLU-RP is the most distinct and reliable pattern for drowsiness detection, regardless of sensor types (i.e., ECG or PPG). In particular, the ReLU-RP based CNN models showed their superiority to other conventional models, providing approximately 6–17% better accuracy for ECG and 4–14% for PPG in drowsy/awake classification.

Download Full-text

A compact and interpretable convolutional neural network for cross-subject driver drowsiness detection from single-channel EEG

Methods ◽

10.1016/j.ymeth.2021.04.017 ◽

2021 ◽

Author(s):

Jian Cui ◽

Zirui Lan ◽

Yisi Liu ◽

Ruilin Li ◽

Fan Li ◽

...

Keyword(s):

Neural Network ◽

Convolutional Neural Network ◽

Single Channel ◽

Drowsiness Detection ◽

Driver Drowsiness

Download Full-text

Using long short term memory and convolutional neural networks for driver drowsiness detection

Accident Analysis & Prevention ◽

10.1016/j.aap.2021.106107 ◽

2021 ◽

Vol 156 ◽

pp. 106107

Author(s):

Azhar Quddus ◽

Ali Shahidi Zandi ◽

Laura Prest ◽

Felix J.E. Comeau

Keyword(s):

Neural Networks ◽

Convolutional Neural Networks ◽

Short Term Memory ◽

Short Term ◽

Term Memory ◽

Drowsiness Detection ◽

Driver Drowsiness ◽

Long Short Term Memory

Download Full-text

Driver Drowsiness Estimation Based on Factorized Bilinear Feature Fusion and a Long-Short-Term Recurrent Convolutional Network

Information ◽

10.3390/info12010003 ◽

2020 ◽

Vol 12 (1) ◽

pp. 3

Author(s):

Shuang Chen ◽

Zengcai Wang ◽

Wenxin Chen

Keyword(s):

Short Term Memory ◽

Feature Fusion ◽

Detection Methods ◽

Video Frame ◽

Estimation Model ◽

Short Term ◽

Convolutional Network ◽

Drowsiness Detection ◽

Driver Drowsiness ◽

Time Information

The effective detection of driver drowsiness is an important measure to prevent traffic accidents. Most existing drowsiness detection methods only use a single facial feature to identify fatigue status, ignoring the complex correlation between fatigue features and the time information of fatigue features, and this reduces the recognition accuracy. To solve these problems, we propose a driver sleepiness estimation model based on factorized bilinear feature fusion and a long- short-term recurrent convolutional network to detect driver drowsiness efficiently and accurately. The proposed framework includes three models: fatigue feature extraction, fatigue feature fusion, and driver drowsiness detection. First, we used a convolutional neural network (CNN) to effectively extract the deep representation of eye and mouth-related fatigue features from the face area detected in each video frame. Then, based on the factorized bilinear feature fusion model, we performed a nonlinear fusion of the deep feature representations of the eyes and mouth. Finally, we input a series of fused frame-level features into a long-short-term memory (LSTM) unit to obtain the time information of the features and used the softmax classifier to detect sleepiness. The proposed framework was evaluated with the National Tsing Hua University drowsy driver detection (NTHU-DDD) video dataset. The experimental results showed that this method had better stability and robustness compared with other methods.

Download Full-text