A Real-time Driver Fatigue Detection Method Based on Two-Stage Convolutional Neural Network

Audio processing, including speech enhancement system, improves speech intelligibility and quality in real-time communication (RTC) such as online meetings and online education. However, such processing, primarily noise suppression and automatic gain control, is harmful to music quality when the captured signal is music instead of speech. A music detector can solve the issue above by switching off the speech processing when the music is detected. In RTC scenarios, the music detector should be low-complexity and cover various situations, including different types of music, background noises, and other acoustical environments. In this paper, a real-time music detection method with low-computation complexity is proposed, based on a convolutional neural network (CNN) using Mel-spectrogram and spectral flux as input features. The proposed method achieves overall 90.63% accuracy under different music types (classical music, instruments solos, singing-songs, etc.), speech languages (English and Mandarin), and noise types. The proposed method is constructed on a lightweight CNN model with a small feature size, which guarantees real-time processing.

Download Full-text

Structural Damage Detection Based on Real-Time Vibration Signal and Convolutional Neural Network

Applied Sciences ◽

10.3390/app10144720 ◽

2020 ◽

Vol 10 (14) ◽

pp. 4720 ◽

Cited By ~ 1

Author(s):

Zhiqiang Teng ◽

Shuai Teng ◽

Jiqiao Zhang ◽

Gongfa Chen ◽

Fangsen Cui

Keyword(s):

Neural Network ◽

Experimental Data ◽

Convolutional Neural Network ◽

Real Time ◽

Structural State ◽

Detection Method ◽

Steel Frame ◽

Engineering Practice ◽

Detection Accuracy ◽

Vibration Signals

The traditional methods of structural health monitoring (SHM) have obvious disadvantages such as being time-consuming, laborious and non-synchronizing, and so on. This paper presents a novel and efficient approach to detect structural damages from real-time vibration signals via a convolutional neural network (CNN). As vibration signals (acceleration) reflect the structural response to the changes of the structural state, hence, a CNN, as a classifier, can map vibration signals to the structural state and detect structural damages. As it is difficult to obtain enough damage samples in practical engineering, finite element analysis (FEA) provides an alternative solution to this problem. In this paper, training samples for the CNN are obtained using FEA of a steel frame, and the effectiveness of the proposed detection method is evaluated by inputting the experimental data into the CNN. The results indicate that, the detection accuracy of the CNN trained using FEA data reaches 94% for damages introduced in the numerical model and 90% for damages in the real steel frame. It is demonstrated that the CNN has an ideal detection effect for both single damage and multiple damages. The combination of FEA and experimental data provides enough training and testing samples for the CNN, which improves the practicability of the CNN-based detection method in engineering practice.

Download Full-text

Real-Time Retail Smart Space Optimization and Personalized Store Assortment with Two-Stage Object Detection Using Faster Regional Convolutional Neural Network

Lecture Notes in Electrical Engineering - Advances in Computing and Network Communications ◽

10.1007/978-981-33-6987-0_33 ◽

2021 ◽

pp. 397-408

Author(s):

Nitin Vamsi Dantu ◽

Shriram K. Vasudevan

Keyword(s):

Neural Network ◽

Object Detection ◽

Convolutional Neural Network ◽

Real Time ◽

Smart Space ◽

Two Stage

Download Full-text

A New Algorithm Based on Cascaded Convolutional Neural Network for Driver Fatigue Detection

CICTP 2019 ◽

10.1061/9780784482292.036 ◽

2019 ◽

Author(s):

Feng You ◽

Xiao-long Li ◽

Hai-qing Tu

Keyword(s):

Neural Network ◽

Convolutional Neural Network ◽

Driver Fatigue ◽

Fatigue Detection

Download Full-text

Driver fatigue detection based on convolutional neural network and face alignment for edge computing device

Proceedings of the Institution of Mechanical Engineers Part D Journal of Automobile Engineering ◽

10.1177/0954407021999485 ◽

2021 ◽

pp. 095440702199948

Author(s):

Xiaofeng Li ◽

Jiahao Xia ◽

Libo Cao ◽

Guanjun Zhang ◽

Xiexing Feng

Keyword(s):

Neural Network ◽

Convolutional Neural Network ◽

Face Detection ◽

Edge Computing ◽

Face Alignment ◽

Driver Fatigue ◽

Computing Device ◽

The Face ◽

Yaw Angle ◽

Fatigue Detection

Most current vision-based fatigue detection methods don’t have high-performance and robust face detector. They detect driver fatigue using single detection feature and cannot achieve real-time efficiency on edge computing devices. Aimed at solving these problems, this paper proposes a driver fatigue detection system based on convolutional neural network that can run in real-time on edge computing devices. The system firstly uses the proposed face detection network LittleFace to locate the face and classify the face into two states: small yaw angle state “normal” and large yaw angle state “distract.” Secondly, the speed-optimized SDM algorithm is conducted only in the face region of the “normal” state to deal with the problem that the face alignment accuracy decreases at large angle profile, and the “distract” state is used to detect driver distraction. Finally, feature parameters EAR, MAR and head pitch angle are calculated from the obtained landmarks and used to detect driver fatigue respectively. Comprehensive experiments are conducted to evaluate the proposed system and the results show its practicality and superiority. Our face detection network LittleFace can achieve 88.53% mAP on AFLW test set at 58 FPS on the edge computing device Nvidia Jetson Nano. Evaluation results on YawDD, 300 W, and DriverEyes show the average detection accuracy of the proposed system can reach 89.55%.

Download Full-text