S3-Net: A Fast Scene Understanding Network by Single-Shot Segmentation for Autonomous Driving

Real-time segmentation and understanding of driving scenes are crucial in autonomous driving. Traditional pixel-wise approaches extract scene information by segmenting all pixels in a frame, and hence are inefficient and slow. Proposal-wise approaches only learn from the proposed object candidates, but still require multiple steps on the expensive proposal methods. Instead, this work presents a fast single-shot segmentation strategy for video scene understanding. The proposed net, called S3-Net, quickly locates and segments target sub-scenes , and meanwhile extracts attention-aware time-series sub-scene features ( ats-features ) as inputs to an attention-aware spatio-temporal model (ASM) . Utilizing tensorization and quantization techniques, S3-Net is intended to be lightweight for edge computing. Experiments results on CityScapes, UCF11, HMDB51, and MOMENTS datasets demonstrate that the proposed S3-Net achieves an accuracy improvement of 8.1% versus the 3D-CNN based approach on UCF11, a storage reduction of 6.9× and an inference speed of 22.8 FPS on CityScapes with a GTX1080Ti GPU.

Prediction of Flexion and Extension Movements of 4 Fingers of the Hand Using a New Labeled Method

This work presents a neural network classifier for identifying the flexion and extension movements for four fingers from the hand, out of the surface electromyography signals in the forearm muscles. A new labeled data method was proposed based on time segmentation to relate the sEMG signal with the corresponding finger movement. This is a different way of labeling the data for training the neural network, a llowing to reduce the amount of training gesture hand. The experiment consists of 10 sessions in which the fingers are flexed randomly, one at a time for 2 minutes with a 16ms sample time. The absolute mean value (MAV) is used as a feature extractor in the time domain to a verage 5 samples a nd the normalized data is used for the neural network. Results show that this system with the labeled data method, provides a 98.83% precision value for the index finger, 93.46% for the ring finger, 80.34% for the middle finger, and 68.46% for the little finger. The results are simila r to those found in the literature where they classify different gestures using the conventional labeling method.

FASSD-Net Model for Person Semantic Segmentation

This paper proposes the use of the FASSD-Net model for semantic segmentation of human silhouettes, these silhouettes can later be used in various applications that require specific characteristics of human interaction observed in video sequences for the understanding of human activities or for human identification. These applications are classified as high-level task semantic understanding. Since semantic segmentation is presented as one solution for human silhouette extraction, it is concluded that convolutional neural networks (CNN) have a clear advantage over traditional methods for computer vision, based on their ability to learn the representations of appropriate characteristics for the task of segmentation. In this work, the FASSD-Net model is used as a novel proposal that promises real-time segmentation in high-resolution images exceeding 20 FPS. To evaluate the proposed scheme, we use the Cityscapes database, which consists of sundry scenarios that represent human interaction with its environment (these scenarios show the semantic segmentation of people, difficult to solve, that favors the evaluation of our proposal), To adapt the FASSD-Net model to human silhouette semantic segmentation, the indexes of the 19 classes traditionally proposed for Cityscapes were modified, leaving only two labels: One for the class of interest labeled as person and one for the background. The Cityscapes database includes the category “human” composed for “rider” and “person” classes, in which the rider class contains incomplete human silhouettes due to self-occlusions for the activity or transport used. For this reason, we only train the model using the person class rather than human category. The implementation of the FASSD-Net model with only two classes shows promising results in both a qualitative and quantitative manner for the segmentation of human silhouettes.