Vehicle Trajectory Estimation Based on Fusion of Visual Motion Features and Deep Learning

Driver situation awareness is critical for safety. In this paper, we propose a fast, accurate method for obtaining real-time situation awareness using a single type of sensor: monocular cameras. The system tracks the host vehicle’s trajectory using sparse optical flow and tracks vehicles in the surrounding environment using convolutional neural networks. Optical flow is used to measure the linear and angular velocity of the host vehicle. The convolutional neural networks are used to measure target vehicles’ positions relative to the host vehicle using image-based detections. Finally, the system fuses host and target vehicle trajectories in the world coordinate system using the velocity of the host vehicle and the target vehicles’ relative positions with the aid of an Extended Kalman Filter (EKF). We implement and test our model quantitatively in simulation and qualitatively on real-world test video. The results show that the algorithm is superior to state-of-the-art sequential state estimation methods such as visual SLAM in performing accurate global localization and trajectory estimation for host and target vehicles.

Download Full-text

Weighted Voting of Multi-Stream Convolutional Neural Networks for Video-Based Action Recognition using Optical Flow Rhythms

Journal of Visual Communication and Image Representation ◽

10.1016/j.jvcir.2021.103112 ◽

2021 ◽

pp. 103112

Author(s):

André Souza Brito ◽

Marcelo Bernardes Vieira ◽

Saulo Moraes Villela ◽

Hemerson Tacon ◽

Hugo Lima Chaves ◽

...

Keyword(s):

Neural Networks ◽

Optical Flow ◽

Convolutional Neural Networks ◽

Action Recognition ◽

Weighted Voting

Download Full-text

Optical Flow and Trajectory Estimation Methods

10.1007/978-3-319-44941-8 ◽

2016 ◽

Author(s):

Joel Gibson ◽

Oge Marques

Keyword(s):

Optical Flow ◽

Estimation Methods ◽

Trajectory Estimation

Download Full-text

Classification of formant estimation methods in transformed auditory feedback experiments using convolutional neural networks

Acoustical Science and Technology ◽

10.1250/ast.41.800 ◽

2020 ◽

Vol 41 (5) ◽

pp. 800-803

Author(s):

Fumiaki Taguchi ◽

Sadao Hiroya ◽

Yasufumi Uezu ◽

Takemi Mochida

Keyword(s):

Neural Networks ◽

Convolutional Neural Networks ◽

Auditory Feedback ◽

Estimation Methods

Download Full-text

Statistical post-processing of wind speed forecasts using convolutional neural networks

Monthly Weather Review ◽

10.1175/mwr-d-20-0219.1 ◽

2021 ◽

Author(s):

Simon Veldkamp ◽

Kirien Whan ◽

Sjoerd Dirksen ◽

Maurice Schmeits

Keyword(s):

Neural Networks ◽

Wind Speed ◽

Convolutional Neural Networks ◽

Weather Prediction ◽

Estimation Methods ◽

Post Processing ◽

Skill Scores ◽

Probabilistic Forecasts ◽

Truncated Normal ◽

Nwp Model

AbstractCurrent statistical post-processing methods for probabilistic weather forecasting are not capable of using full spatial patterns from the numerical weather prediction (NWP) model. In this paper we incorporate spatial wind speed information by using convolutional neural networks (CNNs) and obtain probabilistic wind speed forecasts in the Netherlands for 48 hours ahead, based on KNMI’s deterministic Harmonie-Arome NWP model. The probabilistic forecasts from the CNNs are shown to have higher Brier skill scores for medium to higher wind speeds, as well as a better continuous ranked probability score (CRPS) and logarithmic score, than the forecasts from fully connected neural networks and quantile regression forests. As a secondary result, we have compared the CNNs using 3 different density estimation methods (quantized softmax (QS), kernel mixture networks, and fitting a truncated normal distribution), and found the probabilistic forecasts based on the QS method to be best.

Download Full-text

Fusion of Texture and Optical Flow Using Convolutional Neural Networks for Gender Classification in Videos

Communications in Computer and Information Science - Intelligent Technologies and Applications ◽

10.1007/978-3-030-71711-7_19 ◽

2021 ◽

pp. 227-236

Author(s):

Jag Mohan Singh ◽

Raghavendra Ramachandra ◽

Patrick Bours

Keyword(s):

Neural Networks ◽

Optical Flow ◽

Convolutional Neural Networks ◽

Gender Classification

Download Full-text

Motion-Based Representations For Activity Recognition

10.5753/sibgrapi.est.2020.12988 ◽

2020 ◽

Author(s):

Carlos Caetano ◽

Jefersson Alex Dos Santos ◽

William Robson Schwartz

Keyword(s):

Neural Networks ◽

Optical Flow ◽

Convolutional Neural Networks ◽

Activity Recognition ◽

Temporal Dynamics ◽

State Of The Art ◽

Image Representation ◽

Motion Information ◽

Linear Transformations ◽

Skeleton Image

This work addresses the activity recognition problem. We propose two different representations based on motion information for activity recognition. The first representation is a novel temporal stream for two-stream Convolutional Neural Networks (CNNs) that receives as input images computed from the optical flow magnitude and orientation to learn the motion in a better and richer manner. The method applies simple non-linear transformations on the vertical and horizontal components of the optical flow to generate input images for the temporal stream. The second representation is a novel skeleton image representation to be used as input of CNNs. The approach encodes the temporal dynamics by explicitly computing the magnitude and orientation values of the skeleton joints. Experiments carried out on challenging well-known activity recognition datasets (UCF101, NTU RGB+D 60 and NTU RGB+D 120) demonstrate that the proposed representations achieve results in the state of the art, indicating the suitability of our approaches as video representations.

Download Full-text

Segmentation of Lymph Nodes in Ultrasound Images using U-Net Convolutional Neural Networks and Gabor-based Anisotropic Diffusion

10.21203/rs.3.rs-724851/v1 ◽

2021 ◽

Author(s):

Haobo Chen ◽

Yuqun Wang ◽

Jie Shi ◽

Jingyu Xiong ◽

Jianwei Jiang ◽

...

Keyword(s):

Neural Networks ◽

Lymph Node ◽

Lymph Nodes ◽

Convolutional Neural Networks ◽

Anisotropic Diffusion ◽

Speckle Noise ◽

Accurate Method ◽

Ultrasound Images ◽

Morphological Operations ◽

Sensitivity Specificity

Abstract Objective Automated segmentation of lymph nodes (LNs) in ultrasound images is a challenging task mainly due to the presence of speckle noise and echogenic hila. In this paper, we propose a fully automatic and accurate method for LN segmentation in ultrasound. Methods The proposed segmentation method integrates diffusion-based despeckling, U-Net convolutional neural networks and morphological operations. Firstly, we suppress speckle noise and enhance lymph node edges using the Gabor-based anisotropic diffusion (GAD). Secondly, a modified U-Net model is proposed to segment LNs excluding echogenic hila. Finally, morphological operations are adopted to segment entire LNs by filling the regions of echogenic hila.Results A total of 531 lymph nodes from 526 patients were included to evaluate the proposed method. Quantitative metrics of segmentation performance, including the accuracy, sensitivity, specificity, Jaccard similarity and Dice coefficient, reached 0.934, 0.939, 0.937, 0.763 and 0.865, respectively.Conclusion The proposed method automatically and accurately segments LNs in ultrasound, which may assist artificially intelligent diagnosis of lymph node diseases.

Download Full-text