Optical Flow Estimation with Deep Learning, a Survey on Recent Advances

2020 ◽  
pp. 257-287 ◽  
Author(s):  
Stefano Savian ◽  
Mehdi Elahi ◽  
Tammam Tillo
Keyword(s):  
Deep Learning ◽  
Optical Flow ◽  
Flow Estimation ◽  
Optical Flow Estimation ◽  
Recent Advances

scholarly journals Deep Optical Flow Estimation Via Multi-Scale Correspondence Structure Learning

2017 ◽  
Author(s):  
Shanshan Zhao ◽  
Xi Li ◽  
Omar El Farouk Bourahla
Keyword(s):  
Deep Learning ◽  
Optical Flow ◽  
Structure Learning ◽  
Feature Space ◽  
Image Correlation ◽  
Flow Estimation ◽  
Optical Flow Estimation ◽  
Multi Scale ◽  
Benchmark Datasets ◽  
End To End

As an important and challenging problem in computer vision, learning based optical flow estimation aims to discover the intrinsic correspondence structure between two adjacent video frames through statistical learning. Therefore, a key issue to solve in this area is how to effectively model the multi-scale correspondence structure properties in an adaptive end-to-end learning fashion. Motivated by this observation, we propose an end-to-end multi-scale correspondence structure learning (MSCSL) approach for optical flow estimation. In principle, the proposed MSCSL approach is capable of effectively capturing the multi-scale inter-image-correlation correspondence structures within a multi-level feature space from deep learning. Moreover, the proposed MSCSL approach builds a spatial Conv-GRU neural network model to adaptively model the intrinsic dependency relationships among these multi-scale correspondence structures. Finally, the above procedures for correspondence structure learning and multi-scale dependency modeling are implemented in a unified end-to-end deep learning framework. Experimental results on several benchmark datasets demonstrate the effectiveness of the proposed approach.

scholarly journals Optical Flow Estimation and Denoising of Video Images Based on Deep Learning Models

IEEE Access ◽  
2020 ◽  
Vol 8 ◽  
pp. 144122-144135
Author(s):  
Ang Li ◽  
Baoyu Zheng ◽  
Lei Li ◽  
Chen Zhang
Keyword(s):  
Deep Learning ◽  
Optical Flow ◽  
Learning Models ◽  
Flow Estimation ◽  
Optical Flow Estimation ◽  
Video Images

scholarly journals Traditional and modern strategies for optical flow: an investigation

2021 ◽  
Vol 3 (3) ◽  
Author(s):  
Syed Tafseer Haider Shah ◽  
Xiang Xuezhi
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Optical Flow ◽  
Convolutional Neural Networks ◽  
Traditional Approach ◽  
Flow Estimation ◽  
Optical Flow Estimation ◽  
Advantages And Disadvantages ◽  
Recent Developments ◽  
Almost All

AbstractOptical Flow Estimation is an essential component for many image processing techniques. This field of research in computer vision has seen an amazing development in recent years. In particular, the introduction of Convolutional Neural Networks for optical flow estimation has shifted the paradigm of research from the classical traditional approach to deep learning side. At present, state of the art techniques for optical flow are based on convolutional neural networks and almost all top performing methods incorporate deep learning architectures in their schemes. This paper presents a brief analysis of optical flow estimation techniques and highlights most recent developments in this field. A comparison of the majority of pertinent traditional and deep learning methodologies has been undertaken resulting the detailed establishment of the respective advantages and disadvantages of the traditional and deep learning categories. An insight is provided into the significant factors that affect the success or failure of the two classes of optical flow estimation. In establishing the foremost existing and inherent challenges with traditional and deep learning schemes, probable solutions have been proposed indeed.

scholarly journals DEEP LEARNING BASED OPTICAL FLOW ESTIMATION FOR CHANGE DETECTION: A CASE STUDY IN INDONESIA EARTHQUAKE

2020 ◽  
Vol V-3-2020 ◽  
pp. 317-322
Author(s):  
H. J. Qiao ◽  
X. Wan ◽  
J. Z. Xu ◽  
S. Y. Li ◽  
P. P. He
Keyword(s):  
Deep Learning ◽  
Change Detection ◽  
Optical Flow ◽  
Natural Disasters ◽  
Natural Disaster ◽  
Vertical Direction ◽  
Mean Value ◽  
Image Sequences ◽  
Flow Estimation ◽  
Optical Flow Estimation

Abstract. Real-time change detection and analysis of natural disasters is of great importance to emergency response and disaster rescue. Recently, a number of video satellites that can record the whole process of natural disasters have been launched. These satellites capture high resolution video image sequences and provide researchers with a large number of image frames, which allows for the implementation of a rapid disaster procedure change detection approach based on deep learning. In this paper, pixel change in image sequences is estimated by optical flow based on FlowNet 2.0 for quick change detection in natural disasters. Experiments are carried out by using image frames from Digital Globe WorldView in Indonesia Earthquake took place on Sept. 28, 2018. In order to test the efficiency of FlowNet 2.0 on natural disaster dataset, 7 state-of-the-art optical flow estimation methods are compared. The experimental results show that FlowNet 2.0 is not only robust to large displacements but small displacements in natural disaster dataset. Two evaluation indicators: Root Mean Square Error (RMSE) and Mean Value are used to record the accuracy. For estimation error of RMSE, FlowNet 2.0 achieves 0.30 and 0.11 pixels in horizontal and vertical direction, respectively. The error in horizontal error is similar to other algorithms but the value in vertical direction is significantly lower than them. And the Mean Value are 1.50 and 0.09 pixels in horizontal and vertical direction, which are most close to the ground truth comparing to other algorithms. Combining the superiority of computing time, the paper proves that only the approach based on FlowNet 2.0 is able to achieve real-time change detection with higher accuracy in the case of natural disasters.

scholarly journals Unsupervised Recurrent All-Pairs Field Transforms for Particle Image Velocimetry

2021 ◽  
Vol 1 (1) ◽  
Author(s):  
Christian Lagemann ◽  
Michael Klaas ◽  
Wolfgang Schröder
Keyword(s):  
Experimental Data ◽  
Particle Image Velocimetry ◽  
Deep Learning ◽  
Optical Flow ◽  
Real World ◽  
Particle Image ◽  
State Of The Art ◽  
Flow Estimation ◽  
Optical Flow Estimation ◽  
Image Velocimetry

Convolutional neural networks have been successfully used in a variety of tasks and recently have been adapted to improve processing steps in Particle-Image Velocimetry (PIV). Recurrent All-Pairs Fields Transforms (RAFT) as an optical flow estimation backbone achieve a new state-of-the-art accuracy on public synthetic PIV datasets, generalize well to unknown real-world experimental data, and allow a significantly higher spatial resolution compared to state-of-the-art PIV algorithms based on cross-correlation methods. However, the huge diversity in dynamic flows and varying particle image conditions require PIV processing schemes to have high generalization capabilities to unseen flow and lighting conditions. If these conditions vary strongly compared to the synthetic training data, the performance of fully supervised learning based PIV tools might degrade. To tackle these issues, our training procedure is augmented by an unsupervised learning paradigm which remedy the need of a general synthetic dataset and theoretically boosts the inference capability of a deep learning model in a way being more relevant to challenging real-world experimental data. Therefore, we propose URAFT-PIV, an unsupervised deep neural network architecture for optical flow estimation in PIV applications and show that our combination of state-of-the-art deep learning pipelines and unsupervised learning achieves a new state-of-the-art accuracy for unsupervised PIV networks while performing similar to supervisedly trained LiteFlowNet based competitors. Furthermore, we show that URAFT-PIV also performs well under more challenging flow field and image conditions such as low particle density and changing light conditions and demonstrate its generalization capability based on an outof-the-box application to real-world experimental data. Our tests also suggest that current state-of-the-art loss functions might be a limiting factor for the performance of unsupervised optical flow estimation.

Sign in / Sign up

Export Citation Format

Share Document