Video frame interpolation via optical flow estimation with image inpainting

2020 ◽  
Vol 35 (12) ◽  
pp. 2087-2102
Author(s):  
Xiaozhang Liu ◽  
Hui Liu ◽  
Yuxiu Lin
Keyword(s):  
Optical Flow ◽  
Image Inpainting ◽  
Video Frame ◽  
Frame Interpolation ◽  
Flow Estimation ◽  
Optical Flow Estimation

scholarly journals Enhanced Real-Time Intermediate Flow Estimation for Video Frame Interpolation

2021 ◽  
Author(s):  
Minseop Kim ◽  
Haechul Choi
Keyword(s):  
Optical Flow ◽  
Real Time ◽  
Estimation Method ◽  
Frame Rate ◽  
Video Frame ◽  
High Quality ◽  
Frame Interpolation ◽  
Real Time Processing ◽  
High Frame Rate ◽  
Flow Estimation

Recently, the demand for high-quality video content has rapidly been increasing, led by the development of network technology and the growth in video streaming platforms. In particular, displays with a high refresh rate, such as 120 Hz, have become popular. However, the visual quality is only enhanced if the video stream is produced at the same high frame rate. For the high quality, conventional videos with a low frame rate should be converted into a high frame rate in real time. This paper introduces a bidirectional intermediate flow estimation method for real-time video frame interpolation. A bidirectional intermediate optical flow is directly estimated to predict an accurate intermediate frame. For real-time processing, multiple frames are interpolated with a single intermediate optical flow and parts of the network are implemented in 16-bit floating-point precision. Perceptual loss is also applied to improve the cognitive performance of the interpolated frames. The experimental results showed a high prediction accuracy of 35.54 dB on the Vimeo90K triplet benchmark dataset. The interpolation speed of 84 fps was achieved for 480p resolution.

scholarly journals Channel Attention Is All You Need for Video Frame Interpolation

2020 ◽  
Vol 34 (07) ◽  
pp. 10663-10671 ◽  
Author(s):  
Myungsub Choi ◽  
Heewon Kim ◽  
Bohyung Han ◽  
Ning Xu ◽  
Kyoung Mu Lee
Keyword(s):  
Optical Flow ◽  
Comprehensive Evaluation ◽  
Computational Cost ◽  
Main Idea ◽  
Model Complexity ◽  
Video Frame ◽  
Frame Interpolation ◽  
Flow Estimation ◽  
Flow Computation ◽  
Optical Flow Computation

Prevailing video frame interpolation techniques rely heavily on optical flow estimation and require additional model complexity and computational cost; it is also susceptible to error propagation in challenging scenarios with large motion and heavy occlusion. To alleviate the limitation, we propose a simple but effective deep neural network for video frame interpolation, which is end-to-end trainable and is free from a motion estimation network component. Our algorithm employs a special feature reshaping operation, referred to as PixelShuffle, with a channel attention, which replaces the optical flow computation module. The main idea behind the design is to distribute the information in a feature map into multiple channels and extract motion information by attending the channels for pixel-level frame synthesis. The model given by this principle turns out to be effective in the presence of challenging motion and occlusion. We construct a comprehensive evaluation benchmark and demonstrate that the proposed approach achieves outstanding performance compared to the existing models with a component for optical flow computation.

An accurate optical flow estimation of PIV using fluid velocity decomposition

2021 ◽  
Vol 62 (4) ◽  
Author(s):  
Jin Lu ◽  
Hua Yang ◽  
Qinghu Zhang ◽  
Zhouping Yin
Keyword(s):  
Optical Flow ◽  
Fluid Velocity ◽  
Flow Estimation ◽  
Optical Flow Estimation ◽  
Velocity Decomposition

scholarly journals Every Frame Counts: Joint Learning of Video Segmentation and Optical Flow

2020 ◽  
Vol 34 (07) ◽  
pp. 10713-10720
Author(s):  
Mingyu Ding ◽  
Zhe Wang ◽  
Bolei Zhou ◽  
Jianping Shi ◽  
Zhiwu Lu ◽  
...  
Keyword(s):  
Optical Flow ◽  
Video Segmentation ◽  
Video Clip ◽  
Semantic Segmentation ◽  
Temporal Consistency ◽  
Flow Estimation ◽  
Optical Flow Estimation ◽  
Optical Flows ◽  
Benchmark Datasets ◽  
Spatio Temporal

A major challenge for video semantic segmentation is the lack of labeled data. In most benchmark datasets, only one frame of a video clip is annotated, which makes most supervised methods fail to utilize information from the rest of the frames. To exploit the spatio-temporal information in videos, many previous works use pre-computed optical flows, which encode the temporal consistency to improve the video segmentation. However, the video segmentation and optical flow estimation are still considered as two separate tasks. In this paper, we propose a novel framework for joint video semantic segmentation and optical flow estimation. Semantic segmentation brings semantic information to handle occlusion for more robust optical flow estimation, while the non-occluded optical flow provides accurate pixel-level temporal correspondences to guarantee the temporal consistency of the segmentation. Moreover, our framework is able to utilize both labeled and unlabeled frames in the video through joint training, while no additional calculation is required in inference. Extensive experiments show that the proposed model makes the video semantic segmentation and optical flow estimation benefit from each other and outperforms existing methods under the same settings in both tasks.

A neural-based method for optical flow estimation using phase correlation

2018 ◽  
Vol 8 (5) ◽  
pp. 526
Author(s):  
Khalid Ghoul ◽  
Mohamed Berkane ◽  
Mohamed Chaouki Batouche
Keyword(s):  
Optical Flow ◽  
Phase Correlation ◽  
Flow Estimation ◽  
Optical Flow Estimation
Sign in / Sign up

Export Citation Format

Share Document