A Motion Adaptive Wavelet-Denoising for Frame-Rate Up-Conversion

2016 ◽  
Vol 850 ◽  
pp. 121-128
Author(s):  
Şükrü Görgülü ◽  
Ömer Nezih Gerek
Keyword(s):  
Optical Flow ◽  
Frame Rate ◽  
Motion Path ◽  
Adaptive Wavelet ◽  
Flow Estimation ◽  
Optical Flow Estimation ◽  
Pixel Location ◽  
Frame Rate Up Conversion ◽  
Temporal Flow ◽  
Zero Tree

This study introduces a frame-rate up-conversion method that uses a temporal wavelet zerotree-based shrinkage algorithm over motion trajectory of a video obtained by optical flow. The method starts by optical flow estimation for predicting initial estimates of inserted frame pixels. Then, the predicted frame pixels are denoised using a specific wavelet-based algorithm, where each pixel location is examined independently through its own temporal motion path. The denoising was performed by shrinking zero-tree footprints to remove temporal oddities. The resulting video was observed to have more fluent temporal flow as compared to optical flow - only interpolation.

Real-Time Optical Flow Estimation Using Multiple Frame-Straddling Intervals

2012 ◽  
Vol 24 (4) ◽  
pp. 686-698 ◽  
Author(s):  
Lei Chen ◽  
◽  
Hua Yang ◽  
Takeshi Takaki ◽  
Idaku Ishii
Keyword(s):  
Optical Flow ◽  
Real Time ◽  
High Speed ◽  
Frame Rate ◽  
Low Speed ◽  
Flow Estimation ◽  
Optical Flow Estimation ◽  
Optical Flows ◽  
Multiple Frame ◽  
Measurable Range

In this paper, we propose a novel method for accurate optical flow estimation in real time for both high-speed and low-speed moving objects based on High-Frame-Rate (HFR) videos. We introduce a multiframe-straddling function to select several pairs of images with different frame intervals from an HFR image sequence even when the estimated optical flow is required to output at standard video rates (NTSC at 30 fps and PAL at 25 fps). The multiframestraddling function can remarkably improve the measurable range of velocities in optical flow estimation without heavy computation by adaptively selecting a small frame interval for high-speed objects and a large frame interval for low-speed objects. On the basis of the relationship between the frame intervals and the accuracies of the optical flows estimated by the Lucas–Kanade method, we devise a method to determine multiple frame intervals in optical flow estimation and select an optimal frame interval from these intervals according to the amplitude of the estimated optical flow. Our method was implemented using software on a high-speed vision platform, IDP Express. The estimated optical flows were accurately outputted at intervals of 40 ms in real time by using three pairs of 512×512 images; these images were selected by frame-straddling a 2000-fps video with intervals of 0.5, 1.5, and 5 ms. Several experiments were performed for high-speed movements to verify that our method can remarkably improve the measurable range of velocities in optical flow estimation, compared to optical flows estimated for 25-fps videos with the Lucas–Kanade method.

scholarly journals Every Frame Counts: Joint Learning of Video Segmentation and Optical Flow

2020 ◽  
Vol 34 (07) ◽  
pp. 10713-10720
Author(s):  
Mingyu Ding ◽  
Zhe Wang ◽  
Bolei Zhou ◽  
Jianping Shi ◽  
Zhiwu Lu ◽  
...  
Keyword(s):  
Optical Flow ◽  
Video Segmentation ◽  
Video Clip ◽  
Semantic Segmentation ◽  
Temporal Consistency ◽  
Flow Estimation ◽  
Optical Flow Estimation ◽  
Optical Flows ◽  
Benchmark Datasets ◽  
Spatio Temporal

A major challenge for video semantic segmentation is the lack of labeled data. In most benchmark datasets, only one frame of a video clip is annotated, which makes most supervised methods fail to utilize information from the rest of the frames. To exploit the spatio-temporal information in videos, many previous works use pre-computed optical flows, which encode the temporal consistency to improve the video segmentation. However, the video segmentation and optical flow estimation are still considered as two separate tasks. In this paper, we propose a novel framework for joint video semantic segmentation and optical flow estimation. Semantic segmentation brings semantic information to handle occlusion for more robust optical flow estimation, while the non-occluded optical flow provides accurate pixel-level temporal correspondences to guarantee the temporal consistency of the segmentation. Moreover, our framework is able to utilize both labeled and unlabeled frames in the video through joint training, while no additional calculation is required in inference. Extensive experiments show that the proposed model makes the video semantic segmentation and optical flow estimation benefit from each other and outperforms existing methods under the same settings in both tasks.

Sign in / Sign up

Export Citation Format

Share Document