A New Spatio-Temporal Saliency-Based Video Object Segmentation

2016 ◽  
Vol 8 (4) ◽  
pp. 629-647 ◽  
Author(s):  
Zhengzheng Tu ◽  
Andrew Abel ◽  
Lei Zhang ◽  
Bin Luo ◽  
Amir Hussain
Keyword(s):  
Object Segmentation ◽  
Video Object Segmentation ◽  
Video Object ◽  
Spatio Temporal

scholarly journals Unsupervised video object segmentation: An affinity and edge learning approach.

2021 ◽  
Author(s):  
Sundaram Muthu ◽  
Ruwan Tennakoon ◽  
Reza Hoseinnezhad ◽  
Alireza Bab-Hadiashar
Keyword(s):  
Optic Flow ◽  
Object Segmentation ◽  
Video Object Segmentation ◽  
Video Object ◽  
Post Processing ◽  
Multiple Objects ◽  
Temporal Features ◽  
Image Edge ◽  
Spatio Temporal ◽  
Generalization Errors

<div>This paper presents a new approach to solve unsupervised video object segmentation~(UVOS) problem (called TMNet). The UVOS is still a challenging problem as prior methods suffer from issues like generalization errors to segment multiple objects in unseen test videos (category agnostic), over reliance on inaccurate optic flow, and problem towards capturing fine details at object boundaries. These issues make the UVOS, particularly in presence of multiple objects, an ill-defined problem. Our focus is to constrain the problem and improve the segmentation results by inclusion of multiple available cues such as appearance, motion, image edge, flow edge and tracking information through neural attention. To solve the challenging category agnostic multiple object UVOS, our model is designed to predict neighbourhood affinities for being part of the same object and cluster those to obtain accurate segmentation. To achieve multi cue based neural attention, we designed a Temporal Motion Attention module, as part of our segmentation framework, to learn the spatio-temporal features. To refine and improve the accuracy of object segmentation boundaries, an edge refinement module (using image and optic flow edges) and a geometry based loss function are incorporated. The overall framework is capable of segmenting and finding accurate objects' boundaries without any heuristic post processing. This enables the method to be used for unseen videos. Experimental results on challenging DAVIS16 and multi object DAVIS17 datasets shows that our proposed TMNet performs favourably compared to the state-of-the-art methods without post processing.</div>

scholarly journals Motion-Attentive Transition for Zero-Shot Video Object Segmentation

2020 ◽  
Vol 34 (07) ◽  
pp. 13066-13073 ◽  
Author(s):  
Tianfei Zhou ◽  
Shunzhou Wang ◽  
Yi Zhou ◽  
Yazhou Yao ◽  
Jianwu Li ◽  
...  
Keyword(s):  
Object Representation ◽  
Object Segmentation ◽  
Object Motion ◽  
Video Object Segmentation ◽  
Video Object ◽  
The Arts ◽  
Temporal Object ◽  
Multi Level ◽  
Transition Network ◽  
Spatio Temporal

In this paper, we present a novel Motion-Attentive Transition Network (MATNet) for zero-shot video object segmentation, which provides a new way of leveraging motion information to reinforce spatio-temporal object representation. An asymmetric attention block, called Motion-Attentive Transition (MAT), is designed within a two-stream encoder, which transforms appearance features into motion-attentive representations at each convolutional stage. In this way, the encoder becomes deeply interleaved, allowing for closely hierarchical interactions between object motion and appearance. This is superior to the typical two-stream architecture, which treats motion and appearance separately in each stream and often suffers from overfitting to appearance information. Additionally, a bridge network is proposed to obtain a compact, discriminative and scale-sensitive representation for multi-level encoder features, which is further fed into a decoder to achieve segmentation results. Extensive experiments on three challenging public benchmarks (i.e., DAVIS-16, FBMS and Youtube-Objects) show that our model achieves compelling performance against the state-of-the-arts. Code is available at: https://github.com/tfzhou/MATNet.

Sign in / Sign up

Export Citation Format

Share Document