scholarly journals Learning Saliency-Free Model with Generic Features for Weakly-Supervised Semantic Segmentation

2020 ◽  
Vol 34 (07) ◽  
pp. 11717-11724
Author(s):  
Wenfeng Luo ◽  
Meng Yang
Keyword(s):  
Lower Layer ◽  
Semantic Category ◽  
Ground Truth ◽  
Semantic Segmentation ◽  
Image Features ◽  
Segmentation Methods ◽  
Weakly Supervised ◽  
Validation Set ◽  
Class Labels ◽  
Dimension Space

Current weakly-supervised semantic segmentation methods often estimate initial supervision from class activation maps (CAM), which produce sparse discriminative object seeds and rely on image saliency to provide background cues when only class labels are used. To eliminate the demand of extra data for training saliency detector, we propose to discover class pattern inherent in the lower layer convolution features, which are scarcely explored as in previous CAM methods. Specifically, we first project the convolution features into a low-dimension space and then decide on a decision boundary to generate class-agnostic maps for each semantic category that exists in the image. Features from Lower layer are more generic, thus capable of generating proxy ground-truth with more accurate and integral objects. Experiments on the PASCAL VOC 2012 dataset show that the proposed saliency-free method outperforms the previous approaches under the same weakly-supervised setting and achieves superior segmentation results, which are 64.5% on the validation set and 64.6% on the test set concerning mIoU metric.

scholarly journals Weakly Supervised Conditional Random Fields Model for Semantic Segmentation with Image Patches

2020 ◽  
Vol 10 (5) ◽  
pp. 1679
Author(s):  
Xinying Xu ◽  
Yujing Xue ◽  
Xiaoxia Han ◽  
Zhe Zhang ◽  
Jun Xie ◽  
...  
Keyword(s):  
Random Fields ◽  
Conditional Random Fields ◽  
Semantic Category ◽  
Semantic Segmentation ◽  
Potential Energy Function ◽  
Semantic Class ◽  
Image Patches ◽  
Benchmark Datasets ◽  
Weakly Supervised ◽  
Class Labels

Image semantic segmentation (ISS) is used to segment an image into regions with differently labeled semantic category. Most of the existing ISS methods are based on fully supervised learning, which requires pixel-level labeling for training the model. As a result, it is often very time-consuming and labor-intensive, yet still subject to manual errors and subjective inconsistency. To tackle such difficulties, a weakly supervised ISS approach is proposed, in which the challenging problem of label inference from image-level to pixel-level will be particularly addressed, using image patches and conditional random fields (CRF). An improved simple linear iterative cluster (SLIC) algorithm is employed to extract superpixels. for image segmentation. Specifically, it generates various numbers of superpixels according to different images, which can be used to guide the process of image patch extraction based on the image-level labeled information. Based on the extracted image patches, the CRF model is constructed for inferring semantic class labels, which uses the potential energy function to map from the image-level to pixel-level image labels. Finally, patch based CRF (PBCRF) model is used to accomplish the weakly supervised ISS. Experiments conducted on two publicly available benchmark datasets, MSRC and PASCAL VOC 2012, have demonstrated that our proposed algorithm can yield very promising results compared to quite a few state-of-the-art ISS methods, including some deep learning-based models.

scholarly journals JMLNet: Joint Multi-Label Learning Network for Weakly Supervised Semantic Segmentation in Aerial Images

2020 ◽  
Vol 12 (19) ◽  
pp. 3169
Author(s):  
Rongxin Guo ◽  
Xian Sun ◽  
Kaiqiang Chen ◽  
Xiao Zhou ◽  
Zhiyuan Yan ◽  
...  
Keyword(s):  
Common Knowledge ◽  
Ground Truth ◽  
Semantic Segmentation ◽  
Aerial Images ◽  
Combination Strategy ◽  
Research Attention ◽  
Significant Saving ◽  
Learning Network ◽  
Weakly Supervised ◽  
Segmentation Task

Weakly supervised semantic segmentation in aerial images has attracted growing research attention due to the significant saving in annotation cost. Most of the current approaches are based on one specific pseudo label. These methods easily overfit the wrongly labeled pixels from noisy label and limit the performance and generalization of the segmentation model. To tackle these problems, we propose a novel joint multi-label learning network (JMLNet) to help the model learn common knowledge from multiple noisy labels and prevent the model from overfitting one specific label. Our combination strategy of multiple proposals is that we regard them all as ground truth and propose three new multi-label losses to use the multi-label guide segmentation model in the training process. JMLNet also contains two methods to generate high-quality proposals, which further improve the performance of the segmentation task. First we propose a detection-based GradCAM (GradCAMD) to generate segmentation proposals from object detectors. Then we use GradCAMD to adjust the GrabCut algorithm and generate segmentation proposals (GrabCutC). We report the state-of-the-art results on the semantic segmentation task of iSAID and mapping challenge dataset when training with bounding boxes annotations.

scholarly journals Learning Segmentation Masks with the Independence Prior

2019 ◽  
Vol 33 ◽  
pp. 3429-3436 ◽  
Author(s):  
Songmin Dai ◽  
Xiaoqiang Li ◽  
Lu Wang ◽  
Pin Wu ◽  
Weiqin Tong ◽  
...  
Keyword(s):  
Semantic Segmentation ◽  
Generative Adversarial Networks ◽  
Composite Image ◽  
Specific Instance ◽  
Adversarial Networks ◽  
Segmentation Methods ◽  
Multiple Category ◽  
Complete Category ◽  
Weakly Supervised ◽  
Instance Segmentation

An instance with a bad mask might make a composite image that uses it look fake. This encourages us to learn segmentation by generating realistic composite images. To achieve this, we propose a novel framework that exploits a new proposed prior called the independence prior based on Generative Adversarial Networks (GANs). The generator produces an image with multiple category-specific instance providers, a layout module and a composition module. Firstly, each provider independently outputs a category-specific instance image with a soft mask. Then the provided instances’ poses are corrected by the layout module. Lastly, the composition module combines these instances into a final image. Training with adversarial loss and penalty for mask area, each provider learns a mask that is as small as possible but enough to cover a complete category-specific instance. Weakly supervised semantic segmentation methods widely use grouping cues modeling the association between image parts, which are either artificially designed or learned with costly segmentation labels or only modeled on local pairs. Unlike them, our method automatically models the dependence between any parts and learns instance segmentation. We apply our framework in two cases: (1) Foreground segmentation on category-specific images with box-level annotation. (2) Unsupervised learning of instance appearances and masks with only one image of homogeneous object cluster (HOC). We get appealing results in both tasks, which shows the independence prior is useful for instance segmentation and it is possible to unsupervisedly learn instance masks with only one image.

scholarly journals Adversarial Erasing method based on graph neural network

2021 ◽  
Vol 2083 (4) ◽  
pp. 042083
Author(s):  
Shuhan Liu
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Loss Function ◽  
Deep Neural Networks ◽  
Ground Truth ◽  
Semantic Segmentation ◽  
Data Sets ◽  
Recent Developments ◽  
Weakly Supervised ◽  
Network Weight

Abstract Semantic segmentation is a traditional task that requires a large number of pixel-level ground truth label data sets, which is time-consuming and expensive. Recent developments in weakly-supervised settings have shown that reasonable performance can be obtained using only image-level labels. Classification is often used as an agent task to train deep neural networks and extract attention maps from them. The classification task only needs less supervision information to obtain the most discriminative part of the object. For this purpose, we propose a new end-to-end counter-wipe network. Compared with the baseline network, we propose a method to apply the graph neural network to obtain the first CAM. It is proposed to train the joint loss function to avoid the network weight sharing and cause the network to fall into a saddle point. Our experiments on the Pascal VOC2012 dataset show that 64.9% segmentation performance is obtained, which is an improvement of 2.1% compared to our baseline.

scholarly journals WEAKLY SUPERVISED PSEUDO-LABEL ASSISTED LEARNING FOR ALS POINT CLOUD SEMANTIC SEGMENTATION

2021 ◽  
Vol V-2-2021 ◽  
pp. 43-50
Author(s):  
P. Wang ◽  
W. Yao
Keyword(s):  
Point Cloud ◽  
Three Dimensional ◽  
Ground Truth ◽  
Semantic Segmentation ◽  
Training Data ◽  
Data Annotation ◽  
Cloud Data ◽  
Point Cloud Segmentation ◽  
Prediction Probability ◽  
Weakly Supervised

Abstract. Competitive point cloud semantic segmentation results usually rely on a large amount of labeled data. However, data annotation is a time-consuming and labor-intensive task, particularly for three-dimensional point cloud data. Thus, obtaining accurate results with limited ground truth as training data is considerably important. As a simple and effective method, pseudo labels can use information from unlabeled data for training neural networks. In this study, we propose a pseudo-label-assisted point cloud segmentation method with very few sparsely sampled labels that are normally randomly selected for each class. An adaptive thresholding strategy was proposed to generate a pseudo-label based on the prediction probability. Pseudo-label learning is an iterative process, and pseudo labels were updated solely on ground-truth weak labels as the model converged to improve the training efficiency. Experiments using the ISPRS 3D sematic labeling benchmark dataset indicated that our proposed method achieved an equally competitive result compared to that using a full supervision scheme with only up to 2‰ of labeled points from the original training set, with an overall accuracy of 83.7% and an average F1 score of 70.2%.

scholarly journals Reliability Does Matter: An End-to-End Weakly Supervised Semantic Segmentation Approach

2020 ◽  
Vol 34 (07) ◽  
pp. 12765-12772
Author(s):  
Bingfeng Zhang ◽  
Jimin Xiao ◽  
Yunchao Wei ◽  
Mingjie Sun ◽  
Kaizhu Huang
Keyword(s):  
State Of The Art ◽  
Ground Truth ◽  
Semantic Segmentation ◽  
Step Method ◽  
Pascal Voc ◽  
Segmentation Approach ◽  
Level Information ◽  
One Step ◽  
End To End ◽  
Weakly Supervised

Weakly supervised semantic segmentation is a challenging task as it only takes image-level information as supervision for training but produces pixel-level predictions for testing. To address such a challenging task, most recent state-of-the-art approaches propose to adopt two-step solutions, i.e. 1) learn to generate pseudo pixel-level masks, and 2) engage FCNs to train the semantic segmentation networks with the pseudo masks. However, the two-step solutions usually employ many bells and whistles in producing high-quality pseudo masks, making this kind of methods complicated and inelegant. In this work, we harness the image-level labels to produce reliable pixel-level annotations and design a fully end-to-end network to learn to predict segmentation maps. Concretely, we firstly leverage an image classification branch to generate class activation maps for the annotated categories, which are further pruned into confident yet tiny object/background regions. Such reliable regions are then directly served as ground-truth labels for the parallel segmentation branch, where a newly designed dense energy loss function is adopted for optimization. Despite its apparent simplicity, our one-step solution achieves competitive mIoU scores (val: 62.6, test: 62.9) on Pascal VOC compared with those two-step state-of-the-arts. By extending our one-step method to two-step, we get a new state-of-the-art performance on the Pascal VOC (val: 66.3, test: 66.5).

scholarly journals Weakly-Supervised Recommended Traversable Area Segmentation Using Automatically Labeled Images for Autonomous Driving in Pedestrian Environment with No Edges

Sensors ◽  
2021 ◽  
Vol 21 (2) ◽  
pp. 437
Author(s):  
Yuya Onozuka ◽  
Ryosuke Matsumi ◽  
Motoki Shino
Keyword(s):  
Visual Information ◽  
Data Augmentation ◽  
Semantic Segmentation ◽  
Autonomous Driving ◽  
Weighting Method ◽  
Personal Mobility ◽  
Human Understanding ◽  
Autonomous Mobility ◽  
Weakly Supervised ◽  
Traffic Rules

Detection of traversable areas is essential to navigation of autonomous personal mobility systems in unknown pedestrian environments. However, traffic rules may recommend or require driving in specified areas, such as sidewalks, in environments where roadways and sidewalks coexist. Therefore, it is necessary for such autonomous mobility systems to estimate the areas that are mechanically traversable and recommended by traffic rules and to navigate based on this estimation. In this paper, we propose a method for weakly-supervised recommended traversable area segmentation in environments with no edges using automatically labeled images based on paths selected by humans. This approach is based on the idea that a human-selected driving path more accurately reflects both mechanical traversability and human understanding of traffic rules and visual information. In addition, we propose a data augmentation method and a loss weighting method for detecting the appropriate recommended traversable area from a single human-selected path. Evaluation of the results showed that the proposed learning methods are effective for recommended traversable area detection and found that weakly-supervised semantic segmentation using human-selected path information is useful for recommended area detection in environments with no edges.

CSENet: Cascade semantic erasing network for weakly-supervised semantic segmentation

2020 ◽  
Author(s):  
Jiahui Liu ◽  
Changqian Yu ◽  
Beibei Yang ◽  
Changxin Gao ◽  
Nong Sang
Keyword(s):  
Semantic Segmentation ◽  
Weakly Supervised
Sign in / Sign up

Export Citation Format

Share Document