Y-Net: Dual-branch Joint Network for Semantic Segmentation

2021 ◽  
Vol 17 (4) ◽  
pp. 1-22
Author(s):  
Yizhen Chen ◽  
Haifeng Hu
Keyword(s):  
Feature Vector ◽  
State Of The Art ◽  
Computational Cost ◽  
Receptive Fields ◽  
Semantic Segmentation ◽  
Global Context ◽  
Multi Level ◽  
The One ◽  
Public Datasets ◽  
High Level

Most existing segmentation networks are built upon a “ U -shaped” encoder–decoder structure, where the multi-level features extracted by the encoder are gradually aggregated by the decoder. Although this structure has been proven to be effective in improving segmentation performance, there are two main drawbacks. On the one hand, the introduction of low-level features brings a significant increase in calculations without an obvious performance gain. On the other hand, general strategies of feature aggregation such as addition and concatenation fuse features without considering the usefulness of each feature vector, which mixes the useful information with massive noises. In this article, we abandon the traditional “ U -shaped” architecture and propose Y-Net, a dual-branch joint network for accurate semantic segmentation. Specifically, it only aggregates the high-level features with low-resolution and utilizes the global context guidance generated by the first branch to refine the second branch. The dual branches are effectively connected through a Semantic Enhancing Module, which can be regarded as the combination of spatial attention and channel attention. We also design a novel Channel-Selective Decoder (CSD) to adaptively integrate features from different receptive fields by assigning specific channelwise weights, where the weights are input-dependent. Our Y-Net is capable of breaking through the limit of singe-branch network and attaining higher performance with less computational cost than “ U -shaped” structure. The proposed CSD can better integrate useful information and suppress interference noises. Comprehensive experiments are carried out on three public datasets to evaluate the effectiveness of our method. Eventually, our Y-Net achieves state-of-the-art performance on PASCAL VOC 2012, PASCAL Person-Part, and ADE20K dataset without pre-training on extra datasets.

scholarly journals Multi-Level and Multi-Scale Feature Aggregation Network for Semantic Segmentation in Vehicle-Mounted Scenes

Sensors ◽  
2021 ◽  
Vol 21 (9) ◽  
pp. 3270
Author(s):  
Yong Liao ◽  
Qiong Liu
Keyword(s):  
Receptive Fields ◽  
Semantic Segmentation ◽  
Layer By Layer ◽  
Scale Feature ◽  
Backbone Networks ◽  
Multi Scale ◽  
Backbone Network ◽  
Feature Aggregation ◽  
Multi Level ◽  
High Level

The main challenges of semantic segmentation in vehicle-mounted scenes are object scale variation and trading off model accuracy and efficiency. Lightweight backbone networks for semantic segmentation usually extract single-scale features layer-by-layer only by using a fixed receptive field. Most modern real-time semantic segmentation networks heavily compromise spatial details when encoding semantics, and sacrifice accuracy for speed. Many improving strategies adopt dilated convolution and add a sub-network, in which either intensive computation or redundant parameters are brought. We propose a multi-level and multi-scale feature aggregation network (MMFANet). A spatial pyramid module is designed by cascading dilated convolutions with different receptive fields to extract multi-scale features layer-by-layer. Subseqently, a lightweight backbone network is built by reducing the feature channel capacity of the module. To improve the accuracy of our network, we design two additional modules to separately capture spatial details and high-level semantics from the backbone network without significantly increasing the computation cost. Comprehensive experimental results show that our model achieves 79.3% MIoU on the Cityscapes test dataset at a speed of 58.5 FPS, and it is more accurate than SwiftNet (75.5% MIoU). Furthermore, the number of parameters of our model is at least 53.38% less than that of other state-of-the-art models.

scholarly journals Gated Fully Fusion for Semantic Segmentation

2020 ◽  
Vol 34 (07) ◽  
pp. 11418-11425 ◽  
Author(s):  
Xiangtai Li ◽  
Houlong Zhao ◽  
Lei Han ◽  
Yunhai Tong ◽  
Shaohua Tan ◽  
...  
Keyword(s):  
Neural Networks ◽  
State Of The Art ◽  
Semantic Segmentation ◽  
Semantic Gap ◽  
Comprehensive Understanding ◽  
Deep Convolutional Neural Networks ◽  
Multi Level ◽  
Multiple Levels ◽  
High Level ◽  
Fully Connected

Semantic segmentation generates comprehensive understanding of scenes through densely predicting the category for each pixel. High-level features from Deep Convolutional Neural Networks already demonstrate their effectiveness in semantic segmentation tasks, however the coarse resolution of high-level features often leads to inferior results for small/thin objects where detailed information is important. It is natural to consider importing low level features to compensate for the lost detailed information in high-level features. Unfortunately, simply combining multi-level features suffers from the semantic gap among them. In this paper, we propose a new architecture, named Gated Fully Fusion(GFF), to selectively fuse features from multiple levels using gates in a fully connected way. Specifically, features at each level are enhanced by higher-level features with stronger semantics and lower-level features with more details, and gates are used to control the propagation of useful information which significantly reduces the noises during fusion. We achieve the state of the art results on four challenging scene parsing datasets including Cityscapes, Pascal Context, COCO-stuff and ADE20K.

scholarly journals Context-aware Cross-level Fusion Network for Camouflaged Object Detection

2021 ◽  
Author(s):  
Yujia Sun ◽  
Geng Chen ◽  
Tao Zhou ◽  
Yi Zhang ◽  
Nian Liu
Keyword(s):  
Object Detection ◽  
State Of The Art ◽  
Context Aware ◽  
Global Context ◽  
Feature Representations ◽  
Multi Scale ◽  
Benchmark Datasets ◽  
Multi Level ◽  
High Level ◽  
Level Fusion

Camouflaged object detection (COD) is a challenging task due to the low boundary contrast between the object and its surroundings. In addition, the appearance of camouflaged objects varies significantly, e.g., object size and shape, aggravating the difficulties of accurate COD. In this paper, we propose a novel Context-aware Cross-level Fusion Network (C2F-Net) to address the challenging COD task. Specifically, we propose an Attention-induced Cross-level Fusion Module (ACFM) to integrate the multi-level features with informative attention coefficients. The fused features are then fed to the proposed Dual-branch Global Context Module (DGCM), which yields multi-scale feature representations for exploiting rich global context information. In C2F-Net, the two modules are conducted on high-level features using a cascaded manner. Extensive experiments on three widely used benchmark datasets demonstrate that our C2F-Net is an effective COD model and outperforms state-of-the-art models remarkably. Our code is publicly available at: https://github.com/thograce/C2FNet.

scholarly journals The S66 Non-Covalent Interactions Benchmark Reconsidered Using Explicitly Correlated Methods Near the Basis Set Limit

2018 ◽  
Vol 71 (4) ◽  
pp. 238 ◽  
Author(s):  
Manoj K. Kesharwani ◽  
Amir Karton ◽  
Nitai Sylvetsky ◽  
Jan M. L. Martin
Keyword(s):  
Computational Cost ◽  
The Other ◽  
Basis Set ◽  
Basis Sets ◽  
Explicitly Correlated ◽  
Non Covalent Interactions ◽  
Explicitly Correlated Methods ◽  
The One ◽  
High Level ◽  
Covalent Interactions

The S66 benchmark for non-covalent interactions has been re-evaluated using explicitly correlated methods with basis sets near the one-particle basis set limit. It is found that post-MP2 ‘high-level corrections’ are treated adequately well using a combination of CCSD(F12*) with (aug-)cc-pVTZ-F12 basis sets on the one hand, and (T) extrapolated from conventional CCSD(T)/heavy-aug-cc-pV{D,T}Z on the other hand. Implications for earlier benchmarks on the larger S66×8 problem set in particular, and for accurate calculations on non-covalent interactions in general, are discussed. At a slight cost in accuracy, (T) can be considerably accelerated by using sano-V{D,T}Z+ basis sets, whereas half-counterpoise CCSD(F12*)(T)/cc-pVDZ-F12 offers the best compromise between accuracy and computational cost.

scholarly journals Water Areas Segmentation from Remote Sensing Images Using a Separable Residual SegNet Network

2020 ◽  
Vol 9 (4) ◽  
pp. 256 ◽  
Author(s):  
Liguo Weng ◽  
Yiming Xu ◽  
Min Xia ◽  
Yonghong Zhang ◽  
Jia Liu ◽  
...  
Keyword(s):  
Remote Sensing ◽  
Feature Extraction ◽  
Spatial Information ◽  
Global Climate ◽  
Semantic Segmentation ◽  
Water Area ◽  
Remote Sensing Images ◽  
The One ◽  
High Level ◽  
Almost All

Changes on lakes and rivers are of great significance for the study of global climate change. Accurate segmentation of lakes and rivers is critical to the study of their changes. However, traditional water area segmentation methods almost all share the following deficiencies: high computational requirements, poor generalization performance, and low extraction accuracy. In recent years, semantic segmentation algorithms based on deep learning have been emerging. Addressing problems associated to a very large number of parameters, low accuracy, and network degradation during training process, this paper proposes a separable residual SegNet (SR-SegNet) to perform the water area segmentation using remote sensing images. On the one hand, without compromising the ability of feature extraction, the problem of network degradation is alleviated by adding modified residual blocks into the encoder, the number of parameters is limited by introducing depthwise separable convolutions, and the ability of feature extraction is improved by using dilated convolutions to expand the receptive field. On the other hand, SR-SegNet removes the convolution layers with relatively more convolution kernels in the encoding stage, and uses the cascading method to fuse the low-level and high-level features of the image. As a result, the whole network can obtain more spatial information. Experimental results show that the proposed method exhibits significant improvements over several traditional methods, including FCN, DeconvNet, and SegNet.

scholarly journals A new joint CTC-attention-based speech recognition model with multi-level multi-head attention

2019 ◽  
Vol 2019 (1) ◽  
Author(s):  
Chu-Xiong Qin ◽  
Wen-Lin Zhang ◽  
Dan Qu
Keyword(s):  
Speech Recognition ◽  
Nonnegative Matrix Factorization ◽  
State Of The Art ◽  
Nonnegative Matrix ◽  
Attention Mechanism ◽  
Word Error Rate ◽  
Absolute Value ◽  
Multi Level ◽  
End To End ◽  
High Level

Abstract A method called joint connectionist temporal classification (CTC)-attention-based speech recognition has recently received increasing focus and has achieved impressive performance. A hybrid end-to-end architecture that adds an extra CTC loss to the attention-based model could force extra restrictions on alignments. To explore better the end-to-end models, we propose improvements to the feature extraction and attention mechanism. First, we introduce a joint model trained with nonnegative matrix factorization (NMF)-based high-level features. Then, we put forward a hybrid attention mechanism by incorporating multi-head attentions and calculating attention scores over multi-level outputs. Experiments on TIMIT indicate that the new method achieves state-of-the-art performance with our best model. Experiments on WSJ show that our method exhibits a word error rate (WER) that is only 0.2% worse in absolute value than the best referenced method, which is trained on a much larger dataset, and it beats all present end-to-end methods. Further experiments on LibriSpeech show that our method is also comparable to the state-of-the-art end-to-end system in WER.

scholarly journals A Visual SLAM Robust against Dynamic Objects Based on Hybrid Semantic-Geometry Information

2021 ◽  
Vol 10 (10) ◽  
pp. 673
Author(s):  
Sheng Miao ◽  
Xiaoxiong Liu ◽  
Dazheng Wei ◽  
Changze Li
Keyword(s):  
Moving Objects ◽  
State Of The Art ◽  
Semantic Segmentation ◽  
Visual Localization ◽  
Epipolar Constraint ◽  
Trajectory Error ◽  
Localization And Mapping ◽  
Dynamic Objects ◽  
Public Datasets ◽  
Localization Approach

A visual localization approach for dynamic objects based on hybrid semantic-geometry information is presented. Due to the interference of moving objects in the real environment, the traditional simultaneous localization and mapping (SLAM) system can be corrupted. To address this problem, we propose a method for static/dynamic image segmentation that leverages semantic and geometric modules, including optical flow residual clustering, epipolar constraint checks, semantic segmentation, and outlier elimination. We integrated the proposed approach into the state-of-the-art ORB-SLAM2 and evaluated its performance on both public datasets and a quadcopter platform. Experimental results demonstrated that the root-mean-square error of the absolute trajectory error improved, on average, by 93.63% in highly dynamic benchmarks when compared with ORB-SLAM2. Thus, the proposed method can improve the performance of state-of-the-art SLAM systems in challenging scenarios.

scholarly journals Natural Language Processing in Serious Games: A state of the art.

2015 ◽  
Vol 2 (3) ◽  
Author(s):  
Davide Picca ◽  
Dominique Jaccard ◽  
Gérald Eberlé
Keyword(s):  
Natural Language Processing ◽  
Natural Language ◽  
Language Processing ◽  
Serious Games ◽  
State Of The Art ◽  
Serious Game ◽  
The Other ◽  
Other Hand ◽  
The One ◽  
High Level

In the last decades, Natural Language Processing (NLP) has obtained a high level of success. Interactions between NLP and Serious Games have started and some of them already include NLP techniques. The objectives of this paper are twofold: on the one hand, providing a simple framework to enable analysis of potential uses of NLP in Serious Games and, on the other hand, applying the NLP framework to existing Serious Games and giving an overview of the use of NLP in pedagogical Serious Games. In this paper we present 11 serious games exploiting NLP techniques. We present them systematically, according to the following structure:  first, we highlight possible uses of NLP techniques in Serious Games, second, we describe the type of NLP implemented in the each specific Serious Game and, third, we provide a link to possible purposes of use for the different actors interacting in the Serious Game.

scholarly journals Context-Aware Image Inpainting with Learned Semantic Priors

2021 ◽  
Author(s):  
Wendong Zhang ◽  
Junwei Zhu ◽  
Ying Tai ◽  
Yunbo Wang ◽  
Wenqing Chu ◽  
...  
Keyword(s):  
State Of The Art ◽  
Contextual Information ◽  
Image Inpainting ◽  
Context Aware ◽  
Global Context ◽  
Low Level ◽  
Complex Scenes ◽  
Knowledge Distillation ◽  
Image Generator ◽  
High Level

Recent advances in image inpainting have shown impressive results for generating plausible visual details on rather simple backgrounds. However, for complex scenes, it is still challenging to restore reasonable contents as the contextual information within the missing regions tends to be ambiguous. To tackle this problem, we introduce pretext tasks that are semantically meaningful to estimating the missing contents. In particular, we perform knowledge distillation on pretext models and adapt the features to image inpainting. The learned semantic priors ought to be partially invariant between the high-level pretext task and low-level image inpainting, which not only help to understand the global context but also provide structural guidance for the restoration of local textures. Based on the semantic priors, we further propose a context-aware image inpainting model, which adaptively integrates global semantics and local features in a unified image generator. The semantic learner and the image generator are trained in an end-to-end manner. We name the model SPL to highlight its ability to learn and leverage semantic priors. It achieves the state of the art on Places2, CelebA, and Paris StreetView datasets

MFECN: Multi-level Feature Enhanced Cumulative Network for Scene Text Detection

2021 ◽  
Vol 17 (3) ◽  
pp. 1-22
Author(s):  
Zhandong Liu ◽  
Wengang Zhou ◽  
Houqiang Li
Keyword(s):  
Text Detection ◽  
Detection Algorithms ◽  
Features Fusion ◽  
Irregular Shapes ◽  
Scene Text Detection ◽  
Scene Text ◽  
Text Information ◽  
Multi Level ◽  
Public Datasets ◽  
High Level

Recently, many scene text detection algorithms have achieved impressive performance by using convolutional neural networks. However, most of them do not make full use of the context among the hierarchical multi-level features to improve the performance of scene text detection. In this article, we present an efficient multi-level features enhanced cumulative framework based on instance segmentation for scene text detection. At first, we adopt a Multi-Level Features Enhanced Cumulative ( MFEC ) module to capture features of cumulative enhancement of representational ability. Then, a Multi-Level Features Fusion ( MFF ) module is designed to fully integrate both high-level and low-level MFEC features, which can adaptively encode scene text information. To verify the effectiveness of the proposed method, we perform experiments on six public datasets (namely, CTW1500, Total-text, MSRA-TD500, ICDAR2013, ICDAR2015, and MLT2017), and make comparisons with other state-of-the-art methods. Experimental results demonstrate that the proposed Multi-Level Features Enhanced Cumulative Network (MFECN) detector can well handle scene text instances with irregular shapes (i.e., curved, oriented, and horizontal) and achieves better or comparable results.

Sign in / Sign up

Export Citation Format

Share Document