scholarly journals EfficientPS: Efficient Panoptic Segmentation

2021 ◽  
Author(s):  
Rohit Mohan ◽  
Abhinav Valada
Keyword(s):  
State Of The Art ◽  
Autonomous Robot ◽  
Multi Scale ◽  
Contextual Features ◽  
New Variant ◽  
Competent Functioning ◽  
Segmentation Task

AbstractUnderstanding the scene in which an autonomous robot operates is critical for its competent functioning. Such scene comprehension necessitates recognizing instances of traffic participants along with general scene semantics which can be effectively addressed by the panoptic segmentation task. In this paper, we introduce the Efficient Panoptic Segmentation (EfficientPS) architecture that consists of a shared backbone which efficiently encodes and fuses semantically rich multi-scale features. We incorporate a new semantic head that aggregates fine and contextual features coherently and a new variant of Mask R-CNN as the instance head. We also propose a novel panoptic fusion module that congruously integrates the output logits from both the heads of our EfficientPS architecture to yield the final panoptic segmentation output. Additionally, we introduce the KITTI panoptic segmentation dataset that contains panoptic annotations for the popularly challenging KITTI benchmark. Extensive evaluations on Cityscapes, KITTI, Mapillary Vistas and Indian Driving Dataset demonstrate that our proposed architecture consistently sets the new state-of-the-art on all these four benchmarks while being the most efficient and fast panoptic segmentation architecture to date.

scholarly journals Attention-Based Multi-Context Guiding for Few-Shot Semantic Segmentation

2019 ◽  
Vol 33 ◽  
pp. 8441-8448 ◽  
Author(s):  
Tao Hu ◽  
Pengwan Yang ◽  
Chiliang Zhang ◽  
Gang Yu ◽  
Yadong Mu ◽  
...  
Keyword(s):  
Deep Learning ◽  
Feature Fusion ◽  
State Of The Art ◽  
Semantic Segmentation ◽  
Research Topic ◽  
Context Information ◽  
Multi Scale ◽  
Support Set ◽  
Segmentation Task ◽  
Context Features

Few-shot learning is a nascent research topic, motivated by the fact that traditional deep learning methods require tremendous amounts of data. The scarcity of annotated data becomes even more challenging in semantic segmentation since pixellevel annotation in segmentation task is more labor-intensive to acquire. To tackle this issue, we propose an Attentionbased Multi-Context Guiding (A-MCG) network, which consists of three branches: the support branch, the query branch, the feature fusion branch. A key differentiator of A-MCG is the integration of multi-scale context features between support and query branches, enforcing a better guidance from the support set. In addition, we also adopt a spatial attention along the fusion branch to highlight context information from several scales, enhancing self-supervision in one-shot learning. To address the fusion problem in multi-shot learning, Conv-LSTM is adopted to collaboratively integrate the sequential support features to elevate the final accuracy. Our architecture obtains state-of-the-art on unseen classes in a variant of PASCAL VOC12 dataset and performs favorably against previous work with large gains of 1.1%, 1.4% measured in mIoU in the 1-shot and 5-shot setting.

scholarly journals A Novel Multi-Scale Attention PFE-UNet for Forest Image Segmentation

Forests ◽  
2021 ◽  
Vol 12 (7) ◽  
pp. 937
Author(s):  
Boyang Zhang ◽  
Hongbo Mu ◽  
Mingyu Gao ◽  
Haiming Ni ◽  
Jianfeng Chen ◽  
...  
Keyword(s):  
Feature Extraction ◽  
Image Segmentation ◽  
State Of The Art ◽  
Computational Cost ◽  
Receptive Fields ◽  
Feature Maps ◽  
Multi Scale ◽  
Extensive Evaluation ◽  
Network Transition ◽  
Segmentation Task

The precise segmentation of forest areas is essential for monitoring tasks related to forest exploration, extraction, and statistics. However, the effective and accurate segmentation of forest images will be affected by factors such as blurring and discontinuity of forest boundaries. Therefore, a Pyramid Feature Extraction-UNet network (PFE-UNet) based on traditional UNet is proposed to be applied to end-to-end forest image segmentation. Among them, the Pyramid Feature Extraction module (PFE) is introduced in the network transition layer, which obtains multi-scale forest image information through different receptive fields. The spatial attention module (SA) and the channel-wise attention module (CA) are applied to low-level feature maps and PFE feature maps, respectively, to highlight specific segmentation task features while fusing context information and suppressing irrelevant regions. The standard convolution block is replaced by a novel depthwise separable convolutional unit (DSC Unit), which not only reduces the computational cost but also prevents overfitting. This paper presents an extensive evaluation with the DeepGlobe dataset and a comparative analysis with several state-of-the-art networks. The experimental results show that the PFE-UNet network obtains an accuracy of 94.23% in handling the real-time forest image segmentation, which is significantly higher than other advanced networks. This means that the proposed PFE-UNet also provides a valuable reference for the precise segmentation of forest images.

scholarly journals PCAN—Part-Based Context Attention Network for Thermal Power Plant Detection in Remote Sensing Imagery

2021 ◽  
Vol 13 (7) ◽  
pp. 1243
Author(s):  
Wenxin Yin ◽  
Wenhui Diao ◽  
Peijin Wang ◽  
Xin Gao ◽  
Ya Li ◽  
...  
Keyword(s):  
Remote Sensing ◽  
Power Plants ◽  
State Of The Art ◽  
Thermal Power ◽  
Image Interpretation ◽  
Remote Sensing Image ◽  
Thermal Power Plants ◽  
Average Precision ◽  
Deep Convolutional Neural Networks ◽  
Multi Scale

The detection of Thermal Power Plants (TPPs) is a meaningful task for remote sensing image interpretation. It is a challenging task, because as facility objects TPPs are composed of various distinctive and irregular components. In this paper, we propose a novel end-to-end detection framework for TPPs based on deep convolutional neural networks. Specifically, based on the RetinaNet one-stage detector, a context attention multi-scale feature extraction network is proposed to fuse global spatial attention to strengthen the ability in representing irregular objects. In addition, we design a part-based attention module to adapt to TPPs containing distinctive components. Experiments show that the proposed method outperforms the state-of-the-art methods and can achieve 68.15% mean average precision.

scholarly journals Hybrid Graph Neural Networks for Crowd Counting

2020 ◽  
Vol 34 (07) ◽  
pp. 11693-11700 ◽  
Author(s):  
Ao Luo ◽  
Fan Yang ◽  
Xin Li ◽  
Dong Nie ◽  
Zhicheng Jiao ◽  
...  
Keyword(s):  
Network Architecture ◽  
Message Passing ◽  
Large Scale ◽  
State Of The Art ◽  
Density Variation ◽  
Feature Maps ◽  
Crowd Counting ◽  
Multi Scale ◽  
Crowd Density ◽  
Graph Neural Networks

Crowd counting is an important yet challenging task due to the large scale and density variation. Recent investigations have shown that distilling rich relations among multi-scale features and exploiting useful information from the auxiliary task, i.e., localization, are vital for this task. Nevertheless, how to comprehensively leverage these relations within a unified network architecture is still a challenging problem. In this paper, we present a novel network structure called Hybrid Graph Neural Network (HyGnn) which targets to relieve the problem by interweaving the multi-scale features for crowd density as well as its auxiliary task (localization) together and performing joint reasoning over a graph. Specifically, HyGnn integrates a hybrid graph to jointly represent the task-specific feature maps of different scales as nodes, and two types of relations as edges: (i) multi-scale relations capturing the feature dependencies across scales and (ii) mutual beneficial relations building bridges for the cooperation between counting and localization. Thus, through message passing, HyGnn can capture and distill richer relations between nodes to obtain more powerful representations, providing robust and accurate results. Our HyGnn performs significantly well on four challenging datasets: ShanghaiTech Part A, ShanghaiTech Part B, UCF_CC_50 and UCF_QNRF, outperforming the state-of-the-art algorithms by a large margin.

MULTI-SCALE GRADIENT INVARIANT FOR FACE RECOGNITION UNDER VARYING ILLUMINATION

2012 ◽  
Vol 26 (08) ◽  
pp. 1256016 ◽  
Author(s):  
BIN XU ◽  
YUAN YAN TANG ◽  
BIN FANG ◽  
ZHAO WEI SHANG
Keyword(s):  
Face Recognition ◽  
State Of The Art ◽  
Image Noise ◽  
Scale Analysis ◽  
Image Gradient ◽  
Multi Scale ◽  
Novel Approach ◽  
Gradient Domain ◽  
Inherent Structure ◽  
Multi Scale Analysis

In this paper, a novel approach derived from image gradient domain called multi-scale gradient faces (MGF) is proposed to abstract multi-scale illumination-insensitive measure for face recognition. MGF applies multi-scale analysis on image gradient information, which can discover underlying inherent structure in images and keep the details at most while removing varying lighting. The proposed approach provides state-of-the-art performance on Extended YaleB and PIE: Recognition rates of 99.11% achieved on PIE database and 99.38% achieved on YaleB which outperforms most existing approaches. Furthermore, the experimental results on noised Yale-B validate that MGF is more robust to image noise.

scholarly journals Correlation Tracking via Self-Adaptive Fusion of Multiple Features

Information ◽  
2018 ◽  
Vol 9 (10) ◽  
pp. 241 ◽  
Author(s):  
Zhi Chen ◽  
Peizhong Liu ◽  
Yongzhao Du ◽  
Yanmin Luo ◽  
Wancheng Zhang
Keyword(s):  
State Of The Art ◽  
Correlation Filter ◽  
Multiple Features ◽  
Multi Scale ◽  
Tracking Algorithms ◽  
Model Update ◽  
Adaptive Fusion ◽  
Tracking Model ◽  
Update Strategy ◽  
Self Adaptive

Correlation filter (CF) based tracking algorithms have shown excellent performance in comparison to most state-of-the-art algorithms on the object tracking benchmark (OTB). Nonetheless, most CF based tracking algorithms only consider limited single channel feature, and the tracking model always updated from frame-by-frame. It will generate some erroneous information when the target objects undergo sophisticated scenario changes, such as background clutter, occlusion, out-of-view, and so forth. Long-term accumulation of erroneous model updating will cause tracking drift. In order to address problems that are mentioned above, in this paper, we propose a robust multi-scale correlation filter tracking algorithm via self-adaptive fusion of multiple features. First, we fuse powerful multiple features including histogram of oriented gradients (HOG), color name (CN), and histogram of local intensities (HI) in the response layer. The weights assigned according to the proportion of response scores that are generated by each feature, which achieve self-adaptive fusion of multiple features for preferable feature representation. In the meantime the efficient model update strategy is proposed, which is performed by exploiting a pre-defined response threshold as discriminative condition for updating tracking model. In addition, we introduce an accurate multi-scale estimation method integrate with the model update strategy, which further improves the scale variation adaptability. Both qualitative and quantitative evaluations on challenging video sequences demonstrate that the proposed tracker performs superiorly against the state-of-the-art CF based methods.

scholarly journals DCPNet: A Densely Connected Pyramid Network for Monocular Depth Estimation

Sensors ◽  
2021 ◽  
Vol 21 (20) ◽  
pp. 6780
Author(s):  
Zhitong Lai ◽  
Rui Tian ◽  
Zhiguo Wu ◽  
Nannan Ding ◽  
Linjian Sun ◽  
...  
Keyword(s):  
Multiple Scales ◽  
Feature Fusion ◽  
State Of The Art ◽  
Depth Estimation ◽  
Multi Scale ◽  
Pyramid Structure ◽  
Benchmark Datasets ◽  
The Common ◽  
Monocular Depth ◽  
Multiple Stages

Pyramid architecture is a useful strategy to fuse multi-scale features in deep monocular depth estimation approaches. However, most pyramid networks fuse features only within the adjacent stages in a pyramid structure. To take full advantage of the pyramid structure, inspired by the success of DenseNet, this paper presents DCPNet, a densely connected pyramid network that fuses multi-scale features from multiple stages of the pyramid structure. DCPNet not only performs feature fusion between the adjacent stages, but also non-adjacent stages. To fuse these features, we design a simple and effective dense connection module (DCM). In addition, we offer a new consideration of the common upscale operation in our approach. We believe DCPNet offers a more efficient way to fuse features from multiple scales in a pyramid-like network. We perform extensive experiments using both outdoor and indoor benchmark datasets (i.e., the KITTI and the NYU Depth V2 datasets) and DCPNet achieves the state-of-the-art results.

scholarly journals Encoding Time Series as Multi-Scale Signed Recurrence Plots for Classification Using Fully Convolutional Networks

Sensors ◽  
2020 ◽  
Vol 20 (14) ◽  
pp. 3818
Author(s):  
Ye Zhang ◽  
Yi Hou ◽  
Shilin Zhou ◽  
Kewei Ouyang
Keyword(s):  
Time Series ◽  
State Of The Art ◽  
Recurrence Plot ◽  
The State ◽  
Recurrence Plots ◽  
Convolutional Networks ◽  
Multi Scale ◽  
Fully Convolutional Networks ◽  
Benchmark Datasets ◽  
Visualization Evaluation

Recent advances in time series classification (TSC) have exploited deep neural networks (DNN) to improve the performance. One promising approach encodes time series as recurrence plot (RP) images for the sake of leveraging the state-of-the-art DNN to achieve accuracy. Such an approach has been shown to achieve impressive results, raising the interest of the community in it. However, it remains unsolved how to handle not only the variability in the distinctive region scale and the length of sequences but also the tendency confusion problem. In this paper, we tackle the problem using Multi-scale Signed Recurrence Plots (MS-RP), an improvement of RP, and propose a novel method based on MS-RP images and Fully Convolutional Networks (FCN) for TSC. This method first introduces phase space dimension and time delay embedding of RP to produce multi-scale RP images; then, with the use of asymmetrical structure, constructed RP images can represent very long sequences (>700 points). Next, MS-RP images are obtained by multiplying designed sign masks in order to remove the tendency confusion. Finally, FCN is trained with MS-RP images to perform classification. Experimental results on 45 benchmark datasets demonstrate that our method improves the state-of-the-art in terms of classification accuracy and visualization evaluation.

scholarly journals SPSTracker: Sub-Peak Suppression of Response Map for Robust Object Tracking

2020 ◽  
Vol 34 (07) ◽  
pp. 10989-10996
Author(s):  
Qintao Hu ◽  
Lijun Zhou ◽  
Xiaoxiao Wang ◽  
Yao Mao ◽  
Jianlin Zhang ◽  
...  
Keyword(s):  
Online Learning ◽  
Object Tracking ◽  
Background Noise ◽  
State Of The Art ◽  
Response Suppression ◽  
Learning Approach ◽  
Response Distribution ◽  
Peak Response ◽  
Multi Scale ◽  
Tracking Response

Modern visual trackers usually construct online learning models under the assumption that the feature response has a Gaussian distribution with target-centered peak response. Nevertheless, such an assumption is implausible when there is progressive interference from other targets and/or background noise, which produce sub-peaks on the tracking response map and cause model drift. In this paper, we propose a rectified online learning approach for sub-peak response suppression and peak response enforcement and target at handling progressive interference in a systematic way. Our approach, referred to as SPSTracker, applies simple-yet-efficient Peak Response Pooling (PRP) to aggregate and align discriminative features, as well as leveraging a Boundary Response Truncation (BRT) to reduce the variance of feature response. By fusing with multi-scale features, SPSTracker aggregates the response distribution of multiple sub-peaks to a single maximum peak, which enforces the discriminative capability of features for robust object tracking. Experiments on the OTB, NFS and VOT2018 benchmarks demonstrate that SPSTrack outperforms the state-of-the-art real-time trackers with significant margins1

Sign in / Sign up

Export Citation Format

Share Document