scholarly journals Panoptic Feature Fusion Net: A Novel Instance Segmentation Paradigm for Biomedical and Biological Images

2021 ◽  
Vol 30 ◽  
pp. 2045-2059
Author(s):  
Dongnan Liu ◽  
Donghao Zhang ◽  
Yang Song ◽  
Heng Huang ◽  
Weidong Cai
Keyword(s):  
Feature Fusion ◽  
Biological Images ◽  
Instance Segmentation

scholarly journals JSNet: Joint Instance and Semantic Segmentation of 3D Point Clouds

2020 ◽  
Vol 34 (07) ◽  
pp. 12951-12958 ◽  
Author(s):  
Lin Zhao ◽  
Wenbing Tao
Keyword(s):  
Point Cloud ◽  
Large Scale ◽  
Feature Fusion ◽  
Mean Shift ◽  
Semantic Segmentation ◽  
Point Clouds ◽  
Semantic Features ◽  
Backbone Network ◽  
3D Point Clouds ◽  
Instance Segmentation

In this paper, we propose a novel joint instance and semantic segmentation approach, which is called JSNet, in order to address the instance and semantic segmentation of 3D point clouds simultaneously. Firstly, we build an effective backbone network to extract robust features from the raw point clouds. Secondly, to obtain more discriminative features, a point cloud feature fusion module is proposed to fuse the different layer features of the backbone network. Furthermore, a joint instance semantic segmentation module is developed to transform semantic features into instance embedding space, and then the transformed features are further fused with instance features to facilitate instance segmentation. Meanwhile, this module also aggregates instance features into semantic feature space to promote semantic segmentation. Finally, the instance predictions are generated by applying a simple mean-shift clustering on instance embeddings. As a result, we evaluate the proposed JSNet on a large-scale 3D indoor point cloud dataset S3DIS and a part dataset ShapeNet, and compare it with existing approaches. Experimental results demonstrate our approach outperforms the state-of-the-art method in 3D instance segmentation with a significant improvement in 3D semantic prediction and our method is also beneficial for part segmentation. The source code for this work is available at https://github.com/dlinzhao/JSNet.

scholarly journals Synergistic Attention for Ship Instance Segmentation in SAR Images

2021 ◽  
Vol 13 (21) ◽  
pp. 4384
Author(s):  
Danpei Zhao ◽  
Chunbo Zhu ◽  
Jing Qi ◽  
Xinhu Qi ◽  
Zhenhua Su ◽  
...  
Keyword(s):  
Target Location ◽  
Feature Fusion ◽  
Sar Images ◽  
Ship Detection ◽  
Multi Scale ◽  
Whole Process ◽  
Optical Images ◽  
Segmentation Methods ◽  
Target Characteristics ◽  
Instance Segmentation

This paper takes account of the fact that there is a lack of consideration for imaging methods and target characteristics of synthetic aperture radar (SAR) images among existing instance segmentation methods designed for optical images. Thus, we propose a method for SAR ship instance segmentation based on the synergistic attention mechanism which not only improves the performance of ship detection with multi-task branches but also provides pixel-level contours for subsequent applications such as orientation or category determination. The proposed method—SA R-CNN—presents a synergistic attention strategy at the image, semantic, and target level with the following module corresponding to the different stages in the whole process of the instance segmentation framework. The global attention module (GAM), semantic attention module(SAM), and anchor attention module (AAM) were constructed for feature extraction, feature fusion, and target location, respectively, for multi-scale ship targets under complex background conditions. Compared with several state-of-the-art methods, our method reached 68.7 AP in detection and 56.5 AP in segmentation on the HRSID dataset, and showed 91.5 AP in the detection task on the SSDD dataset.

scholarly journals Object-Guided Instance Segmentation for Biological Images

2020 ◽  
Vol 34 (07) ◽  
pp. 12677-12684
Author(s):  
Jingru Yi ◽  
Hui Tang ◽  
Pengxiang Wu ◽  
Bo Liu ◽  
Daniel J. Hoeppner ◽  
...  
Keyword(s):  
Target Object ◽  
Plant Phenotyping ◽  
Neural Cells ◽  
Cell Nuclei ◽  
Biological Images ◽  
Segmentation Methods ◽  
Level Information ◽  
Bounding Boxes ◽  
The Individual ◽  
Instance Segmentation

Instance segmentation of biological images is essential for studying object behaviors and properties. The challenges, such as clustering, occlusion, and adhesion problems of the objects, make instance segmentation a non-trivial task. Current box-free instance segmentation methods typically rely on local pixel-level information. Due to a lack of global object view, these methods are prone to over- or under-segmentation. On the contrary, the box-based instance segmentation methods incorporate object detection into the segmentation, performing better in identifying the individual instances. In this paper, we propose a new box-based instance segmentation method. Mainly, we locate the object bounding boxes from their center points. The object features are subsequently reused in the segmentation branch as a guide to separate the clustered instances within an RoI patch. Along with the instance normalization, the model is able to recover the target object distribution and suppress the distribution of neighboring attached objects. Consequently, the proposed model performs excellently in segmenting the clustered objects while retaining the target object details. The proposed method achieves state-of-the-art performances on three biological datasets: cell nuclei, plant phenotyping dataset, and neural cells.

scholarly journals Accurate Instance Segmentation for Remote Sensing Images via Adaptive and Dynamic Feature Learning

2021 ◽  
Vol 13 (23) ◽  
pp. 4774
Author(s):  
Feng Yang ◽  
Xiangyue Yuan ◽  
Jie Ran ◽  
Wenqiang Shu ◽  
Yue Zhao ◽  
...  
Keyword(s):  
Remote Sensing ◽  
Feature Fusion ◽  
Contextual Information ◽  
Sample Selection ◽  
Size Variation ◽  
Feature Learning ◽  
Dynamic Feature ◽  
Threshold Strategy ◽  
Remote Sensing Images ◽  
Instance Segmentation

Instance segmentation for high-resolution remote sensing images (HRSIs) is a fundamental yet challenging task in earth observation, which aims at achieving instance-level location and pixel-level classification for instances of interest on the earth’s surface. The main difficulties come from the huge scale variation, arbitrary instance shapes, and numerous densely packed small objects in HRSIs. In this paper, we design an end-to-end multi-category instance segmentation network for HRSIs, where three new modules based on adaptive and dynamic feature learning are proposed to address the above issues. The cross-scale adaptive fusion (CSAF) module introduces a novel multi-scale feature fusion mechanism to enhance the capability of the model to detect and segment objects with noticeable size variation. To predict precise masks for the complex boundaries of remote sensing instances, we embed a context attention upsampling (CAU) kernel instead of deconvolution in the segmentation branch to aggregate contextual information for refined upsampling. Furthermore, we extend the general fixed positive and negative sample judgment threshold strategy into a dynamic sample selection (DSS) module to select more suitable positive and negative samples flexibly for densely packed instances. These three modules enable a better feature learning of the instance segmentation network. Extensive experiments are conducted on the iSAID and NWU VHR-10 instance segmentation datasets to validate the proposed method. Attributing to the three proposed modules, we have achieved 1.9% and 2.9% segmentation performance improvements on these two datasets compared with the baseline method and achieved the state-of-the-art performance.

scholarly journals MSIS: Multispectral Instance Segmentation Method for Power Equipment

2022 ◽  
Vol 2022 ◽  
pp. 1-13
Author(s):  
Jun Shu ◽  
Juncheng He ◽  
Ling Li
Keyword(s):  
Feature Extraction ◽  
Fault Detection ◽  
Feature Fusion ◽  
Infrared Image ◽  
Current Method ◽  
Power Equipment ◽  
Infrared Images ◽  
Low Contrast ◽  
Novel Structure ◽  
Instance Segmentation

Infrared image of power equipment is widely used in power equipment fault detection, and segmentation of infrared images is an important step in power equipment thermal fault detection. Nevertheless, since the overlap of the equipment, the complex background, and the low contrast of the infrared image, the current method still cannot complete the detection and segmentation of the power equipment well. To better segment the power equipment in the infrared image, in this paper, a multispectral instance segmentation (MSIS) based on SOLOv2 is designed, which is an end-to-end and single-stage network. First, we provide a novel structure of multispectral feature extraction, which can simultaneously obtain rich features in visible images and infrared images. Secondly, a module of feature fusion (MARFN) has been constructed to fully obtain fusion features. Finally, the combination of multispectral feature extraction, the module of feature fusion (MARFN), and instance segmentation (SOLOv2) realize multispectral instance segmentation of power equipment. The experimental results show that the proposed MSIS model has an excellent performance in the instance segmentation of power equipment. The MSIS based on ResNet-50 has 40.06% AP.

scholarly journals Mask-Refined R-CNN: A Network for Refining Object Details in Instance Segmentation

Sensors ◽  
2020 ◽  
Vol 20 (4) ◽  
pp. 1010 ◽  
Author(s):  
Yiqing Zhang ◽  
Jun Chu ◽  
Lu Leng ◽  
Jun Miao
Keyword(s):  
Receptive Field ◽  
Spatial Information ◽  
Feature Fusion ◽  
State Of The Art ◽  
Rapid Development ◽  
Experimental Results ◽  
Small Scale ◽  
Feature Maps ◽  
Segmentation Accuracy ◽  
Instance Segmentation

With the rapid development of flexible vision sensors and visual sensor networks, computer vision tasks, such as object detection and tracking, are entering a new phase. Accordingly, the more challenging comprehensive task, including instance segmentation, can develop rapidly. Most state-of-the-art network frameworks, for instance, segmentation, are based on Mask R-CNN (mask region-convolutional neural network). However, the experimental results confirm that Mask R-CNN does not always successfully predict instance details. The scale-invariant fully convolutional network structure of Mask R-CNN ignores the difference in spatial information between receptive fields of different sizes. A large-scale receptive field focuses more on detailed information, whereas a small-scale receptive field focuses more on semantic information. So the network cannot consider the relationship between the pixels at the object edge, and these pixels will be misclassified. To overcome this problem, Mask-Refined R-CNN (MR R-CNN) is proposed, in which the stride of ROIAlign (region of interest align) is adjusted. In addition, the original fully convolutional layer is replaced with a new semantic segmentation layer that realizes feature fusion by constructing a feature pyramid network and summing the forward and backward transmissions of feature maps of the same resolution. The segmentation accuracy is substantially improved by combining the feature layers that focus on the global and detailed information. The experimental results on the COCO (Common Objects in Context) and Cityscapes datasets demonstrate that the segmentation accuracy of MR R-CNN is about 2% higher than that of Mask R-CNN using the same backbone. The average precision of large instances reaches 56.6%, which is higher than those of all state-of-the-art methods. In addition, the proposed method requires low time cost and is easily implemented. The experiments on the Cityscapes dataset also prove that the proposed method has great generalization ability.

scholarly journals Nuclei Segmentation via a Deep Panoptic Model with Semantic Feature Fusion

2019 ◽  
Author(s):  
Dongnan Liu ◽  
Donghao Zhang ◽  
Yang Song ◽  
Chaoyi Zhang ◽  
Fan Zhang ◽  
...  
Keyword(s):  
Feature Fusion ◽  
Semantic Segmentation ◽  
Local Features ◽  
Information Loss ◽  
Nuclei Segmentation ◽  
Diagnosis And Prognosis ◽  
Segmentation Methods ◽  
Global And Local ◽  
Instance Segmentation ◽  
Segmentation Models

Automated detection and segmentation of individual nuclei in histopathology images is important for cancer diagnosis and prognosis. Due to the high variability of nuclei appearances and numerous overlapping objects, this task still remains challenging. Deep learning based semantic and instance segmentation models have been proposed to address the challenges, but these methods tend to concentrate on either the global or local features and hence still suffer from information loss. In this work, we propose a panoptic segmentation model which incorporates an auxiliary semantic segmentation branch with the instance branch to integrate global and local features. Furthermore, we design a feature map fusion mechanism in the instance branch and a new mask generator to prevent information loss. Experimental results on three different histopathology datasets demonstrate that our method outperforms the state-of-the-art nuclei segmentation methods and popular semantic and instance segmentation models by a large margin.

Sign in / Sign up

Export Citation Format

Share Document