ColAtt‐Net : In Reducing the Ambiguity of Pedestrian Orientations on Attribute‐Aware Semantic Segmentation Task

AbstractBiologists who use electron microscopy (EM) images to build nanoscale 3D models of whole cells and their organelles have historically been limited to small numbers of cells and cellular features due to constraints in imaging and analysis. This has been a major factor limiting insight into the complex variability of cellular environments. Modern EM can produce gigavoxel image volumes containing large numbers of cells, but accurate manual segmentation of image features is slow and limits the creation of cell models. Segmentation algorithms based on convolutional neural networks can process large volumes quickly, but achieving EM task accuracy goals often challenges current techniques. Here, we define dense cellular segmentation as a multiclass semantic segmentation task for modeling cells and large numbers of their organelles, and give an example in human blood platelets. We present an algorithm using novel hybrid 2D–3D segmentation networks to produce dense cellular segmentations with accuracy levels that outperform baseline methods and approach those of human annotators. To our knowledge, this work represents the first published approach to automating the creation of cell models with this level of structural detail.

Download Full-text

Adversarial Data Augmentation on Breast MRI Segmentation

Applied Sciences ◽

10.3390/app11104554 ◽

2021 ◽

Vol 11 (10) ◽

pp. 4554

Author(s):

João F. Teixeira ◽

Mariana Dias ◽

Eva Batista ◽

Joana Costa ◽

Luís F. Teixeira ◽

...

Keyword(s):

Data Augmentation ◽

Medical Image Analysis ◽

Synthetic Data ◽

Breast Mri ◽

Semantic Segmentation ◽

Adversarial Networks ◽

Augmentation Strategy ◽

Magnetic Resonance Imaging Mri ◽

Segmentation Task ◽

The Impact

The scarcity of balanced and annotated datasets has been a recurring problem in medical image analysis. Several researchers have tried to fill this gap employing dataset synthesis with adversarial networks (GANs). Breast magnetic resonance imaging (MRI) provides complex, texture-rich medical images, with the same annotation shortage issues, for which, to the best of our knowledge, no previous work tried synthesizing data. Within this context, our work addresses the problem of synthesizing breast MRI images from corresponding annotations and evaluate the impact of this data augmentation strategy on a semantic segmentation task. We explored variations of image-to-image translation using conditional GANs, namely fitting the generator’s architecture with residual blocks and experimenting with cycle consistency approaches. We studied the impact of these changes on visual verisimilarity and how an U-Net segmentation model is affected by the usage of synthetic data. We achieved sufficiently realistic-looking breast MRI images and maintained a stable segmentation score even when completely replacing the dataset with the synthetic set. Our results were promising, especially when concerning to Pix2PixHD and Residual CycleGAN architectures.

Download Full-text

Constructing a Convolutional Neural Network with a Suitable Capacity for a Semantic Segmentation Task

Deep Learning: Concepts and Architectures - Studies in Computational Intelligence ◽

10.1007/978-3-030-31756-0_8 ◽

2019 ◽

pp. 237-268

Author(s):

Yalong Jiang ◽

Zheru Chi

Keyword(s):

Neural Network ◽

Convolutional Neural Network ◽

Semantic Segmentation ◽

Segmentation Task

Download Full-text

Concrete Cracks Detection Based on FCN with Dilated Convolution

Applied Sciences ◽

10.3390/app9132686 ◽

2019 ◽

Vol 9 (13) ◽

pp. 2686 ◽

Cited By ~ 15

Author(s):

Jianming Zhang ◽

Chaoquan Lu ◽

Jin Wang ◽

Lei Wang ◽

Xiao-Guang Yue

Keyword(s):

Crack Detection ◽

Receptive Fields ◽

Semantic Segmentation ◽

Concrete Surface ◽

Input Image ◽

Feature Maps ◽

Test Set ◽

Dilated Convolution ◽

Fully Convolutional Networks ◽

Segmentation Task

In civil engineering, the stability of concrete is of great significance to safety of people’s life and property, so it is necessary to detect concrete damage effectively. In this paper, we treat crack detection on concrete surface as a semantic segmentation task that distinguishes background from crack at the pixel level. Inspired by Fully Convolutional Networks (FCN), we propose a full convolution network based on dilated convolution for concrete crack detection, which consists of an encoder and a decoder. Specifically, we first used the residual network to extract the feature maps of the input image, designed the dilated convolutions with different dilation rates to extract the feature maps of different receptive fields, and fused the extracted features from multiple branches. Then, we exploited the stacked deconvolution to do up-sampling operator in the fused feature maps. Finally, we used the SoftMax function to classify the feature maps at the pixel level. In order to verify the validity of the model, we introduced the commonly used evaluation indicators of semantic segmentation: Pixel Accuracy (PA), Mean Pixel Accuracy (MPA), Mean Intersection over Union (MIoU), and Frequency Weighted Intersection over Union (FWIoU). The experimental results show that the proposed model converges faster and has better generalization performance on the test set by introducing dilated convolutions with different dilation rates and a multi-branch fusion strategy. Our model has a PA of 96.84%, MPA of 92.55%, MIoU of 86.05% and FWIoU of 94.22% on the test set, which is superior to other models.

Download Full-text

Deep learning Q inversion from reflection seismic data with strong attenuation using an encoder-decoder convolutional neural network: an example from South China Sea

10.5194/egusphere-egu2020-3809 ◽

2020 ◽

Author(s):

Hao Zhang ◽

Jianguang Han ◽

Heng Zhang ◽

Yi Zhang

Keyword(s):

Neural Network ◽

Deep Learning ◽

High Resolution ◽

Seismic Data ◽

Semantic Segmentation ◽

Attenuation Effect ◽

Attenuation Model ◽

Reflection Seismic ◽

Q Model ◽

Segmentation Task

<p>The seismic waves exhibit various types of attenuation while propagating through the subsurface, which is strongly related to the complexity of the earth. Anelasticity of the subsurface medium, which is quantified by the quality factor Q, causes dissipation of seismic energy. Attenuation distorts the phase of the seismic data and decays the higher frequencies in the data more than lower frequencies. Strong attenuation effect resulting from geology such as gas pocket is a notoriously challenging problem for high resolution imaging because it strongly reduces the amplitude and downgrade the imaging quality of deeper events. To compensate this attenuation effect, first we need to accurately estimate the attenuation model (Q). However, it is challenging to directly derive a laterally and vertically varying attenuation model in depth domain from the surface reflection seismic data. This research paper proposes a method to derive the anomalous Q model corresponding to strong attenuative media from marine reflection seismic data using a deep-learning approach, the convolutional neural network (CNN). We treat Q anomaly detection problem as a semantic segmentation task and train an encoder-decoder CNN (U-Net) to perform a pixel-by-pixel prediction on the seismic section to invert a pixel group belongs to different level of attenuation probability which can help to build up the attenuation model. The proposed method in this paper uses a volume of marine 3D reflection seismic data for network training and validation, which needs only a very small amount of data as the training set due to the feature of U-Net, a specific encoder-decoder CNN architecture in semantic segmentation task. Finally, in order to evaluate the attenuation model result predicted by the proposed method, we validate the predicted heterogeneous Q model using de-absorption pre-stack depth migration (Q-PSDM), a high-resolution depth imaging result with reasonable compensation is obtained.</p>

Download Full-text

MICRA-Net: MICRoscopy Analysis Neural Network to solve detection, classification, and segmentation from a single simple auxiliary task

10.1101/2021.06.29.448970 ◽

2021 ◽

Author(s):

Anthony Bilodeau ◽

Constantin V.L. Delmas ◽

Martin Parent ◽

Paul De Koninck ◽

Audrey Durand ◽

...

Keyword(s):

Neural Network ◽

Quantitative Analysis ◽

High Throughput ◽

Semantic Segmentation ◽

Feature Maps ◽

Expert Annotation ◽

Microscopy Analysis ◽

Weakly Supervised ◽

Segmentation Task ◽

Microscopy Images

High throughput quantitative analysis of microscopy images presents a challenge due to the complexity of the image content and the difficulty to retrieve precisely annotated datasets. In this paper we introduce a weakly-supervised MICRoscopy Analysis neural network (MICRA-Net) that can be trained on a simple main classification task using image-level annotations to solve multiple the more complex auxiliary semantic segmentation task and other associated tasks such as detection or enumeration. MICRA-Net relies on the latent information embedded within a trained model to achieve performances similar to state-of-the-art fully-supervised learning. This learnt information is extracted from the network using gradient class activation maps, which are combined to generate detailed feature maps of the biological structures of interest. We demonstrate how MICRA-Net significantly alleviates the Expert annotation process on various microscopy datasets and can be used for high-throughput quantitative analysis of microscopy images.

Download Full-text

PlaneNet: an efficient local feature extraction network

PeerJ Computer Science ◽

10.7717/peerj-cs.783 ◽

2021 ◽

Vol 7 ◽

pp. e783

Author(s):

Bin Lin ◽

Houcheng Su ◽

Danyang Li ◽

Ao Feng ◽

Hongxiang Li ◽

...

Keyword(s):

Mobile Devices ◽

State Of The Art ◽

Computing Time ◽

Semantic Segmentation ◽

Vital Role ◽

Light Weight ◽

Practical Application ◽

Classification Tasks ◽

Segmentation Task ◽

Local Feature Extraction

Due to memory and computing resources limitations, deploying convolutional neural networks on embedded and mobile devices is challenging. However, the redundant use of the 1 × 1 convolution in traditional light-weight networks, such as MobileNetV1, has increased the computing time. By utilizing the 1 × 1 convolution that plays a vital role in extracting local features more effectively, a new lightweight network, named PlaneNet, is introduced. PlaneNet can improve the accuracy and reduce the numbers of parameters and multiply-accumulate operations (Madds). Our model is evaluated on classification and semantic segmentation tasks. In the classification tasks, the CIFAR-10, Caltech-101, and ImageNet2012 datasets are used. In the semantic segmentation task, PlaneNet is tested on the VOC2012 datasets. The experimental results demonstrate that PlaneNet (74.48%) can obtain higher accuracy than MobileNetV3-Large (73.99%) and GhostNet (72.87%) and achieves state-of-the-art performance with fewer network parameters in both tasks. In addition, compared with the existing models, it has reached the practical application level on mobile devices. The code of PlaneNet on GitHub: https://github.com/LinB203/planenet.

Download Full-text

Boosting Multilabel Semantic Segmentation for Somata and Vessels in Mouse Brain

Frontiers in Neuroscience ◽

10.3389/fnins.2021.610122 ◽

2021 ◽

Vol 15 ◽

Author(s):

Xinglong Wu ◽

Yuhang Tao ◽

Guangzhi He ◽

Dun Liu ◽

Meiling Fan ◽

...

Keyword(s):

Neural Networks ◽

Mouse Brain ◽

Light And Electron Microscopy ◽

Three Dimensional ◽

Image Data ◽

Semantic Segmentation ◽

Poor Quality ◽

Deep Convolutional Neural Networks ◽

Segmentation Task

Deep convolutional neural networks (DCNNs) are widely utilized for the semantic segmentation of dense nerve tissues from light and electron microscopy (EM) image data; the goal of this technique is to achieve efficient and accurate three-dimensional reconstruction of the vasculature and neural networks in the brain. The success of these tasks heavily depends on the amount, and especially the quality, of the human-annotated labels fed into DCNNs. However, it is often difficult to acquire the gold standard of human-annotated labels for dense nerve tissues; human annotations inevitably contain discrepancies or even errors, which substantially impact the performance of DCNNs. Thus, a novel boosting framework consisting of a DCNN for multilabel semantic segmentation with a customized Dice-logarithmic loss function, a fusion module combining the annotated labels and the corresponding predictions from the DCNN, and a boosting algorithm to sequentially update the sample weights during network training iterations was proposed to systematically improve the quality of the annotated labels; this framework eventually resulted in improved segmentation task performance. The microoptical sectioning tomography (MOST) dataset was then employed to assess the effectiveness of the proposed framework. The result indicated that the framework, even trained with a dataset including some poor-quality human-annotated labels, achieved state-of-the-art performance in the segmentation of somata and vessels in the mouse brain. Thus, the proposed technique of artificial intelligence could advance neuroscience research.

Download Full-text

Document-level Relation Extraction as Semantic Segmentation

Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence ◽

10.24963/ijcai.2021/551 ◽

2021 ◽

Author(s):

Ningyu Zhang ◽

Xiang Chen ◽

Xin Xie ◽

Shumin Deng ◽

Chuanqi Tan ◽

...

Keyword(s):

Computer Vision ◽

State Of The Art ◽

Relation Extraction ◽

Semantic Segmentation ◽

Experimental Results ◽

Context Information ◽

Global Information ◽

Benchmark Datasets ◽

Segmentation Task ◽

Document Level

Document-level relation extraction aims to extract relations among multiple entity pairs from a document. Previously proposed graph-based or transformer-based models utilize the entities independently, regardless of global information among relational triples. This paper approaches the problem by predicting an entity-level relation matrix to capture local and global information, parallel to the semantic segmentation task in computer vision. Herein, we propose a Document U-shaped Network for document-level relation extraction. Specifically, we leverage an encoder module to capture the context information of entities and a U-shaped segmentation module over the image-style feature map to capture global interdependency among triples. Experimental results show that our approach can obtain state-of-the-art performance on three benchmark datasets DocRED, CDR, and GDA.

Download Full-text

Automated Processing of Remote Sensing Imagery Using Deep Semantic Segmentation: A Building Footprint Extraction Case

ISPRS International Journal of Geo-Information ◽

10.3390/ijgi9080486 ◽

2020 ◽

Vol 9 (8) ◽

pp. 486 ◽

Cited By ~ 2

Author(s):

Aleksandar Milosavljević

Keyword(s):

Remote Sensing ◽

Semantic Segmentation ◽

Aerial Image ◽

Empirical Examination ◽

Remote Sensing Imagery ◽

Automated Processing ◽

Building Footprint ◽

Segmentation Task ◽

Segmentation Models ◽

Selection Of

The proliferation of high-resolution remote sensing sensors and platforms imposes the need for effective analyses and automated processing of high volumes of aerial imagery. The recent advance of artificial intelligence (AI) in the form of deep learning (DL) and convolutional neural networks (CNN) showed remarkable results in several image-related tasks, and naturally, gain the focus of the remote sensing community. In this paper, we focus on specifying the processing pipeline that relies on existing state-of-the-art DL segmentation models to automate building footprint extraction. The proposed pipeline is organized in three stages: image preparation, model implementation and training, and predictions fusion. For the first and third stages, we introduced several techniques that leverage remote sensing imagery specifics, while for the selection of the segmentation model, we relied on empirical examination. In the paper, we presented and discussed several experiments that we conducted on Inria Aerial Image Labeling Dataset. Our findings confirmed that automatic processing of remote sensing imagery using DL semantic segmentation is both possible and can provide applicable results. The proposed pipeline can be potentially transferred to any other remote sensing imagery segmentation task if the corresponding dataset is available.

Download Full-text