scholarly journals Robustness Verification of Semantic Segmentation Neural Networks Using Relaxed Reachability

2021 ◽  
pp. 263-286
Author(s):  
Hoang-Dung Tran ◽  
Neelanjana Pal ◽  
Patrick Musau ◽  
Diego Manzanas Lopez ◽  
Nathaniel Hamilton ◽  
...  
Keyword(s):  
Neural Networks ◽  
Robustness Analysis ◽  
Semantic Segmentation ◽  
Network Architectures ◽  
Verification Methods ◽  
Verification Time ◽  
Linear Programming Problems ◽  
Total Analysis ◽  
Evaluation Measure ◽  
Robust Variant

AbstractThis paper introduces robustness verification for semantic segmentation neural networks (in short, semantic segmentation networks [SSNs]), building on and extending recent approaches for robustness verification of image classification neural networks. Despite recent progress in developing verification methods for specifications such as local adversarial robustness in deep neural networks (DNNs) in terms of scalability, precision, and applicability to different network architectures, layers, and activation functions, robustness verification of semantic segmentation has not yet been considered. We address this limitation by developing and applying new robustness analysis methods for several segmentation neural network architectures, specifically by addressing reachability analysis of up-sampling layers, such as transposed convolution and dilated convolution. We consider several definitions of robustness for segmentation, such as the percentage of pixels in the output that can be proven robust under different adversarial perturbations, and a robust variant of intersection-over-union (IoU), the typical performance evaluation measure for segmentation tasks. Our approach is based on a new relaxed reachability method, allowing users to select the percentage of a number of linear programming problems (LPs) to solve when constructing the reachable set, through a relaxation factor percentage. The approach is implemented within NNV, then applied and evaluated on segmentation datasets, such as a multi-digit variant of MNIST known as M2NIST. Thorough experiments show that by using transposed convolution for up-sampling and average-pooling for down-sampling, combined with minimizing the number of ReLU layers in the SSNs, we can obtain SSNs with not only high accuracy (IoU), but also that are more robust to adversarial attacks and amenable to verification. Additionally, using our new relaxed reachability method, we can significantly reduce the verification time for neural networks whose ReLU layers dominate the total analysis time, even in classification tasks.

scholarly journals TGV Upsampling: A Making-Up Operation for Semantic Segmentation

2019 ◽  
Vol 2019 ◽  
pp. 1-12
Author(s):  
Xu Yin ◽  
Yan Li ◽  
Byeong-Seok Shin
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Detailed Analysis ◽  
Semantic Segmentation ◽  
Information Loss ◽  
Network Architectures ◽  
Feature Maps ◽  
Extraction Processes ◽  
Multiple Uses ◽  
Improved Accuracy

With the widespread use of deep learning methods, semantic segmentation has achieved great improvements in recent years. However, many researchers have pointed out that with multiple uses of convolution and pooling operations, great information loss would occur in the extraction processes. To solve this problem, various operations or network architectures have been suggested to make up for the loss of information. We observed a trend in many studies to design a network as a symmetric type, with both parts representing the “encoding” and “decoding” stages. By “upsampling” operations in the “decoding” stage, feature maps are constructed in a certain way that would more or less make up for the losses in previous layers. In this paper, we focus on upsampling operations, make a detailed analysis, and compare current methods used in several famous neural networks. We also combine the knowledge on image restoration and design a new upsampled layer (or operation) named the TGV upsampling algorithm. We successfully replaced upsampling layers in the previous research with our new method. We found that our model can better preserve detailed textures and edges of feature maps and can, on average, achieve 1.4–2.3% improved accuracy compared to the original models.

scholarly journals Neural Graph Embedding for Neural Architecture Search

2020 ◽  
Vol 34 (04) ◽  
pp. 4707-4714 ◽  
Author(s):  
Wei Li ◽  
Shaogang Gong ◽  
Xiatian Zhu
Keyword(s):  
Neural Networks ◽  
Graph Embedding ◽  
Semantic Segmentation ◽  
Directed Acyclic Graphs ◽  
Network Architectures ◽  
Search Performance ◽  
Convolutional Network ◽  
Neural Architecture ◽  
Acyclic Graphs ◽  
Topology Information

Existing neural architecture search (NAS) methods often operate in discrete or continuous spaces directly, which ignores the graphical topology knowledge of neural networks. This leads to suboptimal search performance and efficiency, given the factor that neural networks are essentially directed acyclic graphs (DAG). In this work, we address this limitation by introducing a novel idea of neural graph embedding (NGE). Specifically, we represent the building block (i.e. the cell) of neural networks with a neural DAG, and learn it by leveraging a Graph Convolutional Network to propagate and model the intrinsic topology information of network architectures. This results in a generic neural network representation integrable with different existing NAS frameworks. Extensive experiments show the superiority of NGE over the state-of-the-art methods on image classification and semantic segmentation.

scholarly journals DropConnect is effective in modeling uncertainty of Bayesian deep networks

2021 ◽  
Vol 11 (1) ◽  
Author(s):  
Aryan Mobiny ◽  
Pengyu Yuan ◽  
Supratik K. Moulik ◽  
Naveen Garg ◽  
Carol C. Wu ◽  
...  
Keyword(s):  
Neural Networks ◽  
Prediction Accuracy ◽  
State Of The Art ◽  
Computational Cost ◽  
Semantic Segmentation ◽  
Autonomous Driving ◽  
Learning Technology ◽  
Network Architectures ◽  
Uncertainty Estimates ◽  
Deep Networks

AbstractDeep neural networks (DNNs) have achieved state-of-the-art performance in many important domains, including medical diagnosis, security, and autonomous driving. In domains where safety is highly critical, an erroneous decision can result in serious consequences. While a perfect prediction accuracy is not always achievable, recent work on Bayesian deep networks shows that it is possible to know when DNNs are more likely to make mistakes. Knowing what DNNs do not know is desirable to increase the safety of deep learning technology in sensitive applications; Bayesian neural networks attempt to address this challenge. Traditional approaches are computationally intractable and do not scale well to large, complex neural network architectures. In this paper, we develop a theoretical framework to approximate Bayesian inference for DNNs by imposing a Bernoulli distribution on the model weights. This method called Monte Carlo DropConnect (MC-DropConnect) gives us a tool to represent the model uncertainty with little change in the overall model structure or computational cost. We extensively validate the proposed algorithm on multiple network architectures and datasets for classification and semantic segmentation tasks. We also propose new metrics to quantify uncertainty estimates. This enables an objective comparison between MC-DropConnect and prior approaches. Our empirical results demonstrate that the proposed framework yields significant improvement in both prediction accuracy and uncertainty estimation quality compared to the state of the art.

Color Space Transformation using Neural Networks

2019 ◽  
Vol 2019 (1) ◽  
pp. 153-158
Author(s):  
Lindsay MacDonald
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Color Space ◽  
Reflectance Spectra ◽  
Network Architectures ◽  
Color Spaces ◽  
Natural Materials ◽  
Space Transformation ◽  
Color Space Transformation

We investigated how well a multilayer neural network could implement the mapping between two trichromatic color spaces, specifically from camera R,G,B to tristimulus X,Y,Z. For training the network, a set of 800,000 synthetic reflectance spectra was generated. For testing the network, a set of 8,714 real reflectance spectra was collated from instrumental measurements on textiles, paints and natural materials. Various network architectures were tested, with both linear and sigmoidal activations. Results show that over 85% of all test samples had color errors of less than 1.0 ΔE2000 units, much more accurate than could be achieved by regression.

scholarly journals Convolutional Neural Network for the Semantic Segmentation of Remote Sensing Images

2021 ◽  
Vol 26 (1) ◽  
pp. 200-215
Author(s):  
Muhammad Alam ◽  
Jian-Feng Wang ◽  
Cong Guangpei ◽  
LV Yunrong ◽  
Yuanfang Chen
Keyword(s):  
Neural Network ◽  
Remote Sensing ◽  
Neural Networks ◽  
Image Processing ◽  
Deep Learning ◽  
Semantic Segmentation ◽  
Natural Scene ◽  
Remote Sensing Images ◽  
Advantages And Disadvantages ◽  
Target Segmentation

AbstractIn recent years, the success of deep learning in natural scene image processing boosted its application in the analysis of remote sensing images. In this paper, we applied Convolutional Neural Networks (CNN) on the semantic segmentation of remote sensing images. We improve the Encoder- Decoder CNN structure SegNet with index pooling and U-net to make them suitable for multi-targets semantic segmentation of remote sensing images. The results show that these two models have their own advantages and disadvantages on the segmentation of different objects. In addition, we propose an integrated algorithm that integrates these two models. Experimental results show that the presented integrated algorithm can exploite the advantages of both the models for multi-target segmentation and achieve a better segmentation compared to these two models.

scholarly journals SketchGNN: Semantic Sketch Segmentation with Graph Neural Networks

2021 ◽  
Vol 40 (3) ◽  
pp. 1-13
Author(s):  
Lumin Yang ◽  
Jiajie Zhuang ◽  
Hongbo Fu ◽  
Xiangzhi Wei ◽  
Kun Zhou ◽  
...  
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Network Architecture ◽  
Large Scale ◽  
State Of The Art ◽  
Semantic Segmentation ◽  
Structure Information ◽  
Graph Neural Networks ◽  
Node Labels ◽  
Point Level

We introduce SketchGNN , a convolutional graph neural network for semantic segmentation and labeling of freehand vector sketches. We treat an input stroke-based sketch as a graph with nodes representing the sampled points along input strokes and edges encoding the stroke structure information. To predict the per-node labels, our SketchGNN uses graph convolution and a static-dynamic branching network architecture to extract the features at three levels, i.e., point-level, stroke-level, and sketch-level. SketchGNN significantly improves the accuracy of the state-of-the-art methods for semantic sketch segmentation (by 11.2% in the pixel-based metric and 18.2% in the component-based metric over a large-scale challenging SPG dataset) and has magnitudes fewer parameters than both image-based and sequence-based methods.

scholarly journals Semantic Segmentation of Cerebellum in 2D Fetal Ultrasound Brain Images using Convolutional Neural Networks

IEEE Access ◽  
2021 ◽  
pp. 1-1
Author(s):  
Vishal Singh ◽  
Pradeeba Sridar ◽  
Jinman Kim ◽  
Ralph Nanan ◽  
N. Poornima ◽  
...  
Keyword(s):  
Neural Networks ◽  
Convolutional Neural Networks ◽  
Semantic Segmentation ◽  
Fetal Ultrasound ◽  
Brain Images

scholarly journals Convolutional Neural Networks for Semantic Segmentation as a Tool for Multiclass Face Analysis in Thermal Infrared

2021 ◽  
Vol 40 (1) ◽  
Author(s):  
David Müller ◽  
Andreas Ehlen ◽  
Bernd Valeske
Keyword(s):  
Neural Networks ◽  
Convolutional Neural Networks ◽  
Semantic Segmentation ◽  
Thermal Infrared ◽  
Relative Temperature ◽  
Face Analysis ◽  
Image Translation ◽  
The Face ◽  
Live Image ◽  
Quantitative Thermography

AbstractConvolutional neural networks were used for multiclass segmentation in thermal infrared face analysis. The principle is based on existing image-to-image translation approaches, where each pixel in an image is assigned to a class label. We show that established networks architectures can be trained for the task of multiclass face analysis in thermal infrared. Created class annotations consisted of pixel-accurate locations of different face classes. Subsequently, the trained network can segment an acquired unknown infrared face image into the defined classes. Furthermore, face classification in live image acquisition is shown, in order to be able to display the relative temperature in real-time from the learned areas. This allows a pixel-accurate temperature face analysis e.g. for infection detection like Covid-19. At the same time our approach offers the advantage of concentrating on the relevant areas of the face. Areas of the face irrelevant for the relative temperature calculation or accessories such as glasses, masks and jewelry are not considered. A custom database was created to train the network. The results were quantitatively evaluated with the intersection over union (IoU) metric. The methodology shown can be transferred to similar problems for more quantitative thermography tasks like in materials characterization or quality control in production.

Sign in / Sign up

Export Citation Format

Share Document