scholarly journals Depth Estimation and Semantic Segmentation from a Single RGB Image Using a Hybrid Convolutional Neural Network

Sensors ◽  
2019 ◽  
Vol 19 (8) ◽  
pp. 1795 ◽  
Author(s):  
Xiao Lin ◽  
Dalila Sánchez-Escobedo ◽  
Josep R. Casas ◽  
Montse Pardàs
Keyword(s):  
Neural Network ◽  
Convolutional Neural Network ◽  
State Of The Art ◽  
Semantic Segmentation ◽  
Depth Estimation ◽  
Input Image ◽  
Estimation Accuracy ◽  
Single Task ◽  
Qualitative And Quantitative ◽  
Highly Correlated

Semantic segmentation and depth estimation are two important tasks in computer vision, and many methods have been developed to tackle them. Commonly these two tasks are addressed independently, but recently the idea of merging these two problems into a sole framework has been studied under the assumption that integrating two highly correlated tasks may benefit each other to improve the estimation accuracy. In this paper, depth estimation and semantic segmentation are jointly addressed using a single RGB input image under a unified convolutional neural network. We analyze two different architectures to evaluate which features are more relevant when shared by the two tasks and which features should be kept separated to achieve a mutual improvement. Likewise, our approaches are evaluated under two different scenarios designed to review our results versus single-task and multi-task methods. Qualitative and quantitative experiments demonstrate that the performance of our methodology outperforms the state of the art on single-task approaches, while obtaining competitive results compared with other multi-task methods.

scholarly journals wUUNet: Advanced Fully Convolutional Neural Network for Multiclass Fire Segmentation

Symmetry ◽  
2021 ◽  
Vol 13 (1) ◽  
pp. 98
Author(s):  
Vladimir Sergeevich Bochkov ◽  
Liliya Yurievna Kataeva
Keyword(s):  
Neural Network ◽  
Convolutional Neural Network ◽  
Input Data ◽  
Final Section ◽  
State Of The Art ◽  
Sliding Window ◽  
Input Image ◽  
Gaussian Mixtures ◽  
Flame Suppression ◽  
Segmentation Task

This article describes an AI-based solution to multiclass fire segmentation. The flame contours are divided into red, yellow, and orange areas. This separation is necessary to identify the hottest regions for flame suppression. Flame objects can have a wide variety of shapes (convex and non-convex). In that case, the segmentation task is more applicable than object detection because the center of the fire is much more accurate and reliable information than the center of the bounding box and, therefore, can be used by robotics systems for aiming. The UNet model is used as a baseline for the initial solution because it is the best open-source convolutional neural network. There is no available open dataset for multiclass fire segmentation. Hence, a custom dataset was developed and used in the current study, including 6250 samples from 36 videos. We compared the trained UNet models with several configurations of input data. The first comparison is shown between the calculation schemes of fitting the frame to one window and obtaining non-intersected areas of sliding window over the input image. Secondarily, we chose the best main metric of the loss function (soft Dice and Jaccard). We addressed the problem of detecting flame regions at the boundaries of non-intersected regions, and introduced new combinational methods of obtaining output signal based on weighted summarization and Gaussian mixtures of half-intersected areas as a solution. In the final section, we present UUNet-concatenative and wUUNet models that demonstrate significant improvements in accuracy and are considered to be state-of-the-art. All models use the original UNet-backbone at the encoder layers (i.e., VGG16) to demonstrate the superiority of the proposed architectures. The results can be applied to many robotic firefighting systems.

scholarly journals HIGH QUALITY FACADE SEGMENTATION BASED ON STRUCTURED RANDOM FOREST, REGION PROPOSAL NETWORK AND RECTANGULAR FITTING

2018 ◽  
Vol IV-2 ◽  
pp. 223-230 ◽  
Author(s):  
K. Rahmani ◽  
H. Mayer
Keyword(s):  
Neural Network ◽  
Random Forest ◽  
Convolutional Neural Network ◽  
State Of The Art ◽  
Semantic Segmentation ◽  
High Quality ◽  
Current State ◽  
Forest Region ◽  
Building Facades ◽  
Art Methods

In this paper we present a pipeline for high quality semantic segmentation of building facades using Structured Random Forest (SRF), Region Proposal Network (RPN) based on a Convolutional Neural Network (CNN) as well as rectangular fitting optimization. Our main contribution is that we employ features created by the RPN as channels in the SRF.We empirically show that this is very effective especially for doors and windows. Our pipeline is evaluated on two datasets where we outperform current state-of-the-art methods. Additionally, we quantify the contribution of the RPN and the rectangular fitting optimization on the accuracy of the result.

Bag of tricks for fabric defect detection based on Cascade R-CNN

2020 ◽  
pp. 004051752095522
Author(s):  
Feng Li ◽  
Feng Li
Keyword(s):  
Neural Network ◽  
Convolutional Neural Network ◽  
Defect Detection ◽  
State Of The Art ◽  
Detection Algorithm ◽  
Input Image ◽  
Defect Size ◽  
Fabric Defect Detection ◽  
Fabric Defect ◽  
Single Input

In this paper, a bag of tricks is proposed to improve the precision of fabric defect detection. Although the general state-of-the-art convolutional neural network detection algorithm can achieve a better detection effect, in fact, the detection precision still has enough room to improve on fabric defect detection. Therefore, we propose three tricks to further improve the precision. Firstly, we use multiscale training, which scales the single input image into a number of images of different resolutions for training, so as to be able to adapt to the box distribution of different scales. Secondly, we use the dimension clusters method. By observing the distribution of the width and the height of the defect size in the fabric dataset, we find that the distribution of the defect size in the dataset is extremely unbalanced and the size span is large. We believe that the training results of the default prior boxes setting might not be optimal, so we conduct dimensional clustering for the width and height of the defect size of the dataset, so as to make the network model easier to learn. Thirdly, we use soft non-maximum suppression instead of traditional non-maximum suppression to avoid the situation that the same kinds of defect category in the dataset are overlapped and eliminated as repeated detection. With this bag of tricks, we effectively improve the precision of fabric defect detection by 8.9% mAP on the basis of the baseline of state-of-the-art convolutional neural network detection algorithm.

scholarly journals A Modularized Architecture of Multi-Branch Convolutional Neural Network for Image Captioning

Electronics ◽  
2019 ◽  
Vol 8 (12) ◽  
pp. 1417 ◽  
Author(s):  
Shan He ◽  
Yuanyao Lu
Keyword(s):  
Neural Network ◽  
Computer Vision ◽  
Convolutional Neural Network ◽  
Language Processing ◽  
State Of The Art ◽  
Input Image ◽  
Simple Design ◽  
Complete Conversion ◽  
Practical Application ◽  
Image Captioning

Image captioning is a comprehensive task in computer vision (CV) and natural language processing (NLP). It can complete conversion from image to text, that is, the algorithm automatically generates corresponding descriptive text according to the input image. In this paper, we present an end-to-end model that takes deep convolutional neural network (CNN) as the encoder and recurrent neural network (RNN) as the decoder. In order to get better image captioning extraction, we propose a highly modularized multi-branch CNN, which could increase accuracy while maintaining the number of hyper-parameters unchanged. This strategy provides a simply designed network consists of parallel sub-modules of the same structure. While traditional CNN goes deeper and wider to increase accuracy, our proposed method is more effective with a simple design, which is easier to optimize for practical application. Experiments are conducted on Flickr8k, Flickr30k and MSCOCO entities. Results demonstrate that our method achieves state of the art performances in terms of caption quality.

scholarly journals DTS-Net: Depth-to-Space Networks for Fast and Accurate Semantic Object Segmentation

Sensors ◽  
2022 ◽  
Vol 22 (1) ◽  
pp. 337
Author(s):  
Hatem Ibrahem ◽  
Ahmed Salem ◽  
Hyun-Soo Kang
Keyword(s):  
Neural Network ◽  
Convolutional Neural Network ◽  
Super Resolution ◽  
Semantic Segmentation ◽  
Depth Estimation ◽  
Network Architectures ◽  
Estimation Task ◽  
Space Networks ◽  
Semantic Object ◽  
Filtration Technique

We propose Depth-to-Space Net (DTS-Net), an effective technique for semantic segmentation using the efficient sub-pixel convolutional neural network. This technique is inspired by depth-to-space (DTS) image reconstruction, which was originally used for image and video super-resolution tasks, combined with a mask enhancement filtration technique based on multi-label classification, namely, Nearest Label Filtration. In the proposed technique, we employ depth-wise separable convolution-based architectures. We propose both a deep network, that is, DTS-Net, and a lightweight network, DTS-Net-Lite, for real-time semantic segmentation; these networks employ Xception and MobileNetV2 architectures as the feature extractors, respectively. In addition, we explore the joint semantic segmentation and depth estimation task and demonstrate that the proposed technique can efficiently perform both tasks simultaneously, outperforming state-of-art (SOTA) methods. We train and evaluate the performance of the proposed method on the PASCAL VOC2012, NYUV2, and CITYSCAPES benchmarks. Hence, we obtain high mean intersection over union (mIOU) and mean pixel accuracy (Pix.acc.) values using simple and lightweight convolutional neural network architectures of the developed networks. Notably, the proposed method outperforms SOTA methods that depend on encoder–decoder architectures, although our implementation and computations are far simpler.

scholarly journals CoCoX: Generating Conceptual and Counterfactual Explanations via Fault-Lines

2020 ◽  
Vol 34 (03) ◽  
pp. 2594-2601
Author(s):  
Arjun Akula ◽  
Shuai Wang ◽  
Song-Chun Zhu
Keyword(s):  
Neural Network ◽  
State Of The Art ◽  
Input Image ◽  
Classification Model ◽  
Learning Models ◽  
Fault Line ◽  
Semantic Level ◽  
Explainable Ai ◽  
Fault Lines ◽  
Classification Category

We present CoCoX (short for Conceptual and Counterfactual Explanations), a model for explaining decisions made by a deep convolutional neural network (CNN). In Cognitive Psychology, the factors (or semantic-level features) that humans zoom in on when they imagine an alternative to a model prediction are often referred to as fault-lines. Motivated by this, our CoCoX model explains decisions made by a CNN using fault-lines. Specifically, given an input image I for which a CNN classification model M predicts class cpred, our fault-line based explanation identifies the minimal semantic-level features (e.g., stripes on zebra, pointed ears of dog), referred to as explainable concepts, that need to be added to or deleted from I in order to alter the classification category of I by M to another specified class calt. We argue that, due to the conceptual and counterfactual nature of fault-lines, our CoCoX explanations are practical and more natural for both expert and non-expert users to understand the internal workings of complex deep learning models. Extensive quantitative and qualitative experiments verify our hypotheses, showing that CoCoX significantly outperforms the state-of-the-art explainable AI models. Our implementation is available at https://github.com/arjunakula/CoCoX

scholarly journals SketchGNN: Semantic Sketch Segmentation with Graph Neural Networks

2021 ◽  
Vol 40 (3) ◽  
pp. 1-13
Author(s):  
Lumin Yang ◽  
Jiajie Zhuang ◽  
Hongbo Fu ◽  
Xiangzhi Wei ◽  
Kun Zhou ◽  
...  
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Network Architecture ◽  
Large Scale ◽  
State Of The Art ◽  
Semantic Segmentation ◽  
Structure Information ◽  
Graph Neural Networks ◽  
Node Labels ◽  
Point Level

We introduce SketchGNN , a convolutional graph neural network for semantic segmentation and labeling of freehand vector sketches. We treat an input stroke-based sketch as a graph with nodes representing the sampled points along input strokes and edges encoding the stroke structure information. To predict the per-node labels, our SketchGNN uses graph convolution and a static-dynamic branching network architecture to extract the features at three levels, i.e., point-level, stroke-level, and sketch-level. SketchGNN significantly improves the accuracy of the state-of-the-art methods for semantic sketch segmentation (by 11.2% in the pixel-based metric and 18.2% in the component-based metric over a large-scale challenging SPG dataset) and has magnitudes fewer parameters than both image-based and sequence-based methods.

Sparse-FCM and Deep Convolutional Neural Network for the segmentation and classification of acute lymphoblastic leukaemia

2020 ◽  
Vol 65 (6) ◽  
pp. 759-773
Author(s):  
Segu Praveena ◽  
Sohan Pal Singh
Keyword(s):  
Neural Network ◽  
Acute Lymphoblastic Leukaemia ◽  
Convolutional Neural Network ◽  
Optimization Algorithm ◽  
Lymphoblastic Leukaemia ◽  
Input Image ◽  
Deep Convolutional Neural Network ◽  
Grey Wolf ◽  
Deep Cnn

AbstractLeukaemia detection and diagnosis in advance is the trending topic in the medical applications for reducing the death toll of patients with acute lymphoblastic leukaemia (ALL). For the detection of ALL, it is essential to analyse the white blood cells (WBCs) for which the blood smear images are employed. This paper proposes a new technique for the segmentation and classification of the acute lymphoblastic leukaemia. The proposed method of automatic leukaemia detection is based on the Deep Convolutional Neural Network (Deep CNN) that is trained using an optimization algorithm, named Grey wolf-based Jaya Optimization Algorithm (GreyJOA), which is developed using the Grey Wolf Optimizer (GWO) and Jaya Optimization Algorithm (JOA) that improves the global convergence. Initially, the input image is applied to pre-processing and the segmentation is performed using the Sparse Fuzzy C-Means (Sparse FCM) clustering algorithm. Then, the features, such as Local Directional Patterns (LDP) and colour histogram-based features, are extracted from the segments of the pre-processed input image. Finally, the extracted features are applied to the Deep CNN for the classification. The experimentation evaluation of the method using the images of the ALL IDB2 database reveals that the proposed method acquired a maximal accuracy, sensitivity, and specificity of 0.9350, 0.9528, and 0.9389, respectively.

scholarly journals PDANet: Self-Supervised Monocular Depth Estimation Using Perceptual and Data Augmentation Consistency

2021 ◽  
Vol 11 (12) ◽  
pp. 5383
Author(s):  
Huachen Gao ◽  
Xiaoyu Liu ◽  
Meixia Qu ◽  
Shijie Huang
Keyword(s):  
Data Augmentation ◽  
State Of The Art ◽  
Depth Estimation ◽  
Input Image ◽  
Depth Information ◽  
Disparity Map ◽  
Estimation Model ◽  
Absolute Relative Error ◽  
Texture Region ◽  
Monocular Depth

In recent studies, self-supervised learning methods have been explored for monocular depth estimation. They minimize the reconstruction loss of images instead of depth information as a supervised signal. However, existing methods usually assume that the corresponding points in different views should have the same color, which leads to unreliable unsupervised signals and ultimately damages the reconstruction loss during the training. Meanwhile, in the low texture region, it is unable to predict the disparity value of pixels correctly because of the small number of extracted features. To solve the above issues, we propose a network—PDANet—that integrates perceptual consistency and data augmentation consistency, which are more reliable unsupervised signals, into a regular unsupervised depth estimation model. Specifically, we apply a reliable data augmentation mechanism to minimize the loss of the disparity map generated by the original image and the augmented image, respectively, which will enhance the robustness of the image in the prediction of color fluctuation. At the same time, we aggregate the features of different layers extracted by a pre-trained VGG16 network to explore the higher-level perceptual differences between the input image and the generated one. Ablation studies demonstrate the effectiveness of each components, and PDANet shows high-quality depth estimation results on the KITTI benchmark, which optimizes the state-of-the-art method from 0.114 to 0.084, measured by absolute relative error for depth estimation.

Sign in / Sign up

Export Citation Format

Share Document