Simultaneous pixel-level concrete defect detection and grouping using a fully convolutional model

2021 ◽  
pp. 147592172098543
Author(s):  
Chaobo Zhang ◽  
Chih-chen Chang ◽  
Maziar Jamshidi
Keyword(s):  
Deep Learning ◽  
Defect Detection ◽  
Visual Inspection ◽  
Surface Defects ◽  
Semantic Segmentation ◽  
Input Image ◽  
Image Size ◽  
Bounding Box ◽  
Proposed Model ◽  
Image Pixels

Deep learning techniques have attracted significant attention in the field of visual inspection of civil infrastructure systems recently. Currently, most deep learning-based visual inspection techniques utilize a convolutional neural network to recognize surface defects either by detecting a bounding box of each defect or classifying all pixels on an image without distinguishing between different defect instances. These outputs cannot be directly used for acquiring the geometric properties of each individual defect in an image, thus hindering the development of fully automated structural assessment techniques. In this study, a novel fully convolutional model is proposed for simultaneously detecting and grouping the image pixels for each individual defect on an image. The proposed model integrates an optimized mask subnet with a box-level detection network, where the former outputs a set of position-sensitive score maps for pixel-level defect detection and the latter predicts a bounding box for each defect to group the detected pixels. An image dataset containing three common types of concrete defects, crack, spalling and exposed rebar, is used for training and testing of the model. Results demonstrate that the proposed model is robust to various defect sizes and shapes and can achieve a mask-level mean average precision ( mAP) of 82.4% and a mean intersection over union ( mIoU) of 75.5%, with a processing speed of about 10 FPS at input image size of 576 × 576 when tested on an NVIDIA GeForce GTX 1060 GPU. Its performance is compared with the state-of-the-art instance segmentation network Mask R-CNN and the semantic segmentation network U-Net. The comparative studies show that the proposed model has a distinct defect boundary delineation capability and outperforms the Mask R-CNN and the U-Net in both accuracy and speed.

scholarly journals Visual Saliency Prediction Based on Deep Learning

Information ◽  
2019 ◽  
Vol 10 (8) ◽  
pp. 257 ◽  
Author(s):  
Bashir Ghariba ◽  
Mohamed S. Shehata ◽  
Peter McGuire
Keyword(s):  
Deep Learning ◽  
Saliency Detection ◽  
Visual Saliency ◽  
Semantic Segmentation ◽  
Input Image ◽  
Human Eye ◽  
Proposed Model ◽  
Global Accuracy ◽  
Visual Saliency Detection ◽  
Deep Learning Model

Human eye movement is one of the most important functions for understanding our surroundings. When a human eye processes a scene, it quickly focuses on dominant parts of the scene, commonly known as a visual saliency detection or visual attention prediction. Recently, neural networks have been used to predict visual saliency. This paper proposes a deep learning encoder-decoder architecture, based on a transfer learning technique, to predict visual saliency. In the proposed model, visual features are extracted through convolutional layers from raw images to predict visual saliency. In addition, the proposed model uses the VGG-16 network for semantic segmentation, which uses a pixel classification layer to predict the categorical label for every pixel in an input image. The proposed model is applied to several datasets, including TORONTO, MIT300, MIT1003, and DUT-OMRON, to illustrate its efficiency. The results of the proposed model are quantitatively and qualitatively compared to classic and state-of-the-art deep learning models. Using the proposed deep learning model, a global accuracy of up to 96.22% is achieved for the prediction of visual saliency.

scholarly journals Research on Recognition Technology of Aluminum Profile Surface Defects Based on Deep Learning

2019 ◽  
Author(s):  
Ruofeng Wei ◽  
Yunbo Bi
Keyword(s):  
Deep Learning ◽  
Machine Vision ◽  
Defect Detection ◽  
Visual Inspection ◽  
Surface Defects ◽  
Average Precision ◽  
Saliency Maps ◽  
Aluminum Profile ◽  
Safety And Reliability ◽  
Aluminum Profiles

Aluminum profile surface defects can greatly affect the performance, safety and reliability of products. Traditional human-based visual inspection is low accuracy and time consuming, and machine vision-based methods depend on hand-crafted features which need to be carefully designed and lack robustness. To recognize the multiple types of defects with various size on aluminum profiles, a multiscale defect detection network based on deep learning is proposed. Then, the network is trained and evaluated using aluminum profile surface defects images. Results show 84.6%, 48.5%, 96.9%, 97.9%, 96.9%, 42.5%, 47.2%, 100%, 100%, 43.3% average precision(AP) for the ten defect categories, respectively, with a mean AP of 75.8%, which illustrate the effectiveness of the network in aluminum profile surface defects detection. In addition, saliency maps also show the feasibility of the proposed network.

scholarly journals Research on Recognition Technology of Aluminum Profile Surface Defects Based on Deep Learning

Materials ◽  
2019 ◽  
Vol 12 (10) ◽  
pp. 1681 ◽  
Author(s):  
Ruofeng Wei ◽  
Yunbo Bi
Keyword(s):  
Deep Learning ◽  
Machine Vision ◽  
Defect Detection ◽  
Visual Inspection ◽  
Surface Defects ◽  
Average Precision ◽  
Saliency Maps ◽  
Aluminum Profile ◽  
Safety And Reliability ◽  
Aluminum Profiles

Aluminum profile surface defects can greatly affect the performance, safety, and reliability of products. Traditional human-based visual inspection has low accuracy and is time consuming, and machine vision-based methods depend on hand-crafted features that need to be carefully designed and lack robustness. To recognize the multiple types of defects with various size on aluminum profiles, a multiscale defect-detection network based on deep learning is proposed. Then, the network is trained and evaluated using aluminum profile surface defects images. Results show 84.6%, 48.5%, 96.9%, 97.9%, 96.9%, 42.5%, 47.2%, 100%, 100%, and 43.3% average precision (AP) for the 10 defect categories, respectively, with a mean AP of 75.8%, which illustrate the effectiveness of the network in aluminum profile surface defects detection. In addition, saliency maps also show the feasibility of the proposed network.

scholarly journals Visual Inspection of the Aircraft Surface Using a Teleoperated Reconfigurable Climbing Robot and Enhanced Deep Learning Technique

2019 ◽  
Vol 2019 ◽  
pp. 1-14 ◽  
Author(s):  
Balakrishnan Ramalingam ◽  
Vega-Heredia Manuel ◽  
Mohan Rajesh Elara ◽  
Ayyalusami Vengadesh ◽  
Anirudh Krishna Lakshmanan ◽  
...  
Keyword(s):  
Deep Learning ◽  
Visual Inspection ◽  
Surface Defects ◽  
Learning Algorithm ◽  
Periodic Pattern ◽  
Surface Inspection ◽  
Detection Accuracy ◽  
Climbing Robot ◽  
Aircraft Skin ◽  
Surface Images

Aircraft surface inspection includes detecting surface defects caused by corrosion and cracks and stains from the oil spill, grease, dirt sediments, etc. In the conventional aircraft surface inspection process, human visual inspection is performed which is time-consuming and inefficient whereas robots with onboard vision systems can inspect the aircraft skin safely, quickly, and accurately. This work proposes an aircraft surface defect and stain detection model using a reconfigurable climbing robot and an enhanced deep learning algorithm. A reconfigurable, teleoperated robot, named as “Kiropter,” is designed to capture the aircraft surface images with an onboard RGB camera. An enhanced SSD MobileNet framework is proposed for stain and defect detection from these images. A Self-filtering-based periodic pattern detection filter has been included in the SSD MobileNet deep learning framework to achieve the enhanced detection of the stains and defects on the aircraft skin images. The model has been tested with real aircraft surface images acquired from a Boeing 737 and a compact aircraft’s surface using the teleoperated robot. The experimental results prove that the enhanced SSD MobileNet framework achieves improved detection accuracy of aircraft surface defects and stains as compared to the conventional models.

Image-based Surface Defect Detection Using Deep Learning: A Review

2021 ◽  
pp. 1-23
Author(s):  
Prahar Bhatt ◽  
Rishi K. Malhan ◽  
Pradeep Rajendran ◽  
Brual Shah ◽  
Shantanu Thakar ◽  
...  
Keyword(s):  
Deep Learning ◽  
Defect Detection ◽  
Surface Defects ◽  
Future Research ◽  
Specific Class ◽  
Survey Paper ◽  
Context Learning ◽  
Surface Defect Detection ◽  
Manufacturing Applications ◽  
Processing Techniques

Abstract Automatically detecting surface defects from images is an essential capability in manufacturing applications. Traditional image processing techniques were useful in solving a specific class of problems. However, these techniques were unable to handle noise, variations in lighting conditions, and background with complex textures. Increasingly deep learning is being explored to automate defect detection. This survey paper presents three different ways of classifying various efforts. These are based on defect detection context, learning techniques, and defect localization and classification method. The existing literature is classified using this methodology. The paper also identifies future research directions based on the trends in the deep learning area.

scholarly journals Learning to Combine Local and Global Image Information for Contactless Palmprint Recognition

Sensors ◽  
2021 ◽  
Vol 22 (1) ◽  
pp. 73
Author(s):  
Marjan Stoimchev ◽  
Marija Ivanovska ◽  
Vitomir Štruc
Keyword(s):  
Deep Learning ◽  
State Of The Art ◽  
Input Image ◽  
Palmprint Recognition ◽  
Learning Approaches ◽  
Elastic Deformations ◽  
Feature Representations ◽  
Palmar Surface ◽  
Proposed Model ◽  
Visual Artifacts

In the past few years, there has been a leap from traditional palmprint recognition methodologies, which use handcrafted features, to deep-learning approaches that are able to automatically learn feature representations from the input data. However, the information that is extracted from such deep-learning models typically corresponds to the global image appearance, where only the most discriminative cues from the input image are considered. This characteristic is especially problematic when data is acquired in unconstrained settings, as in the case of contactless palmprint recognition systems, where visual artifacts caused by elastic deformations of the palmar surface are typically present in spatially local parts of the captured images. In this study we address the problem of elastic deformations by introducing a new approach to contactless palmprint recognition based on a novel CNN model, designed as a two-path architecture, where one path processes the input in a holistic manner, while the second path extracts local information from smaller image patches sampled from the input image. As elastic deformations can be assumed to most significantly affect the global appearance, while having a lesser impact on spatially local image areas, the local processing path addresses the issues related to elastic deformations thereby supplementing the information from the global processing path. The model is trained with a learning objective that combines the Additive Angular Margin (ArcFace) Loss and the well-known center loss. By using the proposed model design, the discriminative power of the learned image representation is significantly enhanced compared to standard holistic models, which, as we show in the experimental section, leads to state-of-the-art performance for contactless palmprint recognition. Our approach is tested on two publicly available contactless palmprint datasets—namely, IITD and CASIA—and is demonstrated to perform favorably against state-of-the-art methods from the literature. The source code for the proposed model is made publicly available.

scholarly journals Drain Structural Defect Detection and Mapping Using AI-Enabled Reconfigurable Robot Raptor and IoRT Framework

Sensors ◽  
2021 ◽  
Vol 21 (21) ◽  
pp. 7287
Author(s):  
Povendhan Palanisamy ◽  
Rajesh Elara Mohan ◽  
Archana Semwal ◽  
Lee Ming Jun Melivin ◽  
Braulio Félix Félix Gómez ◽  
...  
Keyword(s):  
Deep Learning ◽  
Defect Detection ◽  
Structural Defect ◽  
Visual Inspection ◽  
Field Trials ◽  
Detection Task ◽  
Robot Assisted ◽  
Inspection Robot ◽  
Reconfigurable Robot ◽  
Learning Frameworks

Human visual inspection of drains is laborious, time-consuming, and prone to accidents. This work presents an AI-enabled robot-assisted remote drain inspection and mapping framework using our in-house developed reconfigurable robot Raptor. The four-layer IoRT serves as a bridge between the users and the robots, through which seamless information sharing takes place. The Faster RCNN ResNet50, Faster RCNN ResNet101, and Faster RCNN Inception-ResNet-v2 deep learning frameworks were trained using a transfer learning scheme with six typical concrete defect classes and deployed in an IoRT framework remote defect detection task. The efficiency of the trained CNN algorithm and drain inspection robot Raptor was evaluated through various real-time drain inspection field trials using the SLAM technique. The experimental results indicate that robot’s maneuverability was stable, and its mapping and localization were also accurate in different drain types. Finally, for effective drain maintenance, the SLAM-based defect map was generated by fusing defect detection results in the lidar-SLAM map.

scholarly journals Robust Cherry Tomatoes Detection Algorithm in Greenhouse Scene Based on SSD

Agriculture ◽  
2020 ◽  
Vol 10 (5) ◽  
pp. 160
Author(s):  
Ting Yuan ◽  
Lin Lv ◽  
Fan Zhang ◽  
Jun Fu ◽  
Jin Gao ◽  
...  
Keyword(s):  
Deep Learning ◽  
Detection Algorithm ◽  
Input Image ◽  
Detection Methods ◽  
Single Shot ◽  
Image Size ◽  
Growth Difference ◽  
Input Size ◽  
Base Network ◽  
Cherry Tomatoes

The detection of cherry tomatoes in greenhouse scene is of great significance for robotic harvesting. This paper states a method based on deep learning for cherry tomatoes detection to reduce the influence of illumination, growth difference, and occlusion. In view of such greenhouse operating environment and accuracy of deep learning, Single Shot multi-box Detector (SSD) was selected because of its excellent anti-interference ability and self-taught from datasets. The first step is to build datasets containing various conditions in greenhouse. According to the characteristics of cherry tomatoes, the image samples with illumination change, images rotation and noise enhancement were used to expand the datasets. Then training datasets were used to train and construct network model. To study the effect of base network and the input size of networks, one contrast experiment was designed on different base networks of VGG16, MobileNet, Inception V2 networks, and the other contrast experiment was conducted on changing the network input image size of 300 pixels by 300 pixels, 512 pixels by 512 pixels. Through the analysis of the experimental results, it is found that the Inception V2 network is the best base network with the average precision of 98.85% in greenhouse environment. Compared with other detection methods, this method shows substantial improvement in cherry tomatoes detection.

Automatic image detection of multi-type surface defects on wind turbine blades based on cascade deep learning network

2021 ◽  
Vol 25 (2) ◽  
pp. 463-482
Author(s):  
Yulin Mao ◽  
Shuangxin Wang ◽  
Dingli Yu ◽  
Juchao Zhao
Keyword(s):  
Deep Learning ◽  
Wind Turbine ◽  
Defect Detection ◽  
Surface Defects ◽  
Oil Pollution ◽  
Turbine Blades ◽  
Critical Factor ◽  
Wind Turbine Blades ◽  
Surface Defect Detection ◽  
Deep Learning Network

A safe operation protocol of the wind blades is a critical factor to ensure the stability of a wind turbine. Sensors are most commonly applied for defect detection on wind turbine blades (WTBs). However, due to the high cost and the sensitivity to stochastic noise, computer vision-guided automatic detection remains a challenge for surface defect detection on WTBs in particularly, its accuracy in locating defects is yet to be optimized. In this paper, we developed a visual inspection model that can automatically and precisely classify and locate the surface defects, through the utilization of a deep learning framework based on the Cascade R-CNN. In order to obtain high mean average precision (mAP) according to the characteristics of the dataset, a model named Contextual Aligned-Deformable Cascade R-CNN (CAD Cascade R-CNN) using improved strategies of transfer learning, Deformable Convolution and Deformable RoI Align, as well as context information fusion is proposed and a dataset with surface defects categorized and labeled as crack, breakage and oil pollution is generated. Moreover to alleviate the problem of false detection under a complex background, an improved bisecting k-means is presented during the test process. The adaptability and generalization of the proposed CAD Cascade R-CNN model were validated by each type of defects in dataset and different IoU thresholds, whereas, each of the above improved strategies was verified by gradual ablation experiments. Finally experiments that compared with the baseline Cascade R-CNN, Faster R-CNN and YOLO-v3 demonstrate its superiority over these existing approaches with a maximum of 92.1% mAP.

scholarly journals A Generic Automated Surface Defect Detection Based on a Bilinear Model

2019 ◽  
Vol 9 (15) ◽  
pp. 3159 ◽  
Author(s):  
Fei Zhou ◽  
Guihua Liu ◽  
Feng Xu ◽  
Hao Deng
Keyword(s):  
Defect Detection ◽  
Surface Defect ◽  
Surface Defects ◽  
Automatic Classification ◽  
Input Image ◽  
Small Sample ◽  
Bilinear Model ◽  
Heat Map ◽  
Surface Defect Detection ◽  
Weakly Supervised

Aiming at the problems of complex texture, variable interference factors and large sample acquisition in surface defect detection, a generic method of automated surface defect detection based on a bilinear model was proposed. To realize the automatic classification and localization of surface defects, a new Double-Visual Geometry Group16 (D-VGG16) is firstly designed as feature functions of the bilinear model. The global and local features fully extracted from the bilinear model by D-VGG16 are output to the soft-max function to realize the automatic classification of surface defects. Then the heat map of the original image is obtained by applying Gradient-weighted Class Activation Mapping (Grad-CAM) to the output features of D-VGG16. Finally, the defects in the original input image can be located automatically after processing the heat map with a threshold segmentation method. The training process of the proposed method is characterized by a small sample, end-to-end, and is weakly-supervised. Furthermore, experiments are performed on two public and two industrial datasets, which have different defective features in texture, shape and color. The results show that the proposed method can simultaneously realize the classification and localization of defects with different defective features. The average precision of the proposed method is above 99% on the four datasets, and is higher than the known latest algorithms.

Sign in / Sign up

Export Citation Format

Share Document