Steel bridge corrosion inspection with combined vision and thermographic images

2021 ◽  
pp. 147592172198940
Author(s):  
Hyung Jin Lim ◽  
Soonkyu Hwang ◽  
Hyeonjin Kim ◽  
Hoon Sohn
Keyword(s):  
Neural Network ◽  
Convolutional Neural Network ◽  
Imaging System ◽  
Three Dimensional ◽  
Steel Bridge ◽  
Two Dimensional ◽  
Infrared Images ◽  
Seamless Integration ◽  
One Dimensional ◽  
Corrosion Detection

In this study, a faster region-based convolutional neural network is constructed and applied to the combined vision and thermographic images for automated detection and classification of surface and subsurface corrosion in steel bridges. First, a hybrid imaging system is developed for the seamless integration of vision and infrared images. Herein, a three-dimensional red/green/blue vision image is obtained with a vision camera, and a one-dimensional active infrared (IR) amplitude image is obtained from the infrared camera for temperature measurements with halogen lamps as the heat source. Subsequently, the three-dimensional red/green/blue vision image is converted to a two-dimensional chroma blue- and red-difference (CbCr) image because the CbCr image is known to be more sensitive to surface corrosion than the red/green/blue image. A combined three-dimensional (CbCr-IR) image is then constructed by fusing the two-dimensional CbCr image and the one-dimensional infrared image. For the automated corrosion detection and classification, a faster region-based convolutional neural network is constructed and trained using the combined three-dimensional CbCr-IR images of surface and subsurface corrosion on steel bridge structures. Finally, the performance of the trained, faster region-based convolutional neural network is evaluated using the images acquired from real bridges and compared with faster region-based convolutional neural networks trained by other vision and IR-based images. The uniqueness of this study is attributed to the (1) corrosion detection reliability improvements based on the fusion of vision and infrared images, (2) automated corrosion detection and classification with a faster region-based convolutional neural network, (3) detection of subsurface corrosion that is not detectable using vision images only, and (4) application to field bridge inspection.

scholarly journals Enlarging smaller images before inputting into convolutional neural network: zero-padding vs. interpolation

2019 ◽  
Vol 6 (1) ◽  
Author(s):  
Mahdi Hashemi
Keyword(s):  
Neural Network ◽  
Convolutional Neural Network ◽  
Feature Vector ◽  
Three Dimensional ◽  
Theoretical Justification ◽  
Training Time ◽  
One Dimensional ◽  
Zero Padding ◽  
Internal Parameters ◽  
Machine Learning Model

AbstractThe input to a machine learning model is a one-dimensional feature vector. However, in recent learning models, such as convolutional and recurrent neural networks, two- and three-dimensional feature tensors can also be inputted to the model. During training, the machine adjusts its internal parameters to project each feature tensor close to its target. After training, the machine can be used to predict the target for previously unseen feature tensors. What this study focuses on is the requirement that feature tensors must be of the same size. In other words, the same number of features must be present for each sample. This creates a barrier in processing images and texts, as they usually have different sizes, and thus different numbers of features. In classifying an image using a convolutional neural network (CNN), the input is a three-dimensional tensor, where the value of each pixel in each channel is one feature. The three-dimensional feature tensor must be the same size for all images. However, images are not usually of the same size and so are not their corresponding feature tensors. Resizing images to the same size without deforming patterns contained therein is a major challenge. This study proposes zero-padding for resizing images to the same size and compares it with the conventional approach of scaling images up (zooming in) using interpolation. Our study showed that zero-padding had no effect on the classification accuracy but considerably reduced the training time. The reason is that neighboring zero input units (pixels) will not activate their corresponding convolutional unit in the next layer. Therefore, the synaptic weights on outgoing links from input units do not need to be updated if they contain a zero value. Theoretical justification along with experimental endorsements are provided in this paper.

scholarly journals Classification Modeling Method for Near-Infrared Spectroscopy of Tobacco Based on Multimodal Convolution Neural Networks

2020 ◽  
Vol 2020 ◽  
pp. 1-13 ◽  
Author(s):  
Lei Zhang ◽  
Xiangqian Ding ◽  
Ruichun Hou
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Infrared Spectroscopy ◽  
Convolutional Neural Network ◽  
Near Infrared Spectroscopy ◽  
Near Infrared ◽  
Two Dimensional ◽  
One Dimensional ◽  
Nirs Data ◽  
The One

The origin of tobacco is the most important factor in determining the style characteristics and intrinsic quality of tobacco. There are many applications for the identification of tobacco origin by near-infrared spectroscopy. In order to improve the accuracy of the tobacco origin classification, a near-infrared spectrum (NIRS) identification method based on multimodal convolutional neural networks (CNN) was proposed, taking advantage of the strong feature extraction ability of the CNN. Firstly, the one-dimensional convolutional neural network (1-D CNN) is used to extract and combine the pattern features of one-dimensional NIRS data, and then the extracted features are used for classification. Secondly, the one-dimensional NIRS data are converted into two-dimensional spectral images, and the structure features are extracted from two-dimensional spectral images by the two-dimensional convolutional neural network (2-D CNN) method. The classification is performed by the combination of global and local training features. Finally, the influences of different network structure parameters on model identification performance are studied, and the optimal CNN models are selected and compared. The multimodal NIR-CNN identification models of tobacco origin were established by using NIRS of 5,200 tobacco samples from 10 major tobacco producing provinces in China and 3 foreign countries. The classification accuracy of 1-D CNN and 2-D CNN models was 93.15% and 93.05%, respectively, which was better than the traditional PLS-DA method. The experimental results show that the application of 1-D CNN and 2-D CNN can accurately and reliably distinguish the NIRS data, and it can be developed into a new rapid identification method of tobacco origin, which has an important promotion value.

Three-dimensional convolutional neural network–based underground object classification using three-dimensional ground penetrating radar data

2020 ◽  
Vol 19 (6) ◽  
pp. 1884-1893
Author(s):  
Shekhroz Khudoyarov ◽  
Namgyu Kim ◽  
Jong-Jae Lee
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Convolutional Neural Network ◽  
Ground Penetrating Radar ◽  
Three Dimensional ◽  
Object Classification ◽  
Radar Data ◽  
Two Dimensional ◽  
Underground Cavities ◽  
Ground Penetrating

Ground-penetrating radar is a typical sensor system for analyzing underground facilities such as pipelines and rebars. The technique also can be used to detect an underground cavity, which is a potential sign of urban sinkholes. Multichannel ground-penetrating radar devices are widely used to detect underground cavities thanks to the capacity of informative three-dimensional data. Nevertheless, the three-dimensional ground-penetrating radar data interpretation is unclear and complicated when recognizing underground cavities because similar ground-penetrating radar data reflected from different underground objects are often mixed with the cavities. As it is prevalently known that the deep learning algorithm-based techniques are powerful at image classification, deep learning-based techniques for underground object detection techniques using two-dimensional GPR (ground-penetrating radar) radargrams have been researched upon in recent years. However, spatial information of underground objects can be characterized better in three-dimensional ground-penetrating radar voxel data than in two-dimensional ground-penetrating radar images. Therefore, in this study, a novel underground object classification technique is proposed by applying deep three-dimensional convolutional neural network on three-dimensional ground-penetrating radar data. First, a deep convolutional neural network architecture was developed using three-dimensional convolutional networks for recognizing spatial underground objects such as, pipe, cavity, manhole, and subsoil. The framework of applying the three-dimensional convolutional neural network into three-dimensional ground-penetrating radar data was then proposed and experimentally validated using real three-dimensional ground-penetrating radar data. In order to do that, three-dimensional ground-penetrating radar block data were used to train the developed three-dimensional convolutional neural network and to classify unclassified three-dimensional ground-penetrating radar data collected from urban roads in Seoul, South Korea. The validation results revealed that four underground objects (pipe, cavity, manhole, and subsoil) are successfully classified, and the average classification accuracy was 97%. In addition, a false alarm was rarely indicated.

Real-time fault diagnosis using deep fusion of features extracted by parallel long short-term memory with peephole and convolutional neural network

2020 ◽  
pp. 095965182094829
Author(s):  
Funa Zhou ◽  
Zhiqiang Zhang ◽  
Danmin Chen
Keyword(s):  
Neural Network ◽  
Fault Diagnosis ◽  
Convolutional Neural Network ◽  
Real Time ◽  
Short Term Memory ◽  
Two Dimensional ◽  
Short Term ◽  
Term Memory ◽  
One Dimensional ◽  
Long Short Term Memory

Analysis of one-dimensional vibration signals is the most common method used for safety analysis and health monitoring of rotary machines. How to effectively extract features involved in one-dimensional sequence data is crucial for the accuracy of real-time fault diagnosis. This article aims to develop more effective means of extracting useful features potentially involved in one-dimensional vibration signals. First, an improved parallel long short-term memory called parallel long short-term memory with peephole is designed by adding a peephole connection before each forget gate to prevent useless information transferring in the cell. It can not only solve the memory bottleneck problem of traditional long short-term memory for long sequence but also can make full use of all possible information helpful for feature extraction. Second, a fusion network with new training mechanism is designed to fuse features extracted from parallel long short-term memory with peephole and convolutional neural network, respectively. The fusion network can incorporate two-dimensional screenshot image into comprehensive feature extraction. It can provide more accurate fault diagnosis result since two-dimensional screenshot image is another form of expression for one-dimensional vibration sequence involving additional trend and locality information. Finally, real-time two-dimensional screenshot image is fed into convolutional neural network to secure a real-time online diagnosis which is the primary requirement of the engineers in health monitoring. Validity of the proposed method is verified by fault diagnosis for rolling bearing and gearbox.

Sign in / Sign up

Export Citation Format

Share Document