BLNN: Multiscale Feature Fusion-Based Bilinear Fine-Grained Convolutional Neural Network for Image Classification of Wood Knot Defects

Wood defects are quickly identified from an optical image based on deep learning methodology, which effectively improves the wood utilization. The traditional neural network technique is unemployed for the wood defect detection of optical image used, which results from a long training time, low recognition accuracy, and nonautomatic extraction of defect image features. In this paper, a wood knot defect detection model (so-called BLNN) combined deep learning is reported. Two subnetworks composed of convolutional neural networks are trained by Pytorch. By using the feature extraction capabilities of the two subnetworks and combining the bilinear join operation, the fine-grained features of the image are obtained. The experimental results show that the accuracy has reached up 99.20%, and the training time is obviously reduced with the speed of defect detection about 0.0795 s/image. It indicates that BLNN has the ability to improve the accuracy of defect recognition and has a potential application in the detection of wood knot defects.

Download Full-text

A Novel Deep Convolutional Neural Network Based on ResNet-18 and Transfer Learning for Detection of Wood Knot Defects

Journal of Sensors ◽

10.1155/2021/4428964 ◽

2021 ◽

Vol 2021 ◽

pp. 1-16

Author(s):

Mingyu Gao ◽

Peng Song ◽

Fei Wang ◽

Junyan Liu ◽

Andreas Mandelis ◽

...

Keyword(s):

Neural Network ◽

Deep Learning ◽

Convolutional Neural Network ◽

Transfer Learning ◽

Defect Detection ◽

Image Features ◽

Deep Convolutional Neural Network ◽

Model Parameters ◽

Basic Block ◽

Training Time

Wood defects are quickly identified from an optical image based on deep learning methodology, which effectively improves wood utilization. Traditional neural network techniques have not yet been employed for wood defect detection due to long training time, low recognition accuracy, and nonautomatical extraction of defect image features. In this work, a model (so-called ReSENet-18) for wood knot defect detection that combined deep learning and transfer learning is proposed. The “squeeze-and-excitation” (SE) module is firstly embedded into the “residual basic block” structure for a “SE-Basic-Block” module construction. This model has the advantages of the features that are extracted in the channel dimension, and it is fused in multiscale with original features. Instantaneously, the fully connected layer is replaced with a global average pooling; consequently, the model parameters could be reduced effectively. The experimental results show that the accuracy has reached 99.02%, meanwhile the training time is also reduced. It shows that the proposed deep convolutional neural network based on ReSENet-18 combined with transfer learning can improve the accuracy of defect recognition and has a potential application in the detection of wood knot defects.

Download Full-text

Weld Defect Detection and Image Defect Recognition Using Deep Learning Technology

10.21203/rs.3.rs-149365/v1 ◽

2021 ◽

Author(s):

Yu Cheng ◽

HongGui Deng ◽

YuXin Feng ◽

JunJiang Xiang

Keyword(s):

Deep Learning ◽

Transfer Learning ◽

Defect Detection ◽

Learning Algorithm ◽

Image Features ◽

Economic Losses ◽

Weld Defects ◽

Weld Defect ◽

Defect Recognition ◽

Image Defect

Abstract Welding defects not only bring several economic losses to enterprises and individuals but also threatens peoples lives. We propose a deep learning model, where the data-trained deep learning algorithm is employed to detect the weld defects, and the Convolutional Neural Networks (CNNs) are utilized to recognize the image features. The Transfer Learning (TL) is adopted to reduce the training time via simple adjustments and hyperparameter regulations. The designed deep learning-based model is compared with other classic models to prove its effectiveness in weld defect detection and image recognition further. The results show this model can accurately identify weld defects and eliminates the complexity of manually extracting features, reaching a recognition accuracy of 92.54%. Hence, the reliability and automation of detection and recognition is improved signifificantly. Actual application also verififies the effectiveness of TL in weld defect detection and image defect recognition. Therefore, our research results can provide theoretical and practical references for effificient automatic detection of steel plates, cost reduction, and the high-quality development of iron and steel enterprises.Index Terms - convolutional neural network, deep learning, image detect recognition, transfer learning, weld defect detection

Download Full-text

Classification of papillary thyroid carcinoma histological images based on deep learning

Journal of Intelligent & Fuzzy Systems ◽

10.3233/jifs-210100 ◽

2021 ◽

pp. 1-11

Author(s):

Yaning Liu ◽

Lin Han ◽

Hexiang Wang ◽

Bo Yin

Keyword(s):

Neural Network ◽

Differential Diagnosis ◽

Deep Learning ◽

Papillary Thyroid Carcinoma ◽

Thyroid Carcinoma ◽

Image Features ◽

Papillary Thyroid ◽

Histological Image ◽

Histological Images

Papillary thyroid carcinoma (PTC) is a common carcinoma in thyroid. As many benign thyroid nodules have the papillary structure which could easily be confused with PTC in morphology. Thus, pathologists have to take a lot of time on differential diagnosis of PTC besides personal diagnostic experience and there is no doubt that it is subjective and difficult to obtain consistency among observers. To address this issue, we applied deep learning to the differential diagnosis of PTC and proposed a histological image classification method for PTC based on the Inception Residual convolutional neural network (IRCNN) and support vector machine (SVM). First, in order to expand the dataset and solve the problem of histological image color inconsistency, a pre-processing module was constructed that included color transfer and mirror transform. Then, to alleviate overfitting of the deep learning model, we optimized the convolution neural network by combining Inception Network and Residual Network to extract image features. Finally, the SVM was trained via image features extracted by IRCNN to perform the classification task. Experimental results show effectiveness of the proposed method in the classification of PTC histological images.

Download Full-text

Sewer pipe defect detection via deep learning with local and global feature fusion

Automation in Construction ◽

10.1016/j.autcon.2021.103823 ◽

2021 ◽

Vol 129 ◽

pp. 103823

Author(s):

Dawei Li ◽

Qian Xie ◽

Zhenghao Yu ◽

Qiaoyun Wu ◽

Jun Zhou ◽

...

Keyword(s):

Deep Learning ◽

Defect Detection ◽

Feature Fusion ◽

Global Feature ◽

Sewer Pipe

Download Full-text

A Novel Deep Learning Approach of Convolutional Neural Network and Random Forest Classifier for Fine-grained Sentiment Classification

International Journal on Electrical Engineering and Informatics ◽

10.15676/ijeei.2021.13.2.13 ◽

2021 ◽

Vol 13 (2) ◽

pp. 465-476

Author(s):

Siji George C. G. ◽

◽

B Sumathi ◽

Keyword(s):

Neural Network ◽

Deep Learning ◽

Random Forest ◽

Convolutional Neural Network ◽

Random Forest Classifier ◽

Sentiment Classification ◽

Learning Approach ◽

Fine Grained

Download Full-text

MultiDefectNet: Multi-Class Defect Detection of Building Façade Based on Deep Convolutional Neural Network

Sustainability ◽

10.3390/su12229785 ◽

2020 ◽

Vol 12 (22) ◽

pp. 9785

Author(s):

Kisu Lee ◽

Goopyo Hong ◽

Lee Sael ◽

Sanghyo Lee ◽

Ha Young Kim

Keyword(s):

Neural Network ◽

Deep Learning ◽

Object Detection ◽

Convolutional Neural Network ◽

Defect Detection ◽

Structural Integrity ◽

Residential Building ◽

Training Environment ◽

Building Facade ◽

Building Facades

Defects in residential building façades affect the structural integrity of buildings and degrade external appearances. Defects in a building façade are typically managed using manpower during maintenance. This approach is time-consuming, yields subjective results, and can lead to accidents or casualties. To address this, we propose a building façade monitoring system that utilizes an object detection method based on deep learning to efficiently manage defects by minimizing the involvement of manpower. The dataset used for training a deep-learning-based network contains actual residential building façade images. Various building designs in these raw images make it difficult to detect defects because of their various types and complex backgrounds. We employed the faster regions with convolutional neural network (Faster R-CNN) structure for more accurate defect detection in such environments, achieving an average precision (intersection over union (IoU) = 0.5) of 62.7% for all types of trained defects. As it is difficult to detect defects in a training environment, it is necessary to improve the performance of the network. However, the object detection network employed in this study yields an excellent performance in complex real-world images, indicating the possibility of developing a system that would detect defects in more types of building façades.

Download Full-text

A Deep Learning-Based Approach for the Detection of Early Signs of Gingivitis in Orthodontic Patients Using Faster Region-Based Convolutional Neural Networks

International Journal of Environmental Research and Public Health ◽

10.3390/ijerph17228447 ◽

2020 ◽

Vol 17 (22) ◽

pp. 8447

Author(s):

Dima M. Alalharith ◽

Hajar M. Alharthi ◽

Wejdan M. Alghamdi ◽

Yasmine M. Alsenbel ◽

Nida Aslam ◽

...

Keyword(s):

Neural Network ◽

Deep Learning ◽

Periodontal Disease ◽

Convolutional Neural Network ◽

Region Of Interest ◽

Training Dataset ◽

Detection Accuracy ◽

Detection Model ◽

Art Object ◽

Orthodontic Patients

Computer-based technologies play a central role in the dentistry field, as they present many methods for diagnosing and detecting various diseases, such as periodontitis. The current study aimed to develop and evaluate the state-of-the-art object detection and recognition techniques and deep learning algorithms for the automatic detection of periodontal disease in orthodontic patients using intraoral images. In this study, a total of 134 intraoral images were divided into a training dataset (n = 107 [80%]) and a test dataset (n = 27 [20%]). Two Faster Region-based Convolutional Neural Network (R-CNN) models using ResNet-50 Convolutional Neural Network (CNN) were developed. The first model detects the teeth to locate the region of interest (ROI), while the second model detects gingival inflammation. The detection accuracy, precision, recall, and mean average precision (mAP) were calculated to verify the significance of the proposed model. The teeth detection model achieved an accuracy, precision, recall, and mAP of 100 %, 100%, 51.85%, and 100%, respectively. The inflammation detection model achieved an accuracy, precision, recall, and mAP of 77.12%, 88.02%, 41.75%, and 68.19%, respectively. This study proved the viability of deep learning models for the detection and diagnosis of gingivitis in intraoral images. Hence, this highlights its potential usability in the field of dentistry and aiding in reducing the severity of periodontal disease globally through preemptive non-invasive diagnosis.

Download Full-text

An Efficient and Lightweight Convolutional Neural Network for Remote Sensing Image Scene Classification

Sensors ◽

10.3390/s20071999 ◽

2020 ◽

Vol 20 (7) ◽

pp. 1999 ◽

Cited By ~ 6

Author(s):

Donghang Yu ◽

Qing Xu ◽

Haitao Guo ◽

Chuan Zhao ◽

Yuzhun Lin ◽

...

Keyword(s):

Neural Network ◽

Remote Sensing ◽

Convolutional Neural Network ◽

Visual Recognition ◽

Feature Fusion ◽

Remote Sensing Image ◽

Classification Performance ◽

Image Features ◽

Training Dataset ◽

Scene Classification

Classifying remote sensing images is vital for interpreting image content. Presently, remote sensing image scene classification methods using convolutional neural networks have drawbacks, including excessive parameters and heavy calculation costs. More efficient and lightweight CNNs have fewer parameters and calculations, but their classification performance is generally weaker. We propose a more efficient and lightweight convolutional neural network method to improve classification accuracy with a small training dataset. Inspired by fine-grained visual recognition, this study introduces a bilinear convolutional neural network model for scene classification. First, the lightweight convolutional neural network, MobileNetv2, is used to extract deep and abstract image features. Each feature is then transformed into two features with two different convolutional layers. The transformed features are subjected to Hadamard product operation to obtain an enhanced bilinear feature. Finally, the bilinear feature after pooling and normalization is used for classification. Experiments are performed on three widely used datasets: UC Merced, AID, and NWPU-RESISC45. Compared with other state-of-art methods, the proposed method has fewer parameters and calculations, while achieving higher accuracy. By including feature fusion with bilinear pooling, performance and accuracy for remote scene classification can greatly improve. This could be applied to any remote sensing image classification task.

Download Full-text

Wood Defect Detection Based on Depth Extreme Learning Machine

Applied Sciences ◽

10.3390/app10217488 ◽

2020 ◽

Vol 10 (21) ◽

pp. 7488

Author(s):

Yutu Yang ◽

Xiaolin Zhou ◽

Ying Liu ◽

Zhongkang Hu ◽

Fenglong Ding

Keyword(s):

Deep Learning ◽

Extreme Learning Machine ◽

Defect Detection ◽

Clustering Algorithm ◽

Classification Performance ◽

Image Features ◽

Test Time ◽

Machine Model ◽

Feature Extraction Method ◽

Learning Machine

The deep learning feature extraction method and extreme learning machine (ELM) classification method are combined to establish a depth extreme learning machine model for wood image defect detection. The convolution neural network (CNN) algorithm alone tends to provide inaccurate defect locations, incomplete defect contour and boundary information, and inaccurate recognition of defect types. The nonsubsampled shearlet transform (NSST) is used here to preprocess the wood images, which reduces the complexity and computation of the image processing. CNN is then applied to manage the deep algorithm design of the wood images. The simple linear iterative clustering algorithm is used to improve the initial model; the obtained image features are used as ELM classification inputs. ELM has faster training speed and stronger generalization ability than other similar neural networks, but the random selection of input weights and thresholds degrades the classification accuracy. A genetic algorithm is used here to optimize the initial parameters of the ELM to stabilize the network classification performance. The depth extreme learning machine can extract high-level abstract information from the data, does not require iterative adjustment of the network weights, has high calculation efficiency, and allows CNN to effectively extract the wood defect contour. The distributed input data feature is automatically expressed in layer form by deep learning pre-training. The wood defect recognition accuracy reached 96.72% in a test time of only 187 ms.

Download Full-text

Utilizing the Road Mark Training Set from Ground-Based Mapping System to Airborne Imagery in Deep Learning Framework

Abstracts of the ICA ◽

10.5194/ica-abs-1-364-2019 ◽

2019 ◽

Vol 1 ◽

pp. 1-1

Author(s):

Tee-Ann Teo

Keyword(s):

Neural Network ◽

Deep Learning ◽

Spatial Resolution ◽

Image Features ◽

Training Data ◽

Training Set ◽

Data Set ◽

The Road ◽

Mapping System ◽

Close Range

Abstract. Deep Learning is a kind of Machine Learning technology which utilizing the deep neural network to learn a promising model from a large training data set. Convolutional Neural Network (CNN) has been successfully applied in image segmentation and classification with high accuracy results. The CNN applies multiple kernels (also called filters) to extract image features via image convolution. It is able to determine multiscale features through the multiple layers of convolution and pooling processes. The variety of training data plays an important role to determine a reliable CNN model. The benchmarking training data for road mark extraction is mainly focused on close-range imagery because it is easier to obtain a close-range image rather than an airborne image. For example, KITTI Vision Benchmark Suite. This study aims to transfer the road mark training data from mobile lidar system to aerial orthoimage in Fully Convolutional Networks (FCN). The transformation of the training data from ground-based system to airborne system may reduce the effort of producing a large training data set.This study uses FCN technology and aerial orthoimage to localize road marks on the road regions. The road regions are first extracted from 2-D large-scale vector map. The input aerial orthoimage is 10&thinsp;cm spatial resolution and the non-road regions are masked out before the road mark localization. The training data are road mark’s polygons, which are originally digitized from ground-based mobile lidar and prepared for the road mark extraction using mobile mapping system. This study reuses these training data and applies them for the road mark extraction using aerial orthoimage. The digitized training road marks are then transformed to road polygon based on mapping coordinates. As the detail of ground-based lidar is much better than the airborne system, the partially occulted parking lot in aerial orthoimage can also be obtained from the ground-based system. The labels (also called annotations) for FCN include road region, non-regions and road mark. The size of a training batch is 500&thinsp;pixel by 500&thinsp;pixel (50&thinsp;m by 50&thinsp;m on the ground), and the total number of training batches for training is 75 batches. After the FCN training stage, an independent aerial orthoimage (Figure 1a) is applied to predict the road marks. The results of FCN provide initial regions for road marks (Figure 1b). Usually, road marks show higher reflectance than road asphalts. Therefore, this study uses this characteristic to refine the road marks (Figure 1c) by a binary classification inside the initial road mark’s region.To compare the automatically extracted road marks (Figure 1c) and manually digitized road marks (Figure 1d), most road marks can be extracted using the training set from ground-based system. This study also selects an area of 600&thinsp;m&thinsp;&times;&thinsp;200&thinsp;m in quantitative analysis. Among the 371 reference road marks, 332 can be extracted from proposed scheme, and the completeness reached 89%. The preliminary experiment demonstrated that most road marks can be successfully extracted by the proposed scheme. Therefore, the training data from the ground-based mapping system can be utilized in airborne orthoimage in similar spatial resolution.

Download Full-text