Infrared Image Superpixel Segmentation Based on Seed Strategy of Contour Encoding

Superpixel segmentation has become a crucial pre-processing tool to reduce computation in many computer vision applications. In this paper, a superpixel extraction algorithm based on a seed strategy of contour encoding (SSCE) for infrared images is presented, which can generate superpixels with high boundary adherence and compactness. Specifically, SSCE can solve the problem of superpixels being unable to self-adapt to the image content. First, a contour encoding map is obtained by ray scanning the binary edge map, which ensures that each connected domain belongs to the same homogeneous region. Second, according to the seed sampling strategy, each seed point can be extracted from the contour encoding map. The initial seed set, which is adaptively scattered based on the local structure, is capable of improving the capability of boundary adherence, especially for small regions. Finally, the initial superpixels limited by the image contour are generated by clustering and refined by merging similar adjacent superpixels in the region adjacency graph (RAG) to reduce redundant superpixels. Experimental results on a self-built infrared dataset and the public datasets BSD500 and 3Dircadb demonstrate the generalization ability in grayscale and medical images, and the superiority of the proposed method over several state-of-the-art methods in terms of accuracy and compactness.

Classification of the Sidewalk Condition Using Self-Supervised Transfer Learning for Wheelchair Safety Driving

The demand for wheelchairs has increased recently as the population of the elderly and patients with disorders increases. However, society still pays less attention to infrastructure that can threaten the wheelchair user, such as sidewalks with cracks/potholes. Although various studies have been proposed to recognize such challenges, they mainly depend on RGB images or IMU sensors, which are sensitive to outdoor conditions such as low illumination, bad weather, and unavoidable vibrations, resulting in unsatisfactory and unstable performance. In this paper, we introduce a novel system based on various convolutional neural networks (CNNs) to automatically classify the condition of sidewalks using images captured with depth and infrared modalities. Moreover, we compare the performance of training CNNs from scratch and the transfer learning approach, where the weights learned from the natural image domain (e.g., ImageNet) are fine-tuned to the depth and infrared image domain. In particular, we propose applying the ResNet-152 model pre-trained with self-supervised learning during transfer learning to leverage better image representations. Performance evaluation on the classification of the sidewalk condition was conducted with 100% and 10% of training data. The experimental results validate the effectiveness and feasibility of the proposed approach and bring future research directions.

MSIS: Multispectral Instance Segmentation Method for Power Equipment

Infrared image of power equipment is widely used in power equipment fault detection, and segmentation of infrared images is an important step in power equipment thermal fault detection. Nevertheless, since the overlap of the equipment, the complex background, and the low contrast of the infrared image, the current method still cannot complete the detection and segmentation of the power equipment well. To better segment the power equipment in the infrared image, in this paper, a multispectral instance segmentation (MSIS) based on SOLOv2 is designed, which is an end-to-end and single-stage network. First, we provide a novel structure of multispectral feature extraction, which can simultaneously obtain rich features in visible images and infrared images. Secondly, a module of feature fusion (MARFN) has been constructed to fully obtain fusion features. Finally, the combination of multispectral feature extraction, the module of feature fusion (MARFN), and instance segmentation (SOLOv2) realize multispectral instance segmentation of power equipment. The experimental results show that the proposed MSIS model has an excellent performance in the instance segmentation of power equipment. The MSIS based on ResNet-50 has 40.06% AP.

Quantitative prediction of moisture content distribution in acetylated wood using near-infrared hyperspectral imaging

The uptake of moisture severely affects the properties of wood in service applications. Even local moisture content variations may be critical, but such variations are typically not detected by traditional methods to quantify the moisture content of the wood. In this study, we used near-infrared hyperspectral imaging to predict the moisture distribution on wood surfaces at the macroscale. A broad range of wood moisture contents were generated by controlling the acetylation degree of wood and the relative humidity during sample conditioning. Near-infrared image spectra were then measured from the surfaces of the conditioned wood samples, and a principal component analysis was applied to separate the useful chemical information from the spectral data. Moreover, a partial least squares regression model was developed to predict moisture content on the wood surfaces. The results show that hyperspectral near-infrared image regression can accurately predict the variations in moisture content across wood surfaces. In addition to sample-to-sample variation in moisture content, our results also revealed differences in the moisture content between earlywood and latewood in acetylated wood. This was in line with our recent studies where we found that thin-walled earlywood cells are acetylated faster than the thicker latewood cells, which decreases the moisture uptake during the conditioning. Dynamic vapor sorption isotherms validated the differences in moisture content within earlywood and latewood cells. Overall, our results demonstrate the capabilities of hyperspectral imaging for process analytics in the modern wood industry. Graphical abstract

Key Points’ Location in Infrared Images of the Human Body Based on Mscf-ResNet

The human body generates infrared radiation through the thermal movement of molecules. Based on this phenomenon, infrared images of the human body are often used for monitoring and tracking. Among them, key point location on infrared images of the human body is an important technology in medical infrared image processing. However, the fuzzy edges, poor detail resolution, and uneven brightness distribution of the infrared image of the human body cause great difficulties in positioning. Therefore, how to improve the positioning accuracy of key points in human infrared images has become the main research direction. In this study, a multi-scale convolution fusion deep residual network (Mscf-ResNet) model is proposed for human body infrared image positioning. This model is based on the traditional ResNet, changing the single-scale convolution to multi-scale and fusing the information of different receptive fields, so that the extracted features are more abundant and the degradation problem, caused by the excessively deep network, is avoided. The experiments show that our proposed method has higher key point positioning accuracy than other methods. At the same time, because the network structure of this paper is too deep, there are too many parameters and a large volume of calculations. Therefore, a more lightweight network model is the direction of future research.

Diagnosis and Classification Decision Analysis of Overheating Defects of Substation Equipment Based on Infrared Detection Technology

Substation equipment is not only the main part of the power grid but also the essential part to ensure the development of the national economy and People's Daily life of one of the important infrastructure. How to ensure its normal operation and find the sudden failure has become a hot issue to be solved urgently. For thermal fault diagnosis needs to classify and identify different power equipment first, this paper designed an SVM infrared image classifier, which can effectively identify three types of common power equipment. The classifier extracts HOG features from the infrared images of power equipment processed by the above segmentation and combines them with SVM multiclassification to achieve the purpose of improving the recognition accuracy. The experiment uses the classifier to identify three kinds of equipment, and the results show that the comprehensive recognition accuracy of the classifier is more than 95.3%, which is better than the traditional classification method and meets the demand for classification accuracy. In this paper, the traditional method of relative temperature difference is improved by using the temperature data of the infrared image, which can automatically judge the thermal failure level of electric power equipment. Experiments show that the diagnosis system designed in this paper can classify faults and give treatment suggestions while judging whether there are thermal faults for three types of power equipment, which verifies the feasibility and effectiveness of the substation infrared diagnosis technology designed in this paper.

A Multi-Source Data Fusion Decision-Making Method for Disease and Pest Detection of Grape Foliage Based on ShuffleNet V2

Disease and pest detection of grape foliage is essential for grape yield and quality. RGB image (RGBI), multispectral image (MSI), and thermal infrared image (TIRI) are widely used in the health detection of plants. In this study, we collected three types of grape foliage images with six common classes (anthracnose, downy mildew, leafhopper, mites, viral disease, and healthy) in the field. ShuffleNet V2 was used to build up detection models. According to the accuracy of RGBI, MSI, TIRI, and multi-source data concatenation (MDC) models, and a multi-source data fusion (MDF) decision-making method was proposed for improving the detection performance for grape foliage, aiming to enhance the decision-making for RGBI of grape foliage by fusing the MSI and TIRI. The results showed that 40% of the incorrect detection outputs were rectified using the MDF decision-making method. The overall accuracy of MDF model was 96.05%, which had improvements of 2.64%, 13.65%, and 27.79%, compared with the RGBI, MSI, and TIRI models using label smoothing, respectively. In addition, the MDF model was based on the lightweight network with 3.785 M total parameters and 0.362 G multiply-accumulate operations, which could be highly portable and easy to be applied.