Reflection interference removal for infrared thermography images based on GAN

2021 ◽  
Vol 63 (9) ◽  
pp. 529-533
Author(s):  
Jiali Zhang ◽  
Yupeng Tian ◽  
LiPing Ren ◽  
Jiaheng Cheng ◽  
JinChen Shi
Keyword(s):  
Neural Networks ◽  
Infrared Thermography ◽  
Metal Surfaces ◽  
Infrared Image ◽  
Infrared Camera ◽  
Infrared Images ◽  
Generative Adversarial Network ◽  
Adversarial Network ◽  
Continuous Application ◽  
Visible Images

Reflection in images is common and the removal of complex noise such as image reflection is still being explored. The problem is difficult and ill-posed, not only because there is no mixing function but also because there are no constraints in the output space (the processed image). When it comes to detecting defects on metal surfaces using infrared thermography, reflection from smooth metal surfaces can easily affect the final detection results. Therefore, it is essential to remove the reflection interference in infrared images. With the continuous application and expansion of neural networks in the field of image processing, researchers have tried to apply neural networks to remove image reflection. However, they have mainly focused on reflection interference removal in visible images and it is believed that no researchers have applied neural networks to remove reflection interference in infrared images. In this paper, the authors introduce the concept of a conditional generative adversarial network (cGAN) and propose an end-to-end trained network based on this with two types of loss: perceptual loss and adversarial loss. A self-built infrared reflection image dataset from an infrared camera is used. The experimental results demonstrate the effectiveness of this GAN for removing infrared image reflection.

scholarly journals Learning a Generative Model for Fusing Infrared and Visible Images via Conditional Generative Adversarial Network with Dual Discriminators

2019 ◽  
Author(s):  
Han Xu ◽  
Pengwei Liang ◽  
Wei Yu ◽  
Junjun Jiang ◽  
Jiayi Ma
Keyword(s):  
Probability Distribution ◽  
State Of The Art ◽  
Infrared Image ◽  
Infrared Images ◽  
Generative Adversarial Network ◽  
Visible Image ◽  
Qualitative And Quantitative ◽  
Adversarial Network ◽  
Fused Image ◽  
Visible Images

In this paper, we propose a new end-to-end model, called dual-discriminator conditional generative adversarial network (DDcGAN), for fusing infrared and visible images of different resolutions. Unlike the pixel-level methods and existing deep learning-based methods, the fusion task is accomplished through the adversarial process between a generator and two discriminators, in addition to the specially designed content loss. The generator is trained to generate real-like fused images to fool discriminators. The two discriminators are trained to calculate the JS divergence between the probability distribution of downsampled fused images and infrared images, and the JS divergence between the probability distribution of gradients of fused images and gradients of visible images, respectively. Thus, the fused images can compensate for the features that are not constrained by the single content loss. Consequently, the prominence of thermal targets in the infrared image and the texture details in the visible image can be preserved or even enhanced in the fused image simultaneously. Moreover, by constraining and distinguishing between the downsampled fused image and the low-resolution infrared image, DDcGAN can be preferably applied to the fusion of different resolution images. Qualitative and quantitative experiments on publicly available datasets demonstrate the superiority of our method over the state-of-the-art.

scholarly journals Generative Adversarial Network-Based Edge-Preserving Superresolution Reconstruction of Infrared Images

2021 ◽  
Vol 2021 ◽  
pp. 1-12
Author(s):  
Yuqing Zhao ◽  
Guangyuan Fu ◽  
Hongqiao Wang ◽  
Shaolei Zhang ◽  
Min Yue
Keyword(s):  
Infrared Image ◽  
Reconstruction Method ◽  
Infrared Images ◽  
Generative Adversarial Network ◽  
Edge Preserving ◽  
Edge Structure ◽  
Second Stage ◽  
Adversarial Network ◽  
Objective Quality ◽  
Image Edges

The convolutional neural network has achieved good results in the superresolution reconstruction of single-frame images. However, due to the shortcomings of infrared images such as lack of details, poor contrast, and blurred edges, superresolution reconstruction of infrared images that preserves the edge structure and better visual quality is still challenging. Aiming at the problems of low resolution and unclear edges of infrared images, this work proposes a two-stage generative adversarial network model to reconstruct realistic superresolution images from four times downsampled infrared images. In the first stage of the generative adversarial network, it focuses on recovering the overall contour information of the image to obtain clear image edges; the second stage of the generative adversarial network focuses on recovering the detailed feature information of the image and has a stronger ability to express details. The infrared image superresolution reconstruction method proposed in this work has highly realistic visual effects and good objective quality evaluation results.

scholarly journals A Generative Adversarial Network for Infrared and Visible Image Fusion Based on Semantic Segmentation

Entropy ◽  
2021 ◽  
Vol 23 (3) ◽  
pp. 376
Author(s):  
Jilei Hou ◽  
Dazhi Zhang ◽  
Wei Wu ◽  
Jiayi Ma ◽  
Huabing Zhou
Keyword(s):  
Image Fusion ◽  
Information Source ◽  
Semantic Segmentation ◽  
Input Image ◽  
Infrared Images ◽  
Generative Adversarial Network ◽  
Visible Image ◽  
Adversarial Network ◽  
Visible Images ◽  
High Level

This paper proposes a new generative adversarial network for infrared and visible image fusion based on semantic segmentation (SSGAN), which can consider not only the low-level features of infrared and visible images, but also the high-level semantic information. Source images can be divided into foregrounds and backgrounds by semantic masks. The generator with a dual-encoder-single-decoder framework is used to extract the feature of foregrounds and backgrounds by different encoder paths. Moreover, the discriminator’s input image is designed based on semantic segmentation, which is obtained by combining the foregrounds of the infrared images with the backgrounds of the visible images. Consequently, the prominence of thermal targets in the infrared images and texture details in the visible images can be preserved in the fused images simultaneously. Qualitative and quantitative experiments on publicly available datasets demonstrate that the proposed approach can significantly outperform the state-of-the-art methods.

scholarly journals Patch-Wise Infrared and Visible Image Fusion Using Spatial Adaptive Weights

2021 ◽  
Vol 11 (19) ◽  
pp. 9255
Author(s):  
Syeda Minahil ◽  
Jun-Hyung Kim ◽  
Youngbae Hwang
Keyword(s):  
Image Fusion ◽  
Infrared Image ◽  
Source Image ◽  
Infrared Images ◽  
Generative Adversarial Network ◽  
Visible Image ◽  
Significant Information ◽  
Adaptive Weights ◽  
Adversarial Network ◽  
Fusion Methods

In infrared (IR) and visible image fusion, the significant information is extracted from each source image and integrated into a single image with comprehensive data. We observe that the salient regions in the infrared image contain targets of interests. Therefore, we enforce spatial adaptive weights derived from the infrared images. In this paper, a Generative Adversarial Network (GAN)-based fusion method is proposed for infrared and visible image fusion. Based on the end-to-end network structure with dual discriminators, a patch-wise discrimination is applied to reduce blurry artifact from the previous image-level approaches. A new loss function is also proposed to use constructed weight maps which direct the adversarial training of GAN in a manner such that the informative regions of the infrared images are preserved. Experiments are performed on the two datasets and ablation studies are also conducted. The qualitative and quantitative analysis shows that we achieve competitive results compared to the existing fusion methods.

scholarly journals U-GAN Model for Infrared and Visible Images Fusion

2020 ◽  
Vol 38 (4) ◽  
pp. 904-912
Author(s):  
Zhuo Chen ◽  
Ming Fang ◽  
Xu Chai ◽  
Feiran Fu ◽  
Lihong Yuan
Keyword(s):  
Image Fusion ◽  
Network Architecture ◽  
Infrared Image ◽  
Data Sets ◽  
Generative Adversarial Network ◽  
Visible Image ◽  
Adversarial Network ◽  
Convolution Structure ◽  
Fusion Images ◽  
Visible Images

Infrared and visible image fusion is an effective method to solve the lack of single sensor imaging. The purpose is that the fusion images are suitable for human eyes and conducive to the next application and processing. In order to solve the problems of incomplete feature extraction, loss of details, and less samples of common data sets, it is not conducive to training, an end-to-end network architecture for image fusion is proposed. U-net is introduced into image fusion, and the final fusion result is obtained by using the generative adversarial network. Through its special convolution structure, the important feature information is extracted to the maximum extent, and the sample does not need to be cut to avoid the problem of reducing the fusion accuracy, but also to improve the training speed. Then the U-net extracted feature is confronted with the discriminator containing infrared image, and the generator model is obtained. The experimental results show that the present algorithm can obtain the fusion image with clear outline, prominent texture and obvious target. SD, SF, SSIM, AG and other indicators are obviously improved.

scholarly journals Remote Sensing Image Dataset Expansion Based on Generative Adversarial Networks with Modified Shuffle Attention

Sensors ◽  
2021 ◽  
Vol 21 (14) ◽  
pp. 4867
Author(s):  
Lu Chen ◽  
Hongjun Wang ◽  
Xianghao Meng
Keyword(s):  
Remote Sensing ◽  
Neural Networks ◽  
Image Processing ◽  
Remote Sensing Image ◽  
Generative Adversarial Networks ◽  
Generative Adversarial Network ◽  
Evaluation Indexes ◽  
Adversarial Network ◽  
Remote Sensing Image Processing ◽  
Data Expansion

With the development of science and technology, neural networks, as an effective tool in image processing, play an important role in gradual remote-sensing image-processing. However, the training of neural networks requires a large sample database. Therefore, expanding datasets with limited samples has gradually become a research hotspot. The emergence of the generative adversarial network (GAN) provides new ideas for data expansion. Traditional GANs either require a large number of input data, or lack detail in the pictures generated. In this paper, we modify a shuffle attention network and introduce it into GAN to generate higher quality pictures with limited inputs. In addition, we improved the existing resize method and proposed an equal stretch resize method to solve the problem of image distortion caused by different input sizes. In the experiment, we also embed the newly proposed coordinate attention (CA) module into the backbone network as a control test. Qualitative indexes and six quantitative evaluation indexes were used to evaluate the experimental results, which show that, compared with other GANs used for picture generation, the modified Shuffle Attention GAN proposed in this paper can generate more refined and high-quality diversified aircraft pictures with more detailed features of the object under limited datasets.

scholarly journals The Feasibility of Identifying Defects Illustrated on Building Facades by Applying Thermal Infrared Images with Shadow

Proceedings ◽  
2019 ◽  
Vol 27 (1) ◽  
pp. 3
Author(s):  
Tsai ◽  
Huang ◽  
Tai
Keyword(s):  
Surface Temperature ◽  
Infrared Thermography ◽  
Infrared Camera ◽  
Thermal Infrared ◽  
Infrared Images ◽  
Building Facades ◽  
Thermal Infrared Images ◽  
Non Destructive

Infrared thermography (IRT) has been widely employed to identify the defects illustrated in building facades. However, the IRT covered with a shadow is hard to be applied to determine the defects shown in the IRT. The study proposed an approach based on the multiplicated model to describe quantitively the shadow effects, and the IRT can be segmented into few classes according to the surface temperature information recorded on the IRT by employing a thermal infrared camera. The segmented results were compared with the non-destructive method (acoustic tracing) to verify the correctness and robustness of the approach. From the processed results, the proposed approach did correctly identify the defects illustrated in building facades through the IRTs were covered with shadow.

scholarly journals Generative Adversarial Networks for Visible to Infrared Video Conversion

2020 ◽  
Author(s):  
Mohammad Shahab Uddin ◽  
Jiang Li
Keyword(s):  
Deep Learning ◽  
Performance Metrics ◽  
Infrared Image ◽  
Image Databases ◽  
Learning Models ◽  
Generative Adversarial Network ◽  
Adversarial Network ◽  
Optical Images ◽  
Infrared Video ◽  
Image Pairs

Deep learning models are data driven. For example, the most popular convolutional neural network (CNN) model used for image classification or object detection requires large labeled databases for training to achieve competitive performances. This requirement is not difficult to be satisfied in the visible domain since there are lots of labeled video and image databases available nowadays. However, given the less popularity of infrared (IR) camera, the availability of labeled infrared videos or image databases is limited. Therefore, training deep learning models in infrared domain is still challenging. In this chapter, we applied the pix2pix generative adversarial network (Pix2Pix GAN) and cycle-consistent GAN (Cycle GAN) models to convert visible videos to infrared videos. The Pix2Pix GAN model requires visible-infrared image pairs for training while the Cycle GAN relaxes this constraint and requires only unpaired images from both domains. We applied the two models to an open-source database where visible and infrared videos provided by the signal multimedia and telecommunications laboratory at the Federal University of Rio de Janeiro. We evaluated conversion results by performance metrics including Inception Score (IS), Frechet Inception Distance (FID) and Kernel Inception Distance (KID). Our experiments suggest that cycle-consistent GAN is more effective than pix2pix GAN for generating IR images from optical images.

scholarly journals Fusion algorithm of visible and infrared image based on anisotropic diffusion and image enhancement (capitalize only the first word in a title (or heading), the first word in a subtitle (or subheading), and any proper nouns)

PLoS ONE ◽  
2021 ◽  
Vol 16 (2) ◽  
pp. e0245563
Author(s):  
Hui Huang ◽  
Linlu Dong ◽  
Zhishuang Xue ◽  
Xiaofang Liu ◽  
Caijian Hua
Keyword(s):  
Image Enhancement ◽  
Anisotropic Diffusion ◽  
Infrared Image ◽  
Source Image ◽  
Laplacian Pyramid ◽  
Infrared Images ◽  
Fusion Algorithm ◽  
Public Data ◽  
Visible Images ◽  
Enhancement Algorithm

Aiming at the situation that the existing visible and infrared images fusion algorithms only focus on highlighting infrared targets and neglect the performance of image details, and cannot take into account the characteristics of infrared and visible images, this paper proposes an image enhancement fusion algorithm combining Karhunen-Loeve transform and Laplacian pyramid fusion. The detail layer of the source image is obtained by anisotropic diffusion to get more abundant texture information. The infrared images adopt adaptive histogram partition and brightness correction enhancement algorithm to highlight thermal radiation targets. A novel power function enhancement algorithm that simulates illumination is proposed for visible images to improve the contrast of visible images and facilitate human observation. In order to improve the fusion quality of images, the source image and the enhanced images are transformed by Karhunen-Loeve to form new visible and infrared images. Laplacian pyramid fusion is performed on the new visible and infrared images, and superimposed with the detail layer images to obtain the fusion result. Experimental results show that the method in this paper is superior to several representative image fusion algorithms in subjective visual effects on public data sets. In terms of objective evaluation, the fusion result performed well on the 8 evaluation indicators, and its own quality was high.

scholarly journals Learning Cartographic Building Generalization with Deep Convolutional Neural Networks

2019 ◽  
Vol 8 (6) ◽  
pp. 258 ◽  
Author(s):  
Yu Feng ◽  
Frank Thiemann ◽  
Monika Sester
Keyword(s):  
Neural Networks ◽  
Computer Vision ◽  
Deep Learning ◽  
Convolutional Neural Networks ◽  
Physical Reality ◽  
Learning Approaches ◽  
Deep Convolutional Neural Networks ◽  
Generative Adversarial Network ◽  
Adversarial Network ◽  
Cartographic Generalization

Cartographic generalization is a problem, which poses interesting challenges to automation. Whereas plenty of algorithms have been developed for the different sub-problems of generalization (e.g., simplification, displacement, aggregation), there are still cases, which are not generalized adequately or in a satisfactory way. The main problem is the interplay between different operators. In those cases the human operator is the benchmark, who is able to design an aesthetic and correct representation of the physical reality. Deep learning methods have shown tremendous success for interpretation problems for which algorithmic methods have deficits. A prominent example is the classification and interpretation of images, where deep learning approaches outperform traditional computer vision methods. In both domains-computer vision and cartography-humans are able to produce good solutions. A prerequisite for the application of deep learning is the availability of many representative training examples for the situation to be learned. As this is given in cartography (there are many existing map series), the idea in this paper is to employ deep convolutional neural networks (DCNNs) for cartographic generalizations tasks, especially for the task of building generalization. Three network architectures, namely U-net, residual U-net and generative adversarial network (GAN), are evaluated both quantitatively and qualitatively in this paper. They are compared based on their performance on this task at target map scales 1:10,000, 1:15,000 and 1:25,000, respectively. The results indicate that deep learning models can successfully learn cartographic generalization operations in one single model in an implicit way. The residual U-net outperforms the others and achieved the best generalization performance.

Sign in / Sign up

Export Citation Format

Share Document