scholarly journals Infrared and visible image fusion based on optimal segmenting and contour extraction

2021 ◽  
Vol 3 (3) ◽  
Author(s):  
Javad Abbasi Aghamaleki ◽  
Alireza Ghorbani
Keyword(s):  
Image Fusion ◽  
Infrared Image ◽  
Fusion Method ◽  
Visible Image ◽  
Output Image ◽  
Contour Lines ◽  
Fused Image ◽  
Fusion Methods ◽  
Visible Images ◽  
The Individual

AbstractImage fusion is the combining process of complementary information of multiple same scene images into an output image. The resultant output image that is named fused image, produces more precise description of the scene than any of the individual input images. In this paper, we propose a novel simple and fast strategy for infrared (IR) and visible images based on local important areas of IR image. The fusion method is completed in three step approach. Firstly, only the segmented regions in the infrared image is extracted. Next, the image fusion is applied on segmented area and finally, contour lines are also used to improve the quality of the results of the second step of fusion method. Using a publicly available database, the proposed method is evaluated and compared to the other fusion methods. The experimental results show the effectiveness of the proposed method compared to the state of the art methods.

scholarly journals Infrared and Visible Image Fusion through Details Preservation

Sensors ◽  
2019 ◽  
Vol 19 (20) ◽  
pp. 4556 ◽  
Author(s):  
Yaochen Liu ◽  
Lili Dong ◽  
Yuanyuan Ji ◽  
Wenhai Xu
Keyword(s):  
Image Fusion ◽  
Subjective Evaluation ◽  
Objective Assessment ◽  
Decomposition Methods ◽  
Source Image ◽  
Fusion Method ◽  
Visible Image ◽  
Weighting Method ◽  
Fused Image ◽  
Fusion Methods

In many actual applications, fused image is essential to contain high-quality details for achieving a comprehensive representation of the real scene. However, existing image fusion methods suffer from loss of details because of the error accumulations of sequential tasks. This paper proposes a novel fusion method to preserve details of infrared and visible images by combining new decomposition, feature extraction, and fusion scheme. For decomposition, different from the most decomposition methods by guided filter, the guidance image contains only the strong edge of the source image but no other interference information so that rich tiny details can be decomposed into the detailed part. Then, according to the different characteristics of infrared and visible detail parts, a rough convolutional neural network (CNN) and a sophisticated CNN are designed so that various features can be fully extracted. To integrate the extracted features, we also present a multi-layer features fusion strategy through discrete cosine transform (DCT), which not only highlights significant features but also enhances details. Moreover, the base parts are fused by weighting method. Finally, the fused image is obtained by adding the fused detail and base part. Different from the general image fusion methods, our method not only retains the target region of source image but also enhances background in the fused image. In addition, compared with state-of-the-art fusion methods, our proposed fusion method has many advantages, including (i) better visual quality of fused-image subjective evaluation, and (ii) better objective assessment for those images.

scholarly journals Infrared and Visible Image Fusion Using Truncated Huber Penalty Function Smoothing and Visual Saliency Based Threshold Optimization

Electronics ◽  
2021 ◽  
Vol 11 (1) ◽  
pp. 33
Author(s):  
Chaowei Duan ◽  
Yiliu Liu ◽  
Changda Xing ◽  
Zhisheng Wang
Keyword(s):  
Image Fusion ◽  
Penalty Function ◽  
Visual Saliency ◽  
Fusion Rule ◽  
Visible Image ◽  
Threshold Optimization ◽  
Fused Image ◽  
Fusion Methods ◽  
Visible Images ◽  
Image Representations

An efficient method for the infrared and visible image fusion is presented using truncated Huber penalty function smoothing and visual saliency based threshold optimization. The method merges complementary information from multimodality source images into a more informative composite image in two-scale domain, in which the significant objects/regions are highlighted and rich feature information is preserved. Firstly, source images are decomposed into two-scale image representations, namely, the approximate and residual layers, using truncated Huber penalty function smoothing. Benefiting from the edge- and structure-preserving characteristics, the significant objects and regions in the source images are effectively extracted without halo artifacts around the edges. Secondly, a visual saliency based threshold optimization fusion rule is designed to fuse the approximate layers aiming to highlight the salient targets in infrared images and remain the high-intensity regions in visible images. The sparse representation based fusion rule is adopted to fuse the residual layers with the goal of acquiring rich detail texture information. Finally, combining the fused approximate and residual layers reconstructs the fused image with more natural visual effects. Sufficient experimental results demonstrate that the proposed method can achieve comparable or superior performances compared with several state-of-the-art fusion methods in visual results and objective assessments.

scholarly journals A Heterogeneous Image Fusion Method Based on DCT and Anisotropic Diffusion for UAVs in Future 5G IoT Scenarios

2020 ◽  
Vol 2020 ◽  
pp. 1-11
Author(s):  
Shuai Hao ◽  
Beiyi An ◽  
Hu Wen ◽  
Xu Ma ◽  
Keping Yu
Keyword(s):  
Image Fusion ◽  
Traffic Management ◽  
Anisotropic Diffusion ◽  
Weighted Averaging ◽  
Base Layer ◽  
Fusion Method ◽  
Linear Superposition ◽  
Visible Image ◽  
Fusion Methods ◽  
Visible Images

Unmanned aerial vehicles, with their inherent fine attributes, such as flexibility, mobility, and autonomy, play an increasingly important role in the Internet of Things (IoT). Airborne infrared and visible image fusion, which constitutes an important data basis for the perception layer of IoT, has been widely used in various fields such as electric power inspection, military reconnaissance, emergency rescue, and traffic management. However, traditional infrared and visible image fusion methods suffer from weak detail resolution. In order to better preserve useful information from source images and produce a more informative image for human observation or unmanned aerial vehicle vision tasks, a novel fusion method based on discrete cosine transform (DCT) and anisotropic diffusion is proposed. First, the infrared and visible images are denoised by using DCT. Second, anisotropic diffusion is applied to the denoised infrared and visible images to obtain the detail and base layers. Third, the base layers are fused by using weighted averaging, and the detail layers are fused by using the Karhunen–Loeve transform, respectively. Finally, the fused image is reconstructed through the linear superposition of the base layer and detail layer. Compared with six other typical fusion methods, the proposed approach shows better fusion performance in both objective and subjective evaluations.

scholarly journals Enhanced Image Fusion Methodology for Low Light Visible and Infrared Images

2019 ◽  
Vol 8 (9) ◽  
pp. 1145-1149
Keyword(s):  
Image Fusion ◽  
Infrared Image ◽  
Weighted Average ◽  
Principal Component ◽  
Fusion Method ◽  
Visible Image ◽  
Fusion Procedure ◽  
Remote Sensing Image Fusion ◽  
Single Scale ◽  
Fused Image

Image fusion is the mechanism in which at least two images are consolidated into a single image holding the imperative features from each one of the first images. Emerging images are upgraded and the image content is been enhanced in the entire context, this out coming image is much more preferable than the base images. Certain circumstances in image processing need both high dimensional and high spectral information in a solitary image, which is crucial in remote sensing. Image fusion procedure incorporates intensifying, filtering, and moulding the images for better results. Efficient and imperative approaches for image fusion are enforced here. The image fusion method comprises two discrete types of images, the visible image and the infrared image. The Single Scale Retinex (SSR) is applied to the visible image to obtain an upgraded image, simultaneously Principal Component Analysis (PCA) is been applied to infrared image to obtain an image with superior contrast and colour. Further these treated images are decomposed into a multilayer image by using Laplacian Pyramid algorithm. To end with Weighted Average fusion method aids in fusing the images to reproduce the augmented fused image.

scholarly journals Fusing Infrared and Visible Images of Different Resolutions via Total Variation Model

Sensors ◽  
2018 ◽  
Vol 18 (11) ◽  
pp. 3827 ◽  
Author(s):  
Qinglei Du ◽  
Han Xu ◽  
Yong Ma ◽  
Jun Huang ◽  
Fan Fan
Keyword(s):  
Thermal Radiation ◽  
Total Variation ◽  
Infrared Image ◽  
Infrared Images ◽  
Visible Image ◽  
Pixel Intensity ◽  
Texture Information ◽  
Fused Image ◽  
Fusion Methods ◽  
Visible Images

In infrared and visible image fusion, existing methods typically have a prerequisite that the source images share the same resolution. However, due to limitations of hardware devices and application environments, infrared images constantly suffer from markedly lower resolution compared with the corresponding visible images. In this case, current fusion methods inevitably cause texture information loss in visible images or blur thermal radiation information in infrared images. Moreover, the principle of existing fusion rules typically focuses on preserving texture details in source images, which may be inappropriate for fusing infrared thermal radiation information because it is characterized by pixel intensities, possibly neglecting the prominence of targets in fused images. Faced with such difficulties and challenges, we propose a novel method to fuse infrared and visible images of different resolutions and generate high-resolution resulting images to obtain clear and accurate fused images. Specifically, the fusion problem is formulated as a total variation (TV) minimization problem. The data fidelity term constrains the pixel intensity similarity of the downsampled fused image with respect to the infrared image, and the regularization term compels the gradient similarity of the fused image with respect to the visible image. The fast iterative shrinkage-thresholding algorithm (FISTA) framework is applied to improve the convergence rate. Our resulting fused images are similar to super-resolved infrared images, which are sharpened by the texture information from visible images. Advantages and innovations of our method are demonstrated by the qualitative and quantitative comparisons with six state-of-the-art methods on publicly available datasets.

An Improved Biologically-Inspired Image Fusion Method

2018 ◽  
Vol 32 (08) ◽  
pp. 1857004 ◽  
Author(s):  
Yuqing Wang ◽  
Yong Wang
Keyword(s):  
Receptive Field ◽  
Image Fusion ◽  
Color Image ◽  
Infrared Image ◽  
Fusion Method ◽  
Fusion Model ◽  
Biologically Inspired ◽  
Visible Image ◽  
Fusion Methods ◽  
Image Fusion Method

A biologically inspired image fusion mechanism is analyzed in this paper. A pseudo-color image fusion method is proposed based on the improvement of a traditional method. The proposed model describes the fusion process using several abstract definitions which correspond to the detailed behaviors of neurons. Firstly, the infrared image and visible image are respectively ON against enhanced and OFF against enhanced. Secondly, we feed back the enhanced visible images given by the ON-antagonism system to the active cells in the center-surrounding antagonism receptive field. The fused [Formula: see text]VIS[Formula: see text]IR signal are obtained by feeding back the OFF-enhanced infrared image to the corresponding surrounding-depressing neurons. Then we feed back the enhanced visible signal from OFF-antagonism system to the depressing cells in the center-surrounding antagonism receptive field. The ON-enhanced infrared image is taken as the input signal of the corresponding active cells in the neurons, then the cell response of infrared-enhance-visible is produced in the process, it is denoted as [Formula: see text]IR[Formula: see text]VIS. The three kinds of signal are considered as R, G and B components in the output composite image. Finally, some experiments are performed in order to evaluate the performance of the proposed method. The information entropy, average gradient and objective image fusion measure are used to assess the performance of the proposed method objectively. Some traditional digital signal processing-based fusion methods are also evaluated for comparison in the experiments. In this paper, the Quantitative assessment indices show that the proposed fusion model is superior to the classical Waxman’s model, and some of its performance is better than the other image fusion methods.

scholarly journals Learning a Generative Model for Fusing Infrared and Visible Images via Conditional Generative Adversarial Network with Dual Discriminators

2019 ◽  
Author(s):  
Han Xu ◽  
Pengwei Liang ◽  
Wei Yu ◽  
Junjun Jiang ◽  
Jiayi Ma
Keyword(s):  
Probability Distribution ◽  
State Of The Art ◽  
Infrared Image ◽  
Infrared Images ◽  
Generative Adversarial Network ◽  
Visible Image ◽  
Qualitative And Quantitative ◽  
Adversarial Network ◽  
Fused Image ◽  
Visible Images

In this paper, we propose a new end-to-end model, called dual-discriminator conditional generative adversarial network (DDcGAN), for fusing infrared and visible images of different resolutions. Unlike the pixel-level methods and existing deep learning-based methods, the fusion task is accomplished through the adversarial process between a generator and two discriminators, in addition to the specially designed content loss. The generator is trained to generate real-like fused images to fool discriminators. The two discriminators are trained to calculate the JS divergence between the probability distribution of downsampled fused images and infrared images, and the JS divergence between the probability distribution of gradients of fused images and gradients of visible images, respectively. Thus, the fused images can compensate for the features that are not constrained by the single content loss. Consequently, the prominence of thermal targets in the infrared image and the texture details in the visible image can be preserved or even enhanced in the fused image simultaneously. Moreover, by constraining and distinguishing between the downsampled fused image and the low-resolution infrared image, DDcGAN can be preferably applied to the fusion of different resolution images. Qualitative and quantitative experiments on publicly available datasets demonstrate the superiority of our method over the state-of-the-art.

scholarly journals Infrared and Visible Image Fusion Based on Nonlinear Enhancement and NSST Decomposition

2020 ◽  
Author(s):  
Xiaoxue XING ◽  
Cheng LIU ◽  
Cong LUO ◽  
Tingfa XU
Keyword(s):  
Thermal Radiation ◽  
High Frequency ◽  
Geometric Analysis ◽  
Low Frequency ◽  
Fusion Rule ◽  
Fusion Algorithm ◽  
Visible Image ◽  
Fused Image ◽  
Fusion Methods ◽  
Visible Images

Abstract In Multi-scale Geometric Analysis (MGA)-based fusion methods for infrared and visible images, adopting the same representation for the two types of the images will result in the non-obvious thermal radiation target in the fused image, which can hardly be distinguished from the background. To solve the problem, a novel fusion algorithm based on nonlinear enhancement and Non-Subsampled Shearlet Transform (NSST) decomposition is proposed. Firstly, NSST is used to decompose the two source images into low- and high-frequency sub-bands. Then, the Wavelet Transform (WT) is used to decompose high-frequency sub-bands into obtain approximate sub-bands and directional detail sub-bands. The “average” fusion rule is performed for fusion for approximate sub-bands. And the “max-absolute” fusion rule is performed for fusion for directional detail sub-bands. The inverse WT is used to reconstruct the high-frequency sub-bands. To highlight the thermal radiation target, we construct a non-linear transform function to determine the fusion weight of low-frequency sub-bands, and whose parameters can be further adjusted to meet different fusion requirements. Finally, the inverse NSST is used to reconstruct the fused image. The experimental results show that the proposed method can simultaneously enhance the thermal target in infrared images and preserve the texture details in visible images, and which is competitive with or even superior to the state-of-the-art fusion methods in terms of both visual and quantitative evaluations.

scholarly journals Infrared and Visible Image Fusion Based on Nonlinear Enhancement and NSST Decomposition

2020 ◽  
Author(s):  
Xiaoxue XING ◽  
Cheng LIU ◽  
Cong LUO ◽  
Tingfa XU
Keyword(s):  
Thermal Radiation ◽  
High Frequency ◽  
Geometric Analysis ◽  
Low Frequency ◽  
Fusion Rule ◽  
Fusion Algorithm ◽  
Visible Image ◽  
Fused Image ◽  
Fusion Methods ◽  
Visible Images

Abstract In Multi-scale Geometric Analysis (MGA)-based fusion methods for infrared and visible images, adopting the same representation for the two types of the images will result in the non-obvious thermal radiation target in the fused image, which can hardly be distinguished from the background. To solve the problem, a novel fusion algorithm based on nonlinear enhancement and Non-Subsampled Shearlet Transform (NSST) decomposition is proposed. Firstly, NSST is used to decompose the two source images into low- and high-frequency sub-bands. Then, the wavelet transform(WT) is used to decompose high-frequency sub-bands into obtain approximate sub-bands and directional detail sub-bands. The “average” fusion rule is performed for fusion for approximate sub-bands. And the “max-absolute” fusion rule is performed for fusion for directional detail sub-bands. The inverse WT is used to reconstruct the high-frequency sub-bands. To highlight the thermal radiation target, we construct a non-linear transform function to determine the fusion weight of low-frequency sub-bands, and whose parameters can be further adjusted to meet different fusion requirements. Finally, the inverse NSST is used to reconstruct the fused image. The experimental results show that the proposed method can simultaneously enhance the thermal target in infrared images and preserve the texture details in visible images, and which is competitive with or even superior to the state-of-the-art fusion methods in terms of both visual and quantitative evaluations.

scholarly journals GANFuse: a novel multi-exposure image fusion method based on generative adversarial networks

2020 ◽  
Author(s):  
Zhiguang Yang ◽  
Youping Chen ◽  
Zhuliang Le ◽  
Yong Ma
Keyword(s):  
Image Fusion ◽  
Activity Level ◽  
Experimental Result ◽  
Generative Adversarial Networks ◽  
Fusion Method ◽  
Adversarial Networks ◽  
Fused Image ◽  
Fusion Methods ◽  
Image Pairs ◽  
Image Fusion Method

Abstract In this paper, a novel multi-exposure image fusion method based on generative adversarial networks (termed as GANFuse) is presented. Conventional multi-exposure image fusion methods improve their fusion performance by designing sophisticated activity-level measurement and fusion rules. However, these methods have a limited success in complex fusion tasks. Inspired by the recent FusionGAN which firstly utilizes generative adversarial networks (GAN) to fuse infrared and visible images and achieves promising performance, we improve its architecture and customize it in the task of extreme exposure image fusion. To be specific, in order to keep content of extreme exposure image pairs in the fused image, we increase the number of discriminators differentiating between fused image and extreme exposure image pairs. While, a generator network is trained to generate fused images. Through the adversarial relationship between generator and discriminators, the fused image will contain more information from extreme exposure image pairs. Thus, this relationship can realize better performance of fusion. In addition, the method we proposed is an end-to-end and unsupervised learning model, which can avoid designing hand-crafted features and does not require a number of ground truth images for training. We conduct qualitative and quantitative experiments on a public dataset, and the experimental result shows that the proposed model demonstrates better fusion ability than existing multi-exposure image fusion methods in both visual effect and evaluation metrics.

Sign in / Sign up

Export Citation Format

Share Document