scholarly journals Infrared and Visible Image Fusion with Hybrid Image Filtering

2020 ◽  
Vol 2020 ◽  
pp. 1-17
Author(s):  
Yongxin Zhang ◽  
Deguang Li ◽  
WenPeng Zhu
Keyword(s):  
Thermal Radiation ◽  
Image Fusion ◽  
Image Filtering ◽  
Divide And Conquer ◽  
Base Layer ◽  
Visible Image ◽  
Composite Image ◽  
Fusion Methods ◽  
Visible Images ◽  
Scene Information

Image fusion is an important technique aiming to generate a composite image from multiple images of the same scene. Infrared and visible images can provide the same scene information from different aspects, which is useful for target recognition. But the existing fusion methods cannot well preserve the thermal radiation and appearance information simultaneously. Thus, we propose an infrared and visible image fusion method by hybrid image filtering. We represent the fusion problem with a divide and conquer strategy. A Gaussian filter is used to decompose the source images into base layers and detail layers. An improved co-occurrence filter fuses the detail layers for preserving the thermal radiation of the source images. A guided filter fuses the base layers for retaining the background appearance information of the source images. Superposition of the fused base layer and fused detail layer generates the final fusion image. Subjective visual and objective quantitative evaluations comparing with other fusion algorithms demonstrate the better performance of the proposed method.

scholarly journals A Heterogeneous Image Fusion Method Based on DCT and Anisotropic Diffusion for UAVs in Future 5G IoT Scenarios

2020 ◽  
Vol 2020 ◽  
pp. 1-11
Author(s):  
Shuai Hao ◽  
Beiyi An ◽  
Hu Wen ◽  
Xu Ma ◽  
Keping Yu
Keyword(s):  
Image Fusion ◽  
Traffic Management ◽  
Anisotropic Diffusion ◽  
Weighted Averaging ◽  
Base Layer ◽  
Fusion Method ◽  
Linear Superposition ◽  
Visible Image ◽  
Fusion Methods ◽  
Visible Images

Unmanned aerial vehicles, with their inherent fine attributes, such as flexibility, mobility, and autonomy, play an increasingly important role in the Internet of Things (IoT). Airborne infrared and visible image fusion, which constitutes an important data basis for the perception layer of IoT, has been widely used in various fields such as electric power inspection, military reconnaissance, emergency rescue, and traffic management. However, traditional infrared and visible image fusion methods suffer from weak detail resolution. In order to better preserve useful information from source images and produce a more informative image for human observation or unmanned aerial vehicle vision tasks, a novel fusion method based on discrete cosine transform (DCT) and anisotropic diffusion is proposed. First, the infrared and visible images are denoised by using DCT. Second, anisotropic diffusion is applied to the denoised infrared and visible images to obtain the detail and base layers. Third, the base layers are fused by using weighted averaging, and the detail layers are fused by using the Karhunen–Loeve transform, respectively. Finally, the fused image is reconstructed through the linear superposition of the base layer and detail layer. Compared with six other typical fusion methods, the proposed approach shows better fusion performance in both objective and subjective evaluations.

scholarly journals Infrared and visible image fusion via octave Gaussian pyramid framework

2021 ◽  
Vol 11 (1) ◽  
Author(s):  
Lei Yan ◽  
Qun Hao ◽  
Jie Cao ◽  
Rizvi Saad ◽  
Kun Li ◽  
...  
Keyword(s):  
Image Fusion ◽  
Low Frequency ◽  
Objective Evaluation ◽  
Visible Image ◽  
Composite Image ◽  
Multiscale Decomposition ◽  
Gaussian Pyramid ◽  
Fusion Methods ◽  
Scale Spaces ◽  
Fusion Framework

AbstractImage fusion integrates information from multiple images (of the same scene) to generate a (more informative) composite image suitable for human and computer vision perception. The method based on multiscale decomposition is one of the commonly fusion methods. In this study, a new fusion framework based on the octave Gaussian pyramid principle is proposed. In comparison with conventional multiscale decomposition, the proposed octave Gaussian pyramid framework retrieves more information by decomposing an image into two scale spaces (octave and interval spaces). Different from traditional multiscale decomposition with one set of detail and base layers, the proposed method decomposes an image into multiple sets of detail and base layers, and it efficiently retains high- and low-frequency information from the original image. The qualitative and quantitative comparison with five existing methods (on publicly available image databases) demonstrate that the proposed method has better visual effects and scores the highest in objective evaluation.

scholarly journals Infrared and Visible Image Fusion Based on Nonlinear Enhancement and NSST Decomposition

2020 ◽  
Author(s):  
Xiaoxue XING ◽  
Cheng LIU ◽  
Cong LUO ◽  
Tingfa XU
Keyword(s):  
Thermal Radiation ◽  
High Frequency ◽  
Geometric Analysis ◽  
Low Frequency ◽  
Fusion Rule ◽  
Fusion Algorithm ◽  
Visible Image ◽  
Fused Image ◽  
Fusion Methods ◽  
Visible Images

Abstract In Multi-scale Geometric Analysis (MGA)-based fusion methods for infrared and visible images, adopting the same representation for the two types of the images will result in the non-obvious thermal radiation target in the fused image, which can hardly be distinguished from the background. To solve the problem, a novel fusion algorithm based on nonlinear enhancement and Non-Subsampled Shearlet Transform (NSST) decomposition is proposed. Firstly, NSST is used to decompose the two source images into low- and high-frequency sub-bands. Then, the Wavelet Transform (WT) is used to decompose high-frequency sub-bands into obtain approximate sub-bands and directional detail sub-bands. The “average” fusion rule is performed for fusion for approximate sub-bands. And the “max-absolute” fusion rule is performed for fusion for directional detail sub-bands. The inverse WT is used to reconstruct the high-frequency sub-bands. To highlight the thermal radiation target, we construct a non-linear transform function to determine the fusion weight of low-frequency sub-bands, and whose parameters can be further adjusted to meet different fusion requirements. Finally, the inverse NSST is used to reconstruct the fused image. The experimental results show that the proposed method can simultaneously enhance the thermal target in infrared images and preserve the texture details in visible images, and which is competitive with or even superior to the state-of-the-art fusion methods in terms of both visual and quantitative evaluations.

scholarly journals Infrared and Visible Image Fusion Using Truncated Huber Penalty Function Smoothing and Visual Saliency Based Threshold Optimization

Electronics ◽  
2021 ◽  
Vol 11 (1) ◽  
pp. 33
Author(s):  
Chaowei Duan ◽  
Yiliu Liu ◽  
Changda Xing ◽  
Zhisheng Wang
Keyword(s):  
Image Fusion ◽  
Penalty Function ◽  
Visual Saliency ◽  
Fusion Rule ◽  
Visible Image ◽  
Threshold Optimization ◽  
Fused Image ◽  
Fusion Methods ◽  
Visible Images ◽  
Image Representations

An efficient method for the infrared and visible image fusion is presented using truncated Huber penalty function smoothing and visual saliency based threshold optimization. The method merges complementary information from multimodality source images into a more informative composite image in two-scale domain, in which the significant objects/regions are highlighted and rich feature information is preserved. Firstly, source images are decomposed into two-scale image representations, namely, the approximate and residual layers, using truncated Huber penalty function smoothing. Benefiting from the edge- and structure-preserving characteristics, the significant objects and regions in the source images are effectively extracted without halo artifacts around the edges. Secondly, a visual saliency based threshold optimization fusion rule is designed to fuse the approximate layers aiming to highlight the salient targets in infrared images and remain the high-intensity regions in visible images. The sparse representation based fusion rule is adopted to fuse the residual layers with the goal of acquiring rich detail texture information. Finally, combining the fused approximate and residual layers reconstructs the fused image with more natural visual effects. Sufficient experimental results demonstrate that the proposed method can achieve comparable or superior performances compared with several state-of-the-art fusion methods in visual results and objective assessments.

scholarly journals Infrared and Visible Image Fusion Based on Nonlinear Enhancement and NSST Decomposition

2020 ◽  
Author(s):  
Xiaoxue XING ◽  
Cheng LIU ◽  
Cong LUO ◽  
Tingfa XU
Keyword(s):  
Thermal Radiation ◽  
High Frequency ◽  
Geometric Analysis ◽  
Low Frequency ◽  
Fusion Rule ◽  
Fusion Algorithm ◽  
Visible Image ◽  
Fused Image ◽  
Fusion Methods ◽  
Visible Images

Abstract In Multi-scale Geometric Analysis (MGA)-based fusion methods for infrared and visible images, adopting the same representation for the two types of the images will result in the non-obvious thermal radiation target in the fused image, which can hardly be distinguished from the background. To solve the problem, a novel fusion algorithm based on nonlinear enhancement and Non-Subsampled Shearlet Transform (NSST) decomposition is proposed. Firstly, NSST is used to decompose the two source images into low- and high-frequency sub-bands. Then, the wavelet transform(WT) is used to decompose high-frequency sub-bands into obtain approximate sub-bands and directional detail sub-bands. The “average” fusion rule is performed for fusion for approximate sub-bands. And the “max-absolute” fusion rule is performed for fusion for directional detail sub-bands. The inverse WT is used to reconstruct the high-frequency sub-bands. To highlight the thermal radiation target, we construct a non-linear transform function to determine the fusion weight of low-frequency sub-bands, and whose parameters can be further adjusted to meet different fusion requirements. Finally, the inverse NSST is used to reconstruct the fused image. The experimental results show that the proposed method can simultaneously enhance the thermal target in infrared images and preserve the texture details in visible images, and which is competitive with or even superior to the state-of-the-art fusion methods in terms of both visual and quantitative evaluations.

scholarly journals Fusing Infrared and Visible Images of Different Resolutions via Total Variation Model

Sensors ◽  
2018 ◽  
Vol 18 (11) ◽  
pp. 3827 ◽  
Author(s):  
Qinglei Du ◽  
Han Xu ◽  
Yong Ma ◽  
Jun Huang ◽  
Fan Fan
Keyword(s):  
Thermal Radiation ◽  
Total Variation ◽  
Infrared Image ◽  
Infrared Images ◽  
Visible Image ◽  
Pixel Intensity ◽  
Texture Information ◽  
Fused Image ◽  
Fusion Methods ◽  
Visible Images

In infrared and visible image fusion, existing methods typically have a prerequisite that the source images share the same resolution. However, due to limitations of hardware devices and application environments, infrared images constantly suffer from markedly lower resolution compared with the corresponding visible images. In this case, current fusion methods inevitably cause texture information loss in visible images or blur thermal radiation information in infrared images. Moreover, the principle of existing fusion rules typically focuses on preserving texture details in source images, which may be inappropriate for fusing infrared thermal radiation information because it is characterized by pixel intensities, possibly neglecting the prominence of targets in fused images. Faced with such difficulties and challenges, we propose a novel method to fuse infrared and visible images of different resolutions and generate high-resolution resulting images to obtain clear and accurate fused images. Specifically, the fusion problem is formulated as a total variation (TV) minimization problem. The data fidelity term constrains the pixel intensity similarity of the downsampled fused image with respect to the infrared image, and the regularization term compels the gradient similarity of the fused image with respect to the visible image. The fast iterative shrinkage-thresholding algorithm (FISTA) framework is applied to improve the convergence rate. Our resulting fused images are similar to super-resolved infrared images, which are sharpened by the texture information from visible images. Advantages and innovations of our method are demonstrated by the qualitative and quantitative comparisons with six state-of-the-art methods on publicly available datasets.

Infrared and visible image fusion via rolling guidance filter and convolutional sparse representation

2021 ◽  
pp. 1-14
Author(s):  
Feiqiang Liu ◽  
Lihui Chen ◽  
Lu Lu ◽  
Gwanggil Jeon ◽  
Xiaomin Yang
Keyword(s):  
Image Fusion ◽  
Sparse Representation ◽  
Objective Assessment ◽  
Base Layer ◽  
Fusion Technology ◽  
Complementary Information ◽  
Visible Image ◽  
Composite Image ◽  
Fused Image ◽  
Imaging Sensors

Infrared (IR) and visible (VIS) image fusion technology combines the complementary information of the same scene from IR and VIS imaging sensors to generate a composite image, which is beneficial to post image-processing tasks. In order to achieve good fusion performance, a method by combining rolling guidance filter (RGF) and convolutional sparse representation (CSR) is proposed. In the proposed method, RGF is performed on every pre-registered IR and VIS source images to obtain their detail layers and base layer. Then, the detail layers are fused with a serious of weighted coefficients produced by joint bilateral filer (JBF). The base layer is decomposed into a sub-detail-layer and a sub-base-layer. CSR is applied to fuse the sub-detail-layer and averaging strategy is used to fuse the sub-base-layer. Finally, the fused image is reconstructed by adding the fused detail layer and base layer. Experimental results demonstrate the superiority of our proposed method both in subjective and objective assessment.

scholarly journals Infrared and visible image fusion based on optimal segmenting and contour extraction

2021 ◽  
Vol 3 (3) ◽  
Author(s):  
Javad Abbasi Aghamaleki ◽  
Alireza Ghorbani
Keyword(s):  
Image Fusion ◽  
Infrared Image ◽  
Fusion Method ◽  
Visible Image ◽  
Output Image ◽  
Contour Lines ◽  
Fused Image ◽  
Fusion Methods ◽  
Visible Images ◽  
The Individual

AbstractImage fusion is the combining process of complementary information of multiple same scene images into an output image. The resultant output image that is named fused image, produces more precise description of the scene than any of the individual input images. In this paper, we propose a novel simple and fast strategy for infrared (IR) and visible images based on local important areas of IR image. The fusion method is completed in three step approach. Firstly, only the segmented regions in the infrared image is extracted. Next, the image fusion is applied on segmented area and finally, contour lines are also used to improve the quality of the results of the second step of fusion method. Using a publicly available database, the proposed method is evaluated and compared to the other fusion methods. The experimental results show the effectiveness of the proposed method compared to the state of the art methods.

Infrared and visible image fusion methods and applications: A survey

2019 ◽  
Vol 45 ◽  
pp. 153-178 ◽  
Author(s):  
Jiayi Ma ◽  
Yong Ma ◽  
Chang Li
Keyword(s):  
Image Fusion ◽  
Visible Image ◽  
Fusion Methods

scholarly journals Infrared and Visible Image Fusion Techniques Based on Deep Learning: A Review

Electronics ◽  
2020 ◽  
Vol 9 (12) ◽  
pp. 2162
Author(s):  
Changqi Sun ◽  
Cong Zhang ◽  
Naixue Xiong
Keyword(s):  
Deep Learning ◽  
Image Fusion ◽  
Research Work ◽  
Image Features ◽  
Future Research ◽  
Learning Methods ◽  
Visible Image ◽  
Advantages And Disadvantages ◽  
Fusion Methods ◽  
Future Work

Infrared and visible image fusion technologies make full use of different image features obtained by different sensors, retain complementary information of the source images during the fusion process, and use redundant information to improve the credibility of the fusion image. In recent years, many researchers have used deep learning methods (DL) to explore the field of image fusion and found that applying DL has improved the time-consuming efficiency of the model and the fusion effect. However, DL includes many branches, and there is currently no detailed investigation of deep learning methods in image fusion. In this work, this survey reports on the development of image fusion algorithms based on deep learning in recent years. Specifically, this paper first conducts a detailed investigation on the fusion method of infrared and visible images based on deep learning, compares the existing fusion algorithms qualitatively and quantitatively with the existing fusion quality indicators, and discusses various fusions. The main contribution, advantages, and disadvantages of the algorithm. Finally, the research status of infrared and visible image fusion is summarized, and future work has prospected. This research can help us realize many image fusion methods in recent years and lay the foundation for future research work.

Sign in / Sign up

Export Citation Format

Share Document