scholarly journals Hyperspectral and Multispectral Image Fusion by Deep Neural Network in a Self-Supervised Manner

2021 ◽  
Vol 13 (16) ◽  
pp. 3226
Author(s):  
Jianhao Gao ◽  
Jie Li ◽  
Menghui Jiang
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
High Resolution ◽  
Image Fusion ◽  
Multispectral Image ◽  
Hyperspectral Images ◽  
Training Dataset ◽  
Diffusion Method ◽  
Simple Diffusion ◽  
Fusion Methods

Compared with multispectral sensors, hyperspectral sensors obtain images with high- spectral resolution at the cost of spatial resolution, which constrains the further and precise application of hyperspectral images. An intelligent idea to obtain high-resolution hyperspectral images is hyperspectral and multispectral image fusion. In recent years, many studies have found that deep learning-based fusion methods outperform the traditional fusion methods due to the strong non-linear fitting ability of convolution neural network. However, the function of deep learning-based methods heavily depends on the size and quality of training dataset, constraining the application of deep learning under the situation where training dataset is not available or of low quality. In this paper, we introduce a novel fusion method, which operates in a self-supervised manner, to the task of hyperspectral and multispectral image fusion without training datasets. Our method proposes two constraints constructed by low-resolution hyperspectral images and fake high-resolution hyperspectral images obtained from a simple diffusion method. Several simulation and real-data experiments are conducted with several popular remote sensing hyperspectral data under the condition where training datasets are unavailable. Quantitative and qualitative results indicate that the proposed method outperforms those traditional methods by a large extent.

scholarly journals An Improved Pulse-Coupled Neural Network Model for Pansharpening

Sensors ◽  
2020 ◽  
Vol 20 (10) ◽  
pp. 2764 ◽  
Author(s):  
Xiaojun Li ◽  
Haowen Yan ◽  
Weiying Xie ◽  
Lu Kang ◽  
Yi Tian
Keyword(s):  
Neural Network ◽  
Image Fusion ◽  
Multispectral Image ◽  
Transform Domain ◽  
Human Visual Perception ◽  
Pulse Coupled Neural Network ◽  
Proposed Model ◽  
Spectral Fidelity ◽  
Fusion Methods ◽  
Injection Mechanism

Pulse-coupled neural network (PCNN) and its modified models are suitable for dealing with multi-focus and medical image fusion tasks. Unfortunately, PCNNs are difficult to directly apply to multispectral image fusion, especially when the spectral fidelity is considered. A key problem is that most fusion methods using PCNNs usually focus on the selection mechanism either in the space domain or in the transform domain, rather than a details injection mechanism, which is of utmost importance in multispectral image fusion. Thus, a novel pansharpening PCNN model for multispectral image fusion is proposed. The new model is designed to acquire the spectral fidelity in terms of human visual perception for the fusion tasks. The experimental results, examined by different kinds of datasets, show the suitability of the proposed model for pansharpening.

scholarly journals Effectiveness of transfer learning for enhancing tumor classification with a convolutional neural network on frozen sections

2020 ◽  
Vol 10 (1) ◽  
Author(s):  
Young-Gon Kim ◽  
Sungchul Kim ◽  
Cristina Eunbee Cho ◽  
In Hye Song ◽  
Hee Jin Lee ◽  
...  
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Convolutional Neural Network ◽  
Transfer Learning ◽  
Frozen Section ◽  
Medical Center ◽  
External Validation ◽  
Model Performance ◽  
Classification Model ◽  
Training Dataset

AbstractFast and accurate confirmation of metastasis on the frozen tissue section of intraoperative sentinel lymph node biopsy is an essential tool for critical surgical decisions. However, accurate diagnosis by pathologists is difficult within the time limitations. Training a robust and accurate deep learning model is also difficult owing to the limited number of frozen datasets with high quality labels. To overcome these issues, we validated the effectiveness of transfer learning from CAMELYON16 to improve performance of the convolutional neural network (CNN)-based classification model on our frozen dataset (N = 297) from Asan Medical Center (AMC). Among the 297 whole slide images (WSIs), 157 and 40 WSIs were used to train deep learning models with different dataset ratios at 2, 4, 8, 20, 40, and 100%. The remaining, i.e., 100 WSIs, were used to validate model performance in terms of patch- and slide-level classification. An additional 228 WSIs from Seoul National University Bundang Hospital (SNUBH) were used as an external validation. Three initial weights, i.e., scratch-based (random initialization), ImageNet-based, and CAMELYON16-based models were used to validate their effectiveness in external validation. In the patch-level classification results on the AMC dataset, CAMELYON16-based models trained with a small dataset (up to 40%, i.e., 62 WSIs) showed a significantly higher area under the curve (AUC) of 0.929 than those of the scratch- and ImageNet-based models at 0.897 and 0.919, respectively, while CAMELYON16-based and ImageNet-based models trained with 100% of the training dataset showed comparable AUCs at 0.944 and 0.943, respectively. For the external validation, CAMELYON16-based models showed higher AUCs than those of the scratch- and ImageNet-based models. Model performance for slide feasibility of the transfer learning to enhance model performance was validated in the case of frozen section datasets with limited numbers.

scholarly journals Orchard Mapping with Deep Learning Semantic Segmentation

Sensors ◽  
2021 ◽  
Vol 21 (11) ◽  
pp. 3813
Author(s):  
Athanasios Anagnostis ◽  
Aristotelis C. Tagarakis ◽  
Dimitrios Kateris ◽  
Vasileios Moysiadis ◽  
Claus Grøn Sørensen ◽  
...  
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Semantic Segmentation ◽  
Automated Detection ◽  
Aerial Images ◽  
Training Dataset ◽  
Field Boundary ◽  
Different Seasons ◽  
Detection And Localization ◽  
Different Levels

This study aimed to propose an approach for orchard trees segmentation using aerial images based on a deep learning convolutional neural network variant, namely the U-net network. The purpose was the automated detection and localization of the canopy of orchard trees under various conditions (i.e., different seasons, different tree ages, different levels of weed coverage). The implemented dataset was composed of images from three different walnut orchards. The achieved variability of the dataset resulted in obtaining images that fell under seven different use cases. The best-trained model achieved 91%, 90%, and 87% accuracy for training, validation, and testing, respectively. The trained model was also tested on never-before-seen orthomosaic images or orchards based on two methods (oversampling and undersampling) in order to tackle issues with out-of-the-field boundary transparent pixels from the image. Even though the training dataset did not contain orthomosaic images, it achieved performance levels that reached up to 99%, demonstrating the robustness of the proposed approach.

Hyperspectral and multispectral image fusion techniques for high resolution applications: a review

2021 ◽  
Author(s):  
Dioline Sara ◽  
Ajay Kumar Mandava ◽  
Arun Kumar ◽  
Shiny Duela ◽  
Anitha Jude
Keyword(s):  
High Resolution ◽  
Image Fusion ◽  
Multispectral Image

scholarly journals Automatic Handgun Detection with Deep Learning in Video Surveillance Images

2021 ◽  
Vol 11 (13) ◽  
pp. 6085
Author(s):  
Jesus Salido ◽  
Vanesa Lomas ◽  
Jesus Ruiz-Santaquiteria ◽  
Oscar Deniz
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Convolutional Neural Network ◽  
Video Surveillance ◽  
Automatic Detection ◽  
Public Spaces ◽  
Detection Methods ◽  
Training Dataset ◽  
Average Precision ◽  
Terrorist Acts

There is a great need to implement preventive mechanisms against shootings and terrorist acts in public spaces with a large influx of people. While surveillance cameras have become common, the need for monitoring 24/7 and real-time response requires automatic detection methods. This paper presents a study based on three convolutional neural network (CNN) models applied to the automatic detection of handguns in video surveillance images. It aims to investigate the reduction of false positives by including pose information associated with the way the handguns are held in the images belonging to the training dataset. The results highlighted the best average precision (96.36%) and recall (97.23%) obtained by RetinaNet fine-tuned with the unfrozen ResNet-50 backbone and the best precision (96.23%) and F1 score values (93.36%) obtained by YOLOv3 when it was trained on the dataset including pose information. This last architecture was the only one that showed a consistent improvement—around 2%—when pose information was expressly considered during training.

scholarly journals Comparative Analysis of Image Fusion Methods According to Spectral Responses of High-Resolution Optical Sensors

2014 ◽  
Vol 30 (2) ◽  
pp. 227-239 ◽  
Author(s):  
Ha-Seong Lee ◽  
Kwan-Young Oh ◽  
Hyung-Sup Jung
Keyword(s):  
Comparative Analysis ◽  
High Resolution ◽  
Image Fusion ◽  
Optical Sensors ◽  
Fusion Methods ◽  
Spectral Responses

scholarly journals Covid-19 detection via deep neural network and occlusion sensitivity maps

2021 ◽  
Author(s):  
Noor Ahmad ◽  
Muhammad Aminu ◽  
Mohd Halim Mohd Noor
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Deep Neural Network ◽  
State Of The Art ◽  
Color Images ◽  
Fine Tuning ◽  
Training Dataset ◽  
Learning Approaches ◽  
Learning Models ◽  
Sensitivity Maps

Deep learning approaches have attracted a lot of attention in the automatic detection of Covid-19 and transfer learning is the most common approach. However, majority of the pre-trained models are trained on color images, which can cause inefficiencies when fine-tuning the models on Covid-19 images which are often grayscale. To address this issue, we propose a deep learning architecture called CovidNet which requires a relatively smaller number of parameters. CovidNet accepts grayscale images as inputs and is suitable for training with limited training dataset. Experimental results show that CovidNet outperforms other state-of-the-art deep learning models for Covid-19 detection.

scholarly journals Encoding Health Records into Pathway Representations for Deep Learning

2021 ◽  
Author(s):  
Marco Luca Sbodio ◽  
Natasha Mulligan ◽  
Stefanie Speichert ◽  
Vanessa Lopez ◽  
Joao Bettencourt-Silva
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Deep Learning ◽  
Source Code ◽  
Training Dataset ◽  
Health Records ◽  
Learning Tasks ◽  
Patient Pathways ◽  
Computational Resources ◽  
The Impact

There is a growing trend in building deep learning patient representations from health records to obtain a comprehensive view of a patient’s data for machine learning tasks. This paper proposes a reproducible approach to generate patient pathways from health records and to transform them into a machine-processable image-like structure useful for deep learning tasks. Based on this approach, we generated over a million pathways from FAIR synthetic health records and used them to train a convolutional neural network. Our initial experiments show the accuracy of the CNN on a prediction task is comparable or better than other autoencoders trained on the same data, while requiring significantly less computational resources for training. We also assess the impact of the size of the training dataset on autoencoders performances. The source code for generating pathways from health records is provided as open source.

Digital object restoration using generalized regression neural network deep learning—Taking Dunhuang mural restoration as an example

2020 ◽  
pp. 002072092092854
Author(s):  
Wenjing She
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Image Restoration ◽  
Anisotropic Diffusion ◽  
Learning Algorithm ◽  
Diffusion Method ◽  
Generalized Regression Neural Network ◽  
Deep Learning Algorithm ◽  
Training Samples ◽  
Restoration Effect

In this research, Dunhuang murals is taken as the object of restoration, and the role of digital repair combined with deep learning algorithm in mural restoration is explored. First, the image restoration technology is described, as well as its advantages and disadvantages are analyzed. Second, the deep learning algorithm based on artificial neural network is described and analyzed. Finally, the deep learning algorithm is integrated into the digital repair technology, and a mural restoration method based on the generalized regression neural network is proposed. The morphological expansion method and anisotropic diffusion method are used to preprocess the image. The MATLAB software is used for the simulation analysis and evaluation of the image restoration effect. The results show that in the restoration of the original image, the accuracy of the digital image restoration technology is not high. The nontexture restoration technology is not applicable in the repair of large-scale texture areas. The predicted value of the mural restoration effect based on the generalized neural network is closer to the true value. The anisotropic diffusion method has a significant effect on the processing of image noise. In the image similarity rate, the different number of training samples and smoothing parameters are compared and analyzed. It is found that when the value of δ is small, the number of training samples should be increased to improve the accuracy of the prediction value. If the number of training samples is small, a larger value of δ is needed to get a better prediction effect, and the best restoration effect is obtained for the restored image. Through this study, it is found that this study has a good effect on the restoration model of Dunhuang murals. It provides experimental reference for the restoration of later murals.

Sign in / Sign up

Export Citation Format

Share Document