Deep Learning Thermal Image Translation for Night Vision Perception

Recent advances in deep learning models have shown promising potential in object removal, which is the task of replacing undesired objects with appropriate pixel values using known context. Object removal-based deep learning can commonly be solved by modeling it as the Img2Img (image to image) translation or Inpainting. Instead of dealing with a large context, this paper aims at a specific application of object removal, that is, erasing braces trace out of an image having teeth with braces (called braces2teeth problem). We solved the problem by three methods corresponding to different datasets. Firstly, we use the CycleGAN model to deal with the problem that paired training data is not available. In the second case, we try to create pseudo-paired data to train the Pix2Pix model. In the last case, we utilize GraphCut combining generative inpainting model to build a user-interactive tool that can improve the result in case the user is not satisfied with previous results. To our best knowledge, this study is one of the first attempts to take the braces2teeth problem into account by using deep learning techniques and it can be applied in various fields, from health care to entertainment.

Download Full-text

An Intelligent Microwave Oven with Thermal Imaging and Temperature Recommendation Using Deep Learning

Applied System Innovation ◽

10.3390/asi3010013 ◽

2020 ◽

Vol 3 (1) ◽

pp. 13 ◽

Cited By ~ 2

Author(s):

Tareq Khan

Keyword(s):

Deep Learning ◽

Real Time ◽

Thermal Imaging ◽

Closed Loop ◽

Thermal Image ◽

Microwave Oven ◽

Closed Loop Control ◽

Target Temperature ◽

Thermal Camera ◽

Loop Control

Whenever food in a microwave oven is heated, the user estimates the time to heat. This estimation can be incorrect, leading the food to be too hot or still cold. In this research, an intelligent microwave oven is designed. After the food is put into the microwave oven and the door is closed, it captures the image of the food, classifies the image and then suggests the food’s target temperature by learning from previous experiences, so the user does not have to recall the target food temperature each time the same food is warmed. The temperature of the food is measured using a thermal camera. The proposed microwave incorporates a display to show a real-time colored thermal image of the food. The microwave automatically stops the heating when the temperature of the food hits the target temperature using closed-loop control. The deep learning-based image classifier gradually learns the type of foods that are consumed in that household and becomes smarter in temperature recommendation. The system can classify and recommend target temperature with 93% accuracy. A prototype is developed using a microcontroller-based system and successfully tested.

Download Full-text

Deep learning for thermal image segmentation to measure canopy temperature of Brassica oleracea in the field

10.13031/aim.201800305 ◽

2018 ◽

Cited By ~ 1

Author(s):

Yu Jiang ◽

Lanshuan Shuang ◽

Changying Li ◽

Andrew H. Paterson ◽

Jon Robertson

Keyword(s):

Image Segmentation ◽

Deep Learning ◽

Brassica Oleracea ◽

Canopy Temperature ◽

Thermal Image

Download Full-text

Visible and Thermal Image-Based Trunk Detection with Deep Learning for Forestry Mobile Robotics

Journal of Imaging ◽

10.3390/jimaging7090176 ◽

2021 ◽

Vol 7 (9) ◽

pp. 176

Author(s):

Daniel Queirós da Silva ◽

Filipe Neves dos Santos ◽

Armando Jorge Sousa ◽

Vítor Filipe

Keyword(s):

Deep Learning ◽

Object Detection ◽

Forest Inventory ◽

Mobile Robotics ◽

Ground Level ◽

Forest Tree ◽

Thermal Image ◽

Detection Methods ◽

Site Management ◽

Forest Wildfires

Mobile robotics in forests is currently a hugely important topic due to the recurring appearance of forest wildfires. Thus, in-site management of forest inventory and biomass is required. To tackle this issue, this work presents a study on detection at the ground level of forest tree trunks in visible and thermal images using deep learning-based object detection methods. For this purpose, a forestry dataset composed of 2895 images was built and made publicly available. Using this dataset, five models were trained and benchmarked to detect the tree trunks. The selected models were SSD MobileNetV2, SSD Inception-v2, SSD ResNet50, SSDLite MobileDet and YOLOv4 Tiny. Promising results were obtained; for instance, YOLOv4 Tiny was the best model that achieved the highest AP (90%) and F1 score (89%). The inference time was also evaluated, for these models, on CPU and GPU. The results showed that YOLOv4 Tiny was the fastest detector running on GPU (8 ms). This work will enhance the development of vision perception systems for smarter forestry robots.

Download Full-text

Deep Learning-Based Thermal Image Analysis for Pavement Defect Detection and Classification Considering Complex Pavement Conditions

Remote Sensing ◽

10.3390/rs14010106 ◽

2021 ◽

Vol 14 (1) ◽

pp. 106

Author(s):

Cheng Chen ◽

Sindhu Chandra ◽

Yufan Han ◽

Hyungjoon Seo

Keyword(s):

Deep Learning ◽

Damage Detection ◽

Defect Detection ◽

Input Data ◽

Thermal Image ◽

Detection Accuracy ◽

Detection Model ◽

Fused Image ◽

Data Source ◽

Pavement Damage

Automatic damage detection using deep learning warrants an extensive data source that captures complex pavement conditions. This paper proposes a thermal-RGB fusion image-based pavement damage detection model, wherein the fused RGB-thermal image is formed through multi-source sensor information to achieve fast and accurate defect detection including complex pavement conditions. The proposed method uses pre-trained EfficientNet B4 as the backbone architecture and generates an argument dataset (containing non-uniform illumination, camera noise, and scales of thermal images too) to achieve high pavement damage detection accuracy. This paper tests separately the performance of different input data (RGB, thermal, MSX, and fused image) to test the influence of input data and network on the detection results. The results proved that the fused image’s damage detection accuracy can be as high as 98.34% and by using the dataset after augmentation, the detection model deems to be more stable to achieve 98.35% precision, 98.34% recall, and 98.34% F1-score.

Download Full-text

Heterogeneous Change Detection on Remote Sensing Data with Self-Supervised Deep Canonically Correlated Autoencoders

Septentrio Reports ◽

10.7557/7.5763 ◽

2021 ◽

Author(s):

Federico Figari Tomenotti

Keyword(s):

Remote Sensing ◽

Deep Learning ◽

Change Detection ◽

Remote Sensing Data ◽

Sensing Data ◽

Learning Framework ◽

Novel Approach ◽

Image Translation ◽

Recent Developments ◽

Different Sources

Change detection is a well-known topic of remote sensing. The goal is to track and monitor the evolution of changes affecting the Earth surface over time. The recently increased availability in remote sensing data for Earth observation and in computational power has raised the interest in this field of research. In particular, the keywords “multitemporal” and “heterogeneous” play prominent roles. The former refers to the availability and the comparison of two or more satellite images of the same place on the ground, in order to find changes and track the evolution of the observed surface, maybe with different time sensitivities. The latter refers to the capability of performing change detection with images coming from different sources, corresponding to different sensors, wavelengths, polarizations, acquisition geometries, etc. This thesis addresses the challenging topic of multitemporal change detection with heterogeneous remote sensing images. It proposes a novel approach, taking inspiration from recent developments in the literature. The proposed method is based on deep learning - involving autoencoders of convolutional neural networks - and represents an exapmple of unsupervised change detection. A major novelty of the work consists in including a prior information model, used to make the method unsupervised, within a well-established algorithm such as the canonical correlation analysis, and in combining these with a deep learning framework to give rise to an image translation method able to compare heterogeneous images regardless of their highly different domains. The theoretical analysis is supported by experimental results, comparing the proposed methodology to the state of the art of this discipline. Two different datasets were used for the experiments, and the results obtained on both of them show the effectiveness of the proposed method.

Download Full-text

Using CycleGAN to Achieve the Sketch Recognition Process of Sketch-Based Modeling

10.1007/978-981-16-5983-6_3 ◽

2021 ◽

pp. 26-34

Author(s):

Yuqian Li ◽

Weiguo Xu

Keyword(s):

Artificial Intelligence ◽

Neural Networks ◽

Deep Learning ◽

Learning Technology ◽

Modeling Technology ◽

Image Translation ◽

Mapping Process ◽

And Training ◽

Difficult Part ◽

2D Sketches

AbstractArchitects usually design ideation and conception by hand-sketching. Sketching is a direct expression of the architect’s creativity. But 2D sketches are often vague, intentional and even ambiguous. In the research of sketch-based modeling, it is the most difficult part to make the computer to recognize the sketches. Because of the development of artificial intelligence, especially deep learning technology, Convolutional Neural Networks (CNNs) have shown obvious advantages in the field of extracting features and matching, and Generative Adversarial Neural Networks (GANs) have made great breakthroughs in the field of architectural generation which make the image-to-image translation become more and more popular. As the building images are gradually developed from the original sketches, in this research, we try to develop a system from the sketches to the images of buildings using CycleGAN algorithm. The experiment demonstrates that this method could achieve the mapping process from the sketches to images, and the results show that the sketches’ features could be recognised in the process. By the learning and training process of the sketches’ reconstruction, the features of the images are also mapped to the sketches, which strengthen the architectural relationship in the sketch, so that the original sketch can gradually approach the building images, and then it is possible to achieve the sketch-based modeling technology.

Download Full-text