scholarly journals Wildfire Smoke Classification Based on Synthetic Images and Pixel- and Feature-Level Domain Adaptation

Sensors ◽  
2021 ◽  
Vol 21 (23) ◽  
pp. 7785
Author(s):  
Jun Mao ◽  
Change Zheng ◽  
Jiyan Yin ◽  
Ye Tian ◽  
Wenbin Cui
Keyword(s):  
Deep Learning ◽  
Domain Adaptation ◽  
Synthetic Data ◽  
Real Data ◽  
Training Image ◽  
Classification Model ◽  
Environmental Diversity ◽  
Wildfire Smoke ◽  
Synthetic Images ◽  
Adaptation Method

Training a deep learning-based classification model for early wildfire smoke images requires a large amount of rich data. However, due to the episodic nature of fire events, it is difficult to obtain wildfire smoke image data, and most of the samples in public datasets suffer from a lack of diversity. To address these issues, a method using synthetic images to train a deep learning classification model for real wildfire smoke was proposed in this paper. Firstly, we constructed a synthetic dataset by simulating a large amount of morphologically rich smoke in 3D modeling software and rendering the virtual smoke against many virtual wildland background images with rich environmental diversity. Secondly, to better use the synthetic data to train a wildfire smoke image classifier, we applied both pixel-level domain adaptation and feature-level domain adaptation. The CycleGAN-based pixel-level domain adaptation method for image translation was employed. On top of this, the feature-level domain adaptation method incorporated ADDA with DeepCORAL was adopted to further reduce the domain shift between the synthetic and real data. The proposed method was evaluated and compared on a test set of real wildfire smoke and achieved an accuracy of 97.39%. The method is applicable to wildfire smoke classification tasks based on RGB single-frame images and would also contribute to training image classification models without sufficient data.

scholarly journals Deep Learning Exoplanets Detection by Combining Real and Synthetic Data

2021 ◽  
Author(s):  
Sara Cuéllar ◽  
Paulo Granados ◽  
Ernesto Fabregas ◽  
Michel Curé ◽  
Hector Vargas ◽  
...  
Keyword(s):  
Deep Learning ◽  
Synthetic Data ◽  
Real Data ◽  
True Positive Rate ◽  
Learning System ◽  
Classification Model ◽  
Training Stage ◽  
Positive Rate ◽  
Optimisation Technique ◽  
Planetary Transits

Scientists and astronomers have attached Scientists and astronomers have attached great importance to the task of discovering new exoplanets, even more so if they are in the habitable zone. To date, more than 4300 exoplanets have been confirmed by NASA, using various discovery techniques, including planetary transits, in addition to the use of various databases provided by space and ground-based telescopes. This article proposes the development of a deep learning system for detecting planetary transits in Kepler Telescope lightcurves. The approach is based on related work from the literature and enhanced to validation with real lightcurves. A CNN classification model is trained from a mixture of real and synthetic data, and validated only with real data and different from those used in the training stage. The best ratio of synthetic data is determined by the perform of an optimisation technique and a sensitivity analysis. The precision, accuracy and true positive rate of the best model obtained are determined and compared with other similar works. The results demonstrate that the use of synthetic data on the training stage can improve the transit detection performance on real light curves.

scholarly journals Real UAV-Bird Image Classification Using CNN with a Synthetic Dataset

2021 ◽  
Vol 11 (9) ◽  
pp. 3863
Author(s):  
Ali Emre Öztürk ◽  
Ergun Erçelebi
Keyword(s):  
Deep Learning ◽  
Image Classification ◽  
Synthetic Data ◽  
Real Data ◽  
Corner Detection ◽  
Batch Size ◽  
Test Accuracy ◽  
Classification Problems ◽  
Auc Value ◽  
Classification Test

A large amount of training image data is required for solving image classification problems using deep learning (DL) networks. In this study, we aimed to train DL networks with synthetic images generated by using a game engine and determine the effects of the networks on performance when solving real-image classification problems. The study presents the results of using corner detection and nearest three-point selection (CDNTS) layers to classify bird and rotary-wing unmanned aerial vehicle (RW-UAV) images, provides a comprehensive comparison of two different experimental setups, and emphasizes the significant improvements in the performance in deep learning-based networks due to the inclusion of a CDNTS layer. Experiment 1 corresponds to training the commonly used deep learning-based networks with synthetic data and an image classification test on real data. Experiment 2 corresponds to training the CDNTS layer and commonly used deep learning-based networks with synthetic data and an image classification test on real data. In experiment 1, the best area under the curve (AUC) value for the image classification test accuracy was measured as 72%. In experiment 2, using the CDNTS layer, the AUC value for the image classification test accuracy was measured as 88.9%. A total of 432 different combinations of trainings were investigated in the experimental setups. The experiments were trained with various DL networks using four different optimizers by considering all combinations of batch size, learning rate, and dropout hyperparameters. The test accuracy AUC values for networks in experiment 1 ranged from 55% to 74%, whereas the test accuracy AUC values in experiment 2 networks with a CDNTS layer ranged from 76% to 89.9%. It was observed that the CDNTS layer has considerable effects on the image classification accuracy performance of deep learning-based networks. AUC, F-score, and test accuracy measures were used to validate the success of the networks.

scholarly journals Vision-Based Multirotor Following Using Synthetic Learning Techniques

Sensors ◽  
2019 ◽  
Vol 19 (21) ◽  
pp. 4794
Author(s):  
Alejandro Rodriguez-Ramos ◽  
Adrian Alvarez-Fernandez ◽  
Hriday Bavle ◽  
Pascual Campoy ◽  
Jonathan P. How
Keyword(s):  
Reinforcement Learning ◽  
Object Detection ◽  
Learning Strategies ◽  
Motion Control ◽  
Domain Adaptation ◽  
Control Strategies ◽  
Synthetic Data ◽  
Real Data ◽  
Stable Convergence ◽  
Learning Techniques

Deep- and reinforcement-learning techniques have increasingly required large sets of real data to achieve stable convergence and generalization, in the context of image-recognition, object-detection or motion-control strategies. On this subject, the research community lacks robust approaches to overcome unavailable real-world extensive data by means of realistic synthetic-information and domain-adaptation techniques. In this work, synthetic-learning strategies have been used for the vision-based autonomous following of a noncooperative multirotor. The complete maneuver was learned with synthetic images and high-dimensional low-level continuous robot states, with deep- and reinforcement-learning techniques for object detection and motion control, respectively. A novel motion-control strategy for object following is introduced where the camera gimbal movement is coupled with the multirotor motion during the multirotor following. Results confirm that our present framework can be used to deploy a vision-based task in real flight using synthetic data. It was extensively validated in both simulated and real-flight scenarios, providing proper results (following a multirotor up to 1.3 m/s in simulation and 0.3 m/s in real flights).

scholarly journals 3dinfogan: 3d Models’ Reconstruction In Infogans

2021 ◽  
Vol 10 (02) ◽  
pp. 95-109
Author(s):  
Du Chunqi ◽  
Shinobu Hasegawa
Keyword(s):  
Computer Vision ◽  
Deep Learning ◽  
Latent Variables ◽  
Data Augmentation ◽  
Synthetic Data ◽  
Real Data ◽  
3D Models ◽  
High Quality ◽  
Shape Constraint ◽  
Real Objects

In computer vision and computer graphics, 3D reconstruction is the process of capturing real objects’ shapes and appearances. 3D models always can be constructed by active methods which use high-quality scanner equipment, or passive methods that learn from the dataset. However, both of these two methods only aimed to construct the 3D models, without showing what element affects the generation of 3D models. Therefore, the goal of this research is to apply deep learning to automatically generating 3D models, and finding the latent variables which affect the reconstructing process. The existing research GANs can be trained in little data with two networks called Generator and Discriminator, respectively. Generator can produce synthetic data, and Discriminator can discriminate between the generator’s output and real data. The existing research shows that InFoGAN can maximize the mutual information between latent variables and observation. In our approach, we will generate the 3D models based on InFoGAN and design two constraints, shape-constraint and parameters-constraint, respectively. Shape-constraint utilizes the data augmentation method to limit the synthetic data generated in the models’ profiles. At the same time, we also try to employ parameters-constraint to find the 3D models’ relationship corresponding to the latent variables. Furthermore, our approach will be a challenge in the architecture of generating 3D models built on InFoGAN. Finally, in the process of generation, we might discover the contribution of the latent variables influencing the 3D models to the whole network.

scholarly journals Easy domain adaptation method for filling the species gap in deep learning-based fruit detection

2021 ◽  
Vol 8 (1) ◽  
Author(s):  
Wenli Zhang ◽  
Kaizhen Chen ◽  
Jiaqi Wang ◽  
Yun Shi ◽  
Wei Guo
Keyword(s):  
Deep Learning ◽  
Technology Development ◽  
Domain Adaptation ◽  
Training Dataset ◽  
Target Domain ◽  
Detection Techniques ◽  
Image Dataset ◽  
Detection And Counting ◽  
Computer Vision Technology ◽  
Adaptation Method

AbstractFruit detection and counting are essential tasks for horticulture research. With computer vision technology development, fruit detection techniques based on deep learning have been widely used in modern orchards. However, most deep learning-based fruit detection models are generated based on fully supervised approaches, which means a model trained with one domain species may not be transferred to another. There is always a need to recreate and label the relevant training dataset, but such a procedure is time-consuming and labor-intensive. This paper proposed a domain adaptation method that can transfer an existing model trained from one domain to a new domain without extra manual labeling. The method includes three main steps: transform the source fruit image (with labeled information) into the target fruit image (without labeled information) through the CycleGAN network; Automatically label the target fruit image by a pseudo-label process; Improve the labeling accuracy by a pseudo-label self-learning approach. Use a labeled orange image dataset as the source domain, unlabeled apple and tomato image dataset as the target domain, the performance of the proposed method from the perspective of fruit detection has been evaluated. Without manual labeling for target domain image, the mean average precision reached 87.5% for apple detection and 76.9% for tomato detection, which shows that the proposed method can potentially fill the species gap in deep learning-based fruit detection.

scholarly journals Synthetic Source Universal Domain Adaptation through Contrastive Learning

Sensors ◽  
2021 ◽  
Vol 21 (22) ◽  
pp. 7539
Author(s):  
Jungchan Cho
Keyword(s):  
Domain Adaptation ◽  
Synthetic Data ◽  
Real Data ◽  
Target Domain ◽  
Source Domain ◽  
Universal Domain ◽  
Model Training ◽  
Using Data ◽  
Imaging Sensors ◽  
Target Data

Universal domain adaptation (UDA) is a crucial research topic for efficient deep learning model training using data from various imaging sensors. However, its development is affected by unlabeled target data. Moreover, the nonexistence of prior knowledge of the source and target domain makes it more challenging for UDA to train models. I hypothesize that the degradation of trained models in the target domain is caused by the lack of direct training loss to improve the discriminative power of the target domain data. As a result, the target data adapted to the source representations is biased toward the source domain. I found that the degradation was more pronounced when I used synthetic data for the source domain and real data for the target domain. In this paper, I propose a UDA method with target domain contrastive learning. The proposed method enables models to leverage synthetic data for the source domain and train the discriminativeness of target features in an unsupervised manner. In addition, the target domain feature extraction network is shared with the source domain classification task, preventing unnecessary computational growth. Extensive experimental results on VisDa-2017 and MNIST to SVHN demonstrated that the proposed method significantly outperforms the baseline by 2.7% and 5.1%, respectively.

scholarly journals DOMAIN ADAPTATION WITH CYCLEGAN FOR CHANGE DETECTION IN THE AMAZON FOREST

2020 ◽  
Vol XLIII-B3-2020 ◽  
pp. 1635-1643
Author(s):  
P. J. Soto ◽  
G. A. O. P. Costa ◽  
R. Q. Feitosa ◽  
P. N. Happ ◽  
M. X. Ortega ◽  
...  
Keyword(s):  
Remote Sensing ◽  
Deep Learning ◽  
Change Detection ◽  
Domain Adaptation ◽  
Fine Tuning ◽  
Classification Model ◽  
Generative Adversarial Networks ◽  
Amazon Forest ◽  
Sensing Applications ◽  
Remote Sensing Applications

Abstract. Deep learning classification models require large amounts of labeled training data to perform properly, but the production of reference data for most Earth observation applications is a labor intensive, costly process. In that sense, transfer learning is an option to mitigate the demand for labeled data. In many remote sensing applications, however, the accuracy of a deep learning-based classification model trained with a specific dataset drops significantly when it is tested on a different dataset, even after fine-tuning. In general, this behavior can be credited to the domain shift phenomenon. In remote sensing applications, domain shift can be associated with changes in the environmental conditions during the acquisition of new data, variations of objects’ appearances, geographical variability and different sensor properties, among other aspects. In recent years, deep learning-based domain adaptation techniques have been used to alleviate the domain shift problem. Recent improvements in domain adaptation technology rely on techniques based on Generative Adversarial Networks (GANs), such as the Cycle-Consistent Generative Adversarial Network (CycleGAN), which adapts images across different domains by learning nonlinear mapping functions between the domains. In this work, we exploit the CycleGAN approach for domain adaptation in a particular change detection application, namely, deforestation detection in the Amazon forest. Experimental results indicate that the proposed approach is capable of alleviating the effects associated with domain shift in the context of the target application.

scholarly journals Recognizing New Classes with Synthetic Data in the Loop: Application to Traffic Sign Recognition

Sensors ◽  
2020 ◽  
Vol 20 (3) ◽  
pp. 583
Author(s):  
Gabriel Villalonga ◽  
Joost Van de Weijer ◽  
Antonio M. López
Keyword(s):  
Domain Adaptation ◽  
Synthetic Data ◽  
Recognition System ◽  
Traffic Sign Recognition ◽  
Traffic Sign ◽  
Generative Adversarial Network ◽  
Adversarial Network ◽  
Sign Recognition ◽  
Short Period ◽  
Synthetic Images

On-board vision systems may need to increase the number of classes that can be recognized in a relatively short period. For instance, a traffic sign recognition system may suddenly be required to recognize new signs. Since collecting and annotating samples of such new classes may need more time than we wish, especially for uncommon signs, we propose a method to generate these samples by combining synthetic images and Generative Adversarial Network (GAN) technology. In particular, the GAN is trained on synthetic and real-world samples from known classes to perform synthetic-to-real domain adaptation, but applied to synthetic samples of the new classes. Using the Tsinghua dataset with a synthetic counterpart, SYNTHIA-TS, we have run an extensive set of experiments. The results show that the proposed method is indeed effective, provided that we use a proper Convolutional Neural Network (CNN) to perform the traffic sign recognition (classification) task as well as a proper GAN to transform the synthetic images. Here, a ResNet101-based classifier and domain adaptation based on CycleGAN performed extremely well for a ratio ∼ 1 / 4 for new/known classes; even for more challenging ratios such as ∼ 4 / 1 , the results are also very positive.

scholarly journals Advancing Medical Imaging Informatics by Deep Learning-Based Domain Adaptation

2020 ◽  
Vol 29 (01) ◽  
pp. 129-138 ◽  
Author(s):  
Anirudh Choudhary ◽  
Li Tong ◽  
Yuanda Zhu ◽  
May D. Wang
Keyword(s):  
Deep Learning ◽  
Medical Imaging ◽  
Large Scale ◽  
Domain Adaptation ◽  
Rapid Development ◽  
Synthetic Data ◽  
Feature Space ◽  
Imaging Informatics ◽  
Training Data ◽  
Domain Transformation

Introduction: There has been a rapid development of deep learning (DL) models for medical imaging. However, DL requires a large labeled dataset for training the models. Getting large-scale labeled data remains a challenge, and multi-center datasets suffer from heterogeneity due to patient diversity and varying imaging protocols. Domain adaptation (DA) has been developed to transfer the knowledge from a labeled data domain to a related but unlabeled domain in either image space or feature space. DA is a type of transfer learning (TL) that can improve the performance of models when applied to multiple different datasets. Objective: In this survey, we review the state-of-the-art DL-based DA methods for medical imaging. We aim to summarize recent advances, highlighting the motivation, challenges, and opportunities, and to discuss promising directions for future work in DA for medical imaging. Methods: We surveyed peer-reviewed publications from leading biomedical journals and conferences between 2017-2020, that reported the use of DA in medical imaging applications, grouping them by methodology, image modality, and learning scenarios. Results: We mainly focused on pathology and radiology as application areas. Among various DA approaches, we discussed domain transformation (DT) and latent feature-space transformation (LFST). We highlighted the role of unsupervised DA in image segmentation and described opportunities for future development. Conclusion: DA has emerged as a promising solution to deal with the lack of annotated training data. Using adversarial techniques, unsupervised DA has achieved good performance, especially for segmentation tasks. Opportunities include domain transferability, multi-modal DA, and applications that benefit from synthetic data.

Sign in / Sign up

Export Citation Format

Share Document