Generalization Capability of Deep Learning

2022 ◽  
pp. 243-266
Author(s):  
Jong Chul Ye
Keyword(s):  
Deep Learning ◽  
Generalization Capability

scholarly journals Meta-FSEO: A Meta-Learning Fast Adaptation with Self-Supervised Embedding Optimization for Few-Shot Remote Sensing Scene Classification

2021 ◽  
Vol 13 (14) ◽  
pp. 2776
Author(s):  
Yong Li ◽  
Zhenfeng Shao ◽  
Xiao Huang ◽  
Bowen Cai ◽  
Song Peng
Keyword(s):  
Remote Sensing ◽  
Deep Learning ◽  
Remote Sensing Data ◽  
Scene Classification ◽  
Generalization Capability ◽  
Urban Scenes ◽  
Shot Classification ◽  
Meta Learning ◽  
Land Surfaces ◽  
Fast Adaptation

The performance of deep learning is heavily influenced by the size of the learning samples, whose labeling process is time consuming and laborious. Deep learning algorithms typically assume that the training and prediction data are independent and uniformly distributed, which is rarely the case given the attributes and properties of different data sources. In remote sensing images, representations of urban land surfaces can vary across regions and by season, demanding rapid generalization of these surfaces in remote sensing data. In this study, we propose Meta-FSEO, a novel model for improving the performance of few-shot remote sensing scene classification in varying urban scenes. The proposed Meta-FSEO model deploys self-supervised embedding optimization for adaptive generalization in new tasks such as classifying features in new urban regions that have never been encountered during the training phase, thus balancing the requirements for feature classification tasks between multiple images collected at different times and places. We also created a loss function by weighting the contrast losses and cross-entropy losses. The proposed Meta-FSEO demonstrates a great generalization capability in remote sensing scene classification among different cities. In a five-way one-shot classification experiment with the Sentinel-1/2 Multi-Spectral (SEN12MS) dataset, the accuracy reached 63.08%. In a five-way five-shot experiment on the same dataset, the accuracy reached 74.29%. These results indicated that the proposed Meta-FSEO model outperformed both the transfer learning-based algorithm and two popular meta-learning-based methods, i.e., MAML and Meta-SGD.

scholarly journals A Novel YOLOv3 Algorithm-Based Deep Learning Approach for Waste Segregation: Towards Smart Waste Management

Electronics ◽  
2020 ◽  
Vol 10 (1) ◽  
pp. 14
Author(s):  
Saurav Kumar ◽  
Drishti Yadav ◽  
Himanshu Gupta ◽  
Om Prakash Verma ◽  
Irshad Ahmad Ansari ◽  
...  
Keyword(s):  
Deep Learning ◽  
Waste Management ◽  
Organic Waste ◽  
Learning Strategy ◽  
Smart Cities ◽  
Learning Approach ◽  
Generalization Capability ◽  
Novel Approach ◽  
Waste Segregation ◽  
Glass Metal

The colossal increase in environmental pollution and degradation, resulting in ecological imbalance, is an eye-catching concern in the contemporary era. Moreover, the proliferation in the development of smart cities across the globe necessitates the emergence of a robust smart waste management system for proper waste segregation based on its biodegradability. The present work investigates a novel approach for waste segregation for its effective recycling and disposal by utilizing a deep learning strategy. The YOLOv3 algorithm has been utilized in the Darknet neural network framework to train a self-made dataset. The network has been trained for 6 object classes (namely: cardboard, glass, metal, paper, plastic and organic waste). Moreover, for comparative assessment, the detection task has also been performed using YOLOv3-tiny to validate the competence of the YOLOv3 algorithm. The experimental results demonstrate that the proposed YOLOv3 methodology yields satisfactory generalization capability for all the classes with a variety of waste items.

scholarly journals A CNN-Based System for Mobile Robot Navigation in Indoor Environments via Visual Localization with a Small Dataset

2021 ◽  
Vol 12 (3) ◽  
pp. 134
Author(s):  
Farzin Foroughi ◽  
Zonghai Chen ◽  
Jikai Wang
Keyword(s):  
Deep Learning ◽  
Mobile Robot ◽  
Loss Function ◽  
Input Data ◽  
Robot Navigation ◽  
Mobile Robot Navigation ◽  
Visual Localization ◽  
Generalization Capability ◽  
Feature Maps ◽  
Topological Map

Deep learning has made great advances in the field of image processing, which allows automotive devices to be more widely used in humans’ daily lives than ever before. Nowadays, the mobile robot navigation system is among the hottest topics that researchers are trying to develop by adopting deep learning methods. In this paper, we present a system that allows the mobile robot to localize and navigate autonomously in the accessible areas of an indoor environment. The proposed system exploits the Convolutional Neural Network (CNN) model’s advantage to extract data feature maps for image classification and visual localization, which attempts to precisely determine the location region of the mobile robot focusing on the topological maps of the real environment. The system attempts to precisely determine the location region of the mobile robot by integrating the CNN model and topological map of the robot workspace. A dataset with small numbers of images is acquired from the MYNT EYE camera. Furthermore, we introduce a new loss function to tackle the bounded generalization capability of the CNN model in small datasets. The proposed loss function not only considers the probability of the input data when it is allocated to its true class but also considers the probability of allocating the input data to other classes rather than its actual class. We investigate the capability of the proposed system by evaluating the empirical studies based on provided datasets. The results illustrate that the proposed system outperforms other state-of-the-art techniques in terms of accuracy and generalization capability.

scholarly journals Random CNN Structure – Tool to Increase Generalization Ability in Deep Learning

2021 ◽  
Author(s):  
Bartosz Swiderski ◽  
Stanislaw Osowski ◽  
Grzegorz Gwardys ◽  
Jaroslaw Kurek ◽  
Monika Slowinska ◽  
...  
Keyword(s):  
Image Processing ◽  
Deep Learning ◽  
Quality Measures ◽  
Activation Function ◽  
Small Population ◽  
Generalization Capability ◽  
Average Quality ◽  
Novel Approach ◽  
Learning Data ◽  
Nonlinear Activation Function

Abstract The paper presents a novel approach to designing the CNN structure of improved generalization capability in the presence of a small population of learning data. In contrast to the classical methods for building CNN, we propose to introduce some randomness in the choice of layers with a different type of nonlinear activation function. Image processing in these layers is performed using either the ReLU or the softplus function. This choice is random. The randomness introduced into the network structure can be interpreted as a special form of regularization. Experiments performed in the recognition of images belonging to either melanoma or non-melanoma cases have shown a significant improvement in the average quality measures, such as the accuracy, sensitivity, precision, and the area under the ROC curve.

scholarly journals Evaluation of Cross-Validation Strategies in Sequence-Based Binding Prediction Using Deep Learning

2018 ◽  
Author(s):  
Angela Lopez-del Rio ◽  
Alfons Nonell-Canals ◽  
David Vidal ◽  
Alexandre Perera-Lluna
Keyword(s):  
Deep Learning ◽  
Deep Neural Networks ◽  
Cross Validation ◽  
Classification Performance ◽  
Generalization Capability ◽  
Binding Prediction ◽  
Source Database ◽  
Simple Logistic Regression Model ◽  
Molecular Databases ◽  
Simple Logistic

Binding prediction between targets and drug-like compounds through Deep Neural Networks have generated promising results in recent years, outperforming traditional machine learning-based methods. However, the generalization capability of these classification models is still an issue to be addressed. In this work, we explored how different cross-validation strategies applied to data from different molecular databases affect to the performance of binding prediction proteochemometrics models. These strategies are: (1) random splitting, (2) splitting based on K-means clustering (both of actives and inactives), (3) splitting based on source database and (4) splitting based both in the clustering and in the source database. These schemas are applied to a Deep Learning proteochemometrics model and to a simple logistic regression model to be used as baseline. Additionally, two different ways of describing molecules in the model are tested: (1) by their SMILES and (2) by three fingerprints. The classification performance of our Deep Learning-based proteochemometrics model is comparable to the state of the art. Our results show that the lack of generalization of these models is due to a bias in public molecular databases and that a restrictive cross-validation schema based on compounds clustering leads to worse but more robust and credible results. Our results also show better performance when representing molecules by their fingerprints.

scholarly journals Deep Learning Based on Residual Networks for Automatic Sorting of Bananas

2021 ◽  
Vol 2021 ◽  
pp. 1-11
Author(s):  
Abdulkader Helwan ◽  
Mohammad Khaleel Sallam Ma’aitah ◽  
Rahib H. Abiyev ◽  
Selin Uzelaltinbulat ◽  
Bengi Sonyel
Keyword(s):  
Deep Learning ◽  
Intelligent System ◽  
Deep Structure ◽  
Real Life ◽  
Generalization Capability ◽  
Residual Learning ◽  
Simulation Results ◽  
Human Operators ◽  
Comparative Results ◽  
Selection Of

This study presents the design of an intelligent system based on deep learning for grading fruits. For this purpose, the recent residual learning-based network “ResNet-50” is designed to sort out fruits, particularly bananas into healthy or defective classes. The design of the system is implemented by using transfer learning that uses the stored knowledge of the deep structure. Datasets of bananas have been collected for the implementation of the deep structure. The simulation results of the designed system have shown a great generalization capability when tested on test (unseen) banana images and obtained high accuracy of 99%. The simulation results of the designed residual learning-based system are compared with the results of other systems used for grading the bananas. Comparative results indicate the efficiency of the designed system. The developed system can be used in food processing industry, in real-life applications where the accuracy, cost, and speed of the intelligent system will enhance the production rate and allow meeting the demand of consumers. The system can replace or assist human operators who can exert their energy on the selection of fruits.

scholarly journals Evaluation of Cross-Validation Strategies in Sequence-Based Binding Prediction Using Deep Learning

2018 ◽  
Author(s):  
Angela Lopez-del Rio ◽  
Alfons Nonell-Canals ◽  
David Vidal ◽  
Alexandre Perera-Lluna
Keyword(s):  
Deep Learning ◽  
Deep Neural Networks ◽  
Cross Validation ◽  
Classification Performance ◽  
Generalization Capability ◽  
Binding Prediction ◽  
Source Database ◽  
Simple Logistic Regression Model ◽  
Molecular Databases ◽  
Simple Logistic

Binding prediction between targets and drug-like compounds through Deep Neural Networks have generated promising results in recent years, outperforming traditional machine learning-based methods. However, the generalization capability of these classification models is still an issue to be addressed. In this work, we explored how different cross-validation strategies applied to data from different molecular databases affect to the performance of binding prediction proteochemometrics models. These strategies are: (1) random splitting, (2) splitting based on K-means clustering (both of actives and inactives), (3) splitting based on source database and (4) splitting based both in the clustering and in the source database. These schemas are applied to a Deep Learning proteochemometrics model and to a simple logistic regression model to be used as baseline. Additionally, two different ways of describing molecules in the model are tested: (1) by their SMILES and (2) by three fingerprints. The classification performance of our Deep Learning-based proteochemometrics model is comparable to the state of the art. Our results show that the lack of generalization of these models is due to a bias in public molecular databases and that a restrictive cross-validation schema based on compounds clustering leads to worse but more robust and credible results. Our results also show better performance when representing molecules by their fingerprints.

scholarly journals MAGnitude-Image-to-Complex K-space (MAGIC-K) Net: A Data Augmentation Network for Image Reconstruction

Diagnostics ◽  
2021 ◽  
Vol 11 (10) ◽  
pp. 1935
Author(s):  
Fanwen Wang ◽  
Hui Zhang ◽  
Fei Dai ◽  
Weibo Chen ◽  
Chengyan Wang ◽  
...  
Keyword(s):  
Deep Learning ◽  
Image Reconstruction ◽  
Data Augmentation ◽  
Iterative Algorithms ◽  
Training Data ◽  
Superior Performance ◽  
Generalization Capability ◽  
Reconstruction Performance ◽  
Deep Learning Network ◽  
Space Data

Deep learning has demonstrated superior performance in image reconstruction compared to most conventional iterative algorithms. However, their effectiveness and generalization capability are highly dependent on the sample size and diversity of the training data. Deep learning-based reconstruction requires multi-coil raw k-space data, which are not collected by routine scans. On the other hand, large amounts of magnitude images are readily available in hospitals. Hence, we proposed the MAGnitude Images to Complex K-space (MAGIC-K) Net to generate multi-coil k-space data from existing magnitude images and a limited number of required raw k-space data to facilitate the reconstruction. Compared to some basic data augmentation methods applying global intensity and displacement transformations to the source images, the MAGIC-K Net can generate more realistic intensity variations and displacements from pairs of anatomical Digital Imaging and Communications in Medicine (DICOM) images. The reconstruction performance was validated in 30 healthy volunteers and 6 patients with different types of tumors. The experimental results demonstrated that the high-resolution Diffusion Weighted Image (DWI) reconstruction benefited from the proposed augmentation method. The MAGIC-K Net enabled the deep learning network to reconstruct images with superior performance in both healthy and tumor patients, qualitatively and quantitatively.

scholarly journals A Practical Alzheimer Disease Classifier via Brain Imaging-Based Deep Learning on 85,721 Samples

2021 ◽  
Author(s):  
Bin Lu ◽  
Hui-Xian Li ◽  
Zhi-Kai Chang ◽  
Le Li ◽  
Ning-Xuan Chen ◽  
...  
Keyword(s):  
Alzheimer’S Disease ◽  
Alzheimer's Disease ◽  
Deep Learning ◽  
Transfer Learning ◽  
Brain Mri ◽  
Machine Learning Algorithms ◽  
Training Data ◽  
Generalization Capability ◽  
Brain Images ◽  
Magnetic Resonance Imaging Mri

Abstract BackgroundBeyond detecting brain lesions or tumors, comparatively little success has been attained in identifying brain disorders such as Alzheimer’s disease (AD), based on magnetic resonance imaging (MRI). Many machine learning algorithms to detect AD have been trained using limited training data, meaning they often generalize poorly when applied to scans from previously unseen populations. Therefore, we built a practical brain MRI-based AD diagnostic classifier using deep learning/transfer learning on dataset of unprecedented size and diversity. MethodsA retrospective MRI dataset pooled from more than 217 sites/scanners constituted the largest brain MRI sample to date (85,721 scans from 50,876 participants) between January 2017 and August 2021. Next, a state-of-the-art deep convolutional neural network, Inception-ResNet-V2, was built as a sex classifier with high generalization capability. The sex classifier achieved 94.9% accuracy and served as a base model in transfer learning for the objective diagnosis of AD. FindingsAfter transfer learning, the model fine-tuned for AD classification achieved 91.3% accuracy in leave-sites-out cross-validation on the Alzheimer's Disease Neuroimaging Initiative (ADNI, 6,857 samples) dataset and 94.2%/93.6%/90.5% accuracy for direct tests on three unseen independent datasets (AIBL, 669 samples / MIRIAD, 644 samples / OASIS, 1,123 samples). When this AD classifier was tested on brain images from unseen mild cognitive impairment (MCI) patients, MCI patients who finally converted to AD were 3 times more likely to be predicted as AD than MCI patients who did not convert (65.2% vs 20.6%). Predicted scores from the AD classifier showed significant correlations with illness severity. InterpretationIn sum, the proposed AD classifier could offer a medical-grade marker that have potential to be integrated into AD diagnostic practice.

Deep Learning

2009 ◽  
Author(s):  
Stellan Ohlsson
Keyword(s):  
Deep Learning
Sign in / Sign up

Export Citation Format

Share Document