scholarly journals Utilizing Information Bottleneck to Evaluate the Capability of Deep Neural Networks for Image Classification

Entropy ◽  
2019 ◽  
Vol 21 (5) ◽  
pp. 456 ◽  
Author(s):  
Hao Cheng ◽  
Dongze Lian ◽  
Shenghua Gao ◽  
Yanlin Geng
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Model Selection ◽  
Image Classification ◽  
Transfer Learning ◽  
Deep Neural Networks ◽  
Classification Problem ◽  
Distribution Model ◽  
Classification Problems ◽  
Information Bottleneck

Inspired by the pioneering work of the information bottleneck (IB) principle for Deep Neural Networks’ (DNNs) analysis, we thoroughly study the relationship among the model accuracy, I ( X ; T ) and I ( T ; Y ) , where I ( X ; T ) and I ( T ; Y ) are the mutual information of DNN’s output T with input X and label Y. Then, we design an information plane-based framework to evaluate the capability of DNNs (including CNNs) for image classification. Instead of each hidden layer’s output, our framework focuses on the model output T. We successfully apply our framework to many application scenarios arising in deep learning and image classification problems, such as image classification with unbalanced data distribution, model selection, and transfer learning. The experimental results verify the effectiveness of the information plane-based framework: Our framework may facilitate a quick model selection and determine the number of samples needed for each class in the unbalanced classification problem. Furthermore, the framework explains the efficiency of transfer learning in the deep learning area.

scholarly journals Transfer learning-based Plant Disease Detection

2021 ◽  
pp. 469-477
Keyword(s):  
Neural Networks ◽  
Image Classification ◽  
Transfer Learning ◽  
Plant Disease ◽  
Deep Neural Networks ◽  
Main Tool ◽  
Classification Problem ◽  
Disease Detection ◽  
Classification Problems ◽  
Crop Species

Deep Neural Networks in the field of Machine Learning (ML) are broadly used for deep learning. Among many of DNN structures, the Convolutional Neural Networks (CNN) are currently the main tool used for the image analysis and classification problems. Deep neural networks have been highly successful in image classification problems. In this paper, we have shown the use of deep neural networks for plant disease detection, through image classification. This study provides a transfer learning-based solution for detecting multiple diseases in several plant varieties using simple leaf images of healthy and diseased plants taken from PlantVillage dataset. We have addressed a multi-class classification problem in which the models were trained, validated and tested using 11,333 images from 10 different classes containing 2 crop species and 8 diseases. Six different CNN architectures VGG16, InceptionV3, Xception, Resnet50, MobileNet, and DenseNet121 are compared. We found that DenseNet121 achieves best accuracy of 95.48 on test data.

scholarly journals On Using Transfer Learning For Plant Disease Detection

2020 ◽  
Author(s):  
Abhinav Sagar ◽  
J Dheeba
Keyword(s):  
Neural Networks ◽  
Image Classification ◽  
Transfer Learning ◽  
Plant Disease ◽  
Deep Neural Networks ◽  
Classification Problem ◽  
Disease Detection ◽  
Classification Problems ◽  
Test Set ◽  
Multi Class Classification

ABSTRACTDeep neural networks have been highly successful in image classification problems. In this paper, we show how deep neural networks can be used for plant disease recognition in the context of image classification. We have used a publicly available Plant Village dataset which has 38 classes of diseases. Hence the problem that we have addressed is a multi class classification problem. We have compared five different architectures including VGG16, ResNet50, InceptionV3, InceptionResNet and DenseNet169 as the backbones for our work. We found that ResNet50 which uses skip connections using a residual layer archives the best result on the test set. For evaluating the results, we have used metrics like accuracy, precision, recall, F1 score and class wise confusion metric. Our model achieves the best of results using ResNet50 with accuracy of 0.982, precision of 0.94, recall of 0.94 and F1 score of 0.94.

scholarly journals Natural Images Allow Universal Adversarial Attacks on Medical Image Classification Using Deep Neural Networks with Transfer Learning

2021 ◽  
Author(s):  
Akinori Minagi ◽  
Hokuto Hirano ◽  
Kazuhiro Takemoto
Keyword(s):  
Neural Networks ◽  
Image Classification ◽  
Transfer Learning ◽  
Medical Image ◽  
Deep Neural Networks ◽  
Disease Diagnosis ◽  
Natural Images ◽  
Fine Tuning ◽  
Security And Privacy ◽  
Medical Image Classification

Abstract Transfer learning from natural images is well used in deep neural networks (DNNs) for medical image classification to achieve computer-aided clinical diagnosis. Although the adversarial vulnerability of DNNs hinders practical applications owing to the high stakes of diagnosis, adversarial attacks are expected to be limited because training data — which are often required for adversarial attacks — are generally unavailable in terms of security and privacy preservation. Nevertheless, we hypothesized that adversarial attacks are also possible using natural images because pre-trained models do not change significantly after fine-tuning. We focused on three representative DNN-based medical image classification tasks (i.e., skin cancer, referable diabetic retinopathy, and pneumonia classifications) and investigated whether medical DNN models with transfer learning are vulnerable to universal adversarial perturbations (UAPs), generated using natural images. UAPs from natural images are useful for both non-targeted and targeted attacks. The performance of UAPs from natural images was significantly higher than that of random controls, although slightly lower than that of UAPs from training images. Vulnerability to UAPs from natural images was observed between different natural image datasets and between different model architectures. The use of transfer learning causes a security hole, which decreases the reliability and safety of computer-based disease diagnosis. Model training from random initialization (without transfer learning) reduced the performance of UAPs from natural images; however, it did not completely avoid vulnerability to UAPs. The vulnerability of UAPs from natural images will become a remarkable security threat.

scholarly journals Convolutional neural network for image classification based on transfer learning technique

2021 ◽  
Author(s):  
Ghassan Mohammed Halawani
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Deep Learning ◽  
Convolutional Neural Network ◽  
Image Classification ◽  
Transfer Learning ◽  
Learning Networks ◽  
The Neural Network ◽  
Learning Technique ◽  
Common Architecture

The main purpose of this project is to modify a convolutional neural network for image classification, based on a deep-learning framework. A transfer learning technique is used by the MATLAB interface to Alex-Net to train and modify the parameters in the last two fully connected layers of Alex-Net with a new dataset to perform classifications of thousands of images. First, the general common architecture of most neural networks and their benefits are presented. The mathematical models and the role of each part in the neural network are explained in detail. Second, different neural networks are studied in terms of architecture, application, and the working method to highlight the strengths and weaknesses of each of neural network. The final part conducts a detailed study on one of the most powerful deep-learning networks in image classification – i.e. the convolutional neural network – and how it can be modified to suit different classification tasks by using transfer learning technique in MATLAB.

Network attributes describe a similarity between deep neural networks and large scale brain networks

2019 ◽  
Author(s):  
Kosuke Takagi
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Human Brain ◽  
Large Scale ◽  
Deep Neural Networks ◽  
Distribution Model ◽  
Common Mechanism ◽  
Learning Models ◽  
Connection Weight ◽  
Wide Range

Abstract Despite the recent success of deep learning models in solving various problems, their ability is still limited compared with human intelligence, which has the flexibility to adapt to a changing environment. To obtain a model which achieves adaptability to a wide range of problems and tasks is a challenging problem. To achieve this, an issue that must be addressed is identification of the similarities and differences between the human brain and deep neural networks. In this article, inspired by the human flexibility which might suggest the existence of a common mechanism allowing solution of different kinds of tasks, we consider a general learning process in neural networks, on which no specific conditions and constraints are imposed. Subsequently, we theoretically show that, according to the learning progress, the network structure converges to the state, which is characterized by a unique distribution model with respect to network quantities such as the connection weight and node strength. Noting that the empirical data indicate that this state emerges in the large scale network in the human brain, we show that the same state can be reproduced in a simple example of deep learning models. Although further research is needed, our findings provide an insight into the common inherent mechanism underlying the human brain and deep learning. Thus, our findings provide suggestions for designing efficient learning algorithms for solving a wide variety of tasks in the future.

scholarly journals Comparative Analysis of Deep Neural Networks Architectures for Visual Recognition in the Autonomous Transport Systems

2021 ◽  
Vol 2096 (1) ◽  
pp. 012101
Author(s):  
Y S Ivanov ◽  
S V Zhiganov ◽  
N N Liubushkina
Keyword(s):  
Neural Networks ◽  
Visual Recognition ◽  
Deep Neural Networks ◽  
Classification Problem ◽  
Transport Systems ◽  
Classification Problems ◽  
Comparison Results ◽  
Computer Vision Systems ◽  
Marine Vessels ◽  
Use Of Models

Abstract This paper analyses and presents an experimental investigation of the efficiency of modern models for object recognition in computer vision systems of robotic complexes. In this article, the applicability of transformers for experimental classification problems has been investigated. The comparison results are presented taking into account various limitations specific to robotics. Based on the results of the undertaken studies, recommendations on the use of models in the marine vessels classification problem are proposed

scholarly journals A Neuronal Morphology Classification Approach Based on Locally Cumulative Connected Deep Neural Networks

2019 ◽  
Vol 9 (18) ◽  
pp. 3876
Author(s):  
Xianghong Lin ◽  
Jianyang Zheng
Keyword(s):  
Neural Networks ◽  
Deep Neural Networks ◽  
Building Blocks ◽  
Classification Problem ◽  
Neuronal Morphology ◽  
Batch Size ◽  
Learning Approaches ◽  
Learning Networks ◽  
Classification Problems ◽  
Classification Approach

Neurons are the basic building and computational units of the nervous system, and have complex and diverse spatial geometric structures. By solving the neuronal classification problem, we can further understand the characteristics of neurons and the process of information transmission. This paper presents a neuronal morphology classification approach based on locally cumulative connected deep neural networks, where 43 geometric features were extracted from two different neuron datasets and applied to classify types of neurons. Then, the effects of different parameters of deep learning networks on the performance of neuron classification were analyzed including mini-batch size, number of intermediate layers, and number of building blocks. The accuracy of the approach was also compared with that of the other mainstream machine learning approaches. The experimental results showed that the proposed approach is effective for solving complex neuronal morphology classification problems.

scholarly journals Gender and Handedness Prediction from Offline Handwriting Using Convolutional Neural Networks

Complexity ◽  
2018 ◽  
Vol 2018 ◽  
pp. 1-14 ◽  
Author(s):  
Ángel Morera ◽  
Ángel Sánchez ◽  
José Francisco Vélez ◽  
Ana Belén Moreno
Keyword(s):  
Neural Networks ◽  
Experimental Study ◽  
Convolutional Neural Networks ◽  
Deep Neural Networks ◽  
Classification Problem ◽  
Gender Classification ◽  
Network Configuration ◽  
Classification Problems ◽  
Design Complexity ◽  
Average Accuracy

Demographic handwriting-based classification problems, such as gender and handedness categorizations, present interesting applications in disciplines like Forensic Biometrics. This work describes an experimental study on the suitability of deep neural networks to three automatic demographic problems: gender, handedness, and combined gender-and-handedness classifications, respectively. Our research was carried out on two public handwriting databases: the IAM dataset containing English texts and the KHATT one with Arabic texts. The considered problems present a high intrinsic difficulty when extracting specific relevant features for discriminating the involved subclasses. Our solution is based on convolutional neural networks since these models had proven better capabilities to extract good features when compared to hand-crafted ones. Our work also describes the first approach to the combined gender-and-handedness prediction, which has not been addressed before by other researchers. Moreover, the proposed solutions have been designed using a unique network configuration for the three considered demographic problems, which has the advantage of simplifying the design complexity and debugging of these deep architectures when handling related handwriting problems. Finally, the comparison of achieved results to those presented in related works revealed the best average accuracy in the gender classification problem for the considered datasets.

scholarly journals A Study of Adversarial Attacks and Detection on Deep Learning-Based Plant Disease Identification

2021 ◽  
Vol 11 (4) ◽  
pp. 1878
Author(s):  
Zhirui Luo ◽  
Qingqing Li ◽  
Jun Zheng
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Transfer Learning ◽  
Plant Disease ◽  
Deep Neural Networks ◽  
Economic Losses ◽  
Significant Delay ◽  
Disease Identification

Transfer learning using pre-trained deep neural networks (DNNs) has been widely used for plant disease identification recently. However, pre-trained DNNs are susceptible to adversarial attacks which generate adversarial samples causing DNN models to make wrong predictions. Successful adversarial attacks on deep learning (DL)-based plant disease identification systems could result in a significant delay of treatments and huge economic losses. This paper is the first attempt to study adversarial attacks and detection on DL-based plant disease identification. Our results show that adversarial attacks with a small number of perturbations can dramatically degrade the performance of DNN models for plant disease identification. We also find that adversarial attacks can be effectively defended by using adversarial sample detection with an appropriate choice of features. Our work will serve as a basis for developing more robust DNN models for plant disease identification and guiding the defense against adversarial attacks.

Sign in / Sign up

Export Citation Format

Share Document