Evolutionary Multi-Objective One-Shot Filter Pruning for Designing Lightweight Convolutional Neural Network

Deep neural networks have achieved significant development and wide applications for their amazing performance. However, their complex structure, high computation and storage resource limit their applications in mobile or embedding devices such as sensor platforms. Neural network pruning is an efficient way to design a lightweight model from a well-trained complex deep neural network. In this paper, we propose an evolutionary multi-objective one-shot filter pruning method for designing a lightweight convolutional neural network. Firstly, unlike some famous iterative pruning methods, a one-shot pruning framework only needs to perform filter pruning and model fine-tuning once. Moreover, we built a constraint multi-objective filter pruning problem in which two objectives represent the filter pruning ratio and the accuracy of the pruned convolutional neural network, respectively. A non-dominated sorting-based evolutionary multi-objective algorithm was used to solve the filter pruning problem, and it provides a set of Pareto solutions which consists of a series of different trade-off pruned models. Finally, some models are uniformly selected from the set of Pareto solutions to be fine-tuned as the output of our method. The effectiveness of our method was demonstrated in experimental studies on four designed models, LeNet and AlexNet. Our method can prune over 85%, 82%, 75%, 65%, 91% and 68% filters with little accuracy loss on four designed models, LeNet and AlexNet, respectively.

Download Full-text

Knowledge Transferred Fine-Tuning for Anti-Aliased Convolutional Neural Network in Data-Limited Situation

10.1109/icip42928.2021.9506696 ◽

2021 ◽

Author(s):

Satoshi Suzuki ◽

Shoichiro Takeda ◽

Ryuichi Tanida ◽

Hideaki Kimata ◽

Hayaru Shouno

Keyword(s):

Neural Network ◽

Convolutional Neural Network ◽

Fine Tuning

Download Full-text

Research on Real-Time Multiple Single Garbage Classification Based on Convolutional Neural Network

Mathematical Problems in Engineering ◽

10.1155/2020/5795976 ◽

2020 ◽

Vol 2020 ◽

pp. 1-6

Author(s):

Jian-ye Yuan ◽

Xin-yuan Nan ◽

Cheng-rong Li ◽

Le-le Sun

Keyword(s):

Neural Network ◽

Convolutional Neural Network ◽

Real Time ◽

Environmental Science ◽

Fine Tuning ◽

Classification Model ◽

Classification Models ◽

Data Set ◽

Classification Tasks ◽

Accuracy Rates

Considering that the garbage classification is urgent, a 23-layer convolutional neural network (CNN) model is designed in this paper, with the emphasis on the real-time garbage classification, to solve the low accuracy of garbage classification and recycling and difficulty in manual recycling. Firstly, the depthwise separable convolution was used to reduce the Params of the model. Then, the attention mechanism was used to improve the accuracy of the garbage classification model. Finally, the model fine-tuning method was used to further improve the performance of the garbage classification model. Besides, we compared the model with classic image classification models including AlexNet, VGG16, and ResNet18 and lightweight classification models including MobileNetV2 and SuffleNetV2 and found that the model GAF_dense has a higher accuracy rate, fewer Params, and FLOPs. To further check the performance of the model, we tested the CIFAR-10 data set and found the accuracy rates of the model (GAF_dense) are 0.018 and 0.03 higher than ResNet18 and SufflenetV2, respectively. In the ImageNet data set, the accuracy rates of the model (GAF_dense) are 0.225 and 0.146 higher than Resnet18 and SufflenetV2, respectively. Therefore, the garbage classification model proposed in this paper is suitable for garbage classification and other classification tasks to protect the ecological environment, which can be applied to classification tasks such as environmental science, children’s education, and environmental protection.

Download Full-text

"LeukNet" - A Model of Convolutional Neural Network for the Diagnosis of Leukemia

10.5753/sibgrapi.est.2020.12993 ◽

2020 ◽

Author(s):

Luis H. S. Vogado ◽

Rodrigo M. S. Veras ◽

Kelson R. T. Aires

Keyword(s):

Neural Network ◽

Convolutional Neural Network ◽

Cross Validation ◽

Data Augmentation ◽

Fine Tuning ◽

Current State ◽

Leukemia Diagnosis ◽

New Guidelines ◽

Texture Contrast ◽

Tuning Methods

Leukemia is a disorder that affects the bone marrow, causing uncontrolled production of leukocytes, impairing the transport of oxygen and causing blood coagulation problems. In this article, we propose a new computational tool, named LeukNet, a Convolutional Neural Network (CNN) architecture based on the VGG-16 convolutional blocks, to facilitate the leukemia diagnosis from blood smear images. We evaluated different architectures and fine-tuning methods using 18 datasets containing 3536 images with distinct characteristics of color, texture, contrast, and resolution. Additionally, data augmentation operations were applied to increase the training set by up to 20 times. The k-fold cross-validation (k = 5) results achieved 98.28% of accuracy. A cross-dataset validation technique, named LeaveOne-Dataset-Out Cross-Validation (LODOCV), is also proposed to evaluate the developed model’s generalization capability. The accuracy of using LODOCV on the ALL-IDB 1, ALL-IDB 2, and UFG datasets was 97.04%, 82.46%, and 70.24%, respectively, overcoming the current state-of-the-art results and offering new guidelines for image-based computer-aided diagnosis (CAD) systems in this area.

Download Full-text

Performance Evaluation of Fine-tuned Faster R-CNN on specific MS COCO Objects

International Journal of Electrical and Computer Engineering (IJECE) ◽

10.11591/ijece.v9i4.pp2548-2555 ◽

2019 ◽

Vol 9 (4) ◽

pp. 2548

Author(s):

Garima Devnani ◽

Ayush Jaiswal ◽

Roshni John ◽

Rajat Chaurasia ◽

Neha Tirpude

Keyword(s):

Neural Network ◽

Performance Evaluation ◽

Convolutional Neural Network ◽

Training Model ◽

Fine Tuning ◽

Training Process

<span lang="EN-US">Fine-tuning of a model is a method that is most often required to cater to the users’ explicit requirements. But the question remains whether the model is accurate enough to be used for a certain application. This paper strives to present the metrics used for performance evaluation of a Convolutional Neural Network (CNN) model. The evaluation is based on the training process which provides us with intermediate models after every 1000 iterations. While 1000 iterations are not substantial enough over the range of 490k iterations, the groups are sized with 100k iterations each. Now, the intention was to compare the recorded metrics to evaluate the model in terms of accuracy. The training model used the set of specific categories chosen from the Microsoft Common Objects in Context (MS COCO) dataset while allowing the users to use their externally available images to test the model’s accuracy. Our trained model ensured that all the objects are detected that are present in the image to depict the effect of precision.</span>

Download Full-text

Tomato pest classification using deep convolutional neural network with transfer learning, fine tuning and scratch learning

Intelligent Decision Technologies ◽

10.3233/idt-200192 ◽

2021 ◽

pp. 1-10

Author(s):

Gayatri Pattnaik ◽

Vimal K. Shrivastava ◽

K. Parvathi

Keyword(s):

Neural Network ◽

Convolutional Neural Network ◽

Transfer Learning ◽

Data Augmentation ◽

State Of The Art ◽

Deep Convolutional Neural Network ◽

Fine Tuning ◽

Tomato Plants ◽

Random Weights

Pests are major threat to economic growth of a country. Application of pesticide is the easiest way to control the pest infection. However, excessive utilization of pesticide is hazardous to environment. The recent advances in deep learning have paved the way for early detection and improved classification of pest in tomato plants which will benefit the farmers. This paper presents a comprehensive analysis of 11 state-of-the-art deep convolutional neural network (CNN) models with three configurations: transfers learning, fine-tuning and scratch learning. The training in transfer learning and fine tuning initiates from pre-trained weights whereas random weights are used in case of scratch learning. In addition, the concept of data augmentation has been explored to improve the performance. Our dataset consists of 859 tomato pest images from 10 categories. The results demonstrate that the highest classification accuracy of 94.87% has been achieved in the transfer learning approach by DenseNet201 model with data augmentation.

Download Full-text

A Technical Review of Convolutional Neural Network-Based Mammographic Breast Cancer Diagnosis

Computational and Mathematical Methods in Medicine ◽

10.1155/2019/6509357 ◽

2019 ◽

Vol 2019 ◽

pp. 1-16 ◽

Cited By ~ 15

Author(s):

Lian Zou ◽

Shaode Yu ◽

Tiebao Meng ◽

Zhicheng Zhang ◽

Xiaokun Liang ◽

...

Keyword(s):

Breast Cancer ◽

Neural Network ◽

Convolutional Neural Network ◽

Cancer Diagnosis ◽

Early Stage ◽

Breast Cancer Diagnosis ◽

Fine Tuning ◽

Learning Tools ◽

Pros And Cons ◽

Future Work

This study reviews the technique of convolutional neural network (CNN) applied in a specific field of mammographic breast cancer diagnosis (MBCD). It aims to provide several clues on how to use CNN for related tasks. MBCD is a long-standing problem, and massive computer-aided diagnosis models have been proposed. The models of CNN-based MBCD can be broadly categorized into three groups. One is to design shallow or to modify existing models to decrease the time cost as well as the number of instances for training; another is to make the best use of a pretrained CNN by transfer learning and fine-tuning; the third is to take advantage of CNN models for feature extraction, and the differentiation of malignant lesions from benign ones is fulfilled by using machine learning classifiers. This study enrolls peer-reviewed journal publications and presents technical details and pros and cons of each model. Furthermore, the findings, challenges and limitations are summarized and some clues on the future work are also given. Conclusively, CNN-based MBCD is at its early stage, and there is still a long way ahead in achieving the ultimate goal of using deep learning tools to facilitate clinical practice. This review benefits scientific researchers, industrial engineers, and those who are devoted to intelligent cancer diagnosis.

Download Full-text