scholarly journals Soft Filter Pruning for Accelerating Deep Convolutional Neural Networks

2018 ◽  
Author(s):  
Yang He ◽  
Guoliang Kang ◽  
Xuanyi Dong ◽  
Yanwei Fu ◽  
Yi Yang
Keyword(s):  
Neural Networks ◽  
Convolutional Neural Networks ◽  
State Of The Art ◽  
The State ◽  
Training Data ◽  
Deep Convolutional Neural Networks ◽  
Inference Procedure ◽  
Accuracy Improvement ◽  
Large Capacity ◽  
Pruning Methods

This paper proposed a Soft Filter Pruning (SFP) method to accelerate the inference procedure of deep Convolutional Neural Networks (CNNs). Specifically, the proposed SFP enables the pruned filters to be updated when training the model after pruning. SFP has two advantages over previous works: (1) Larger model capacity. Updating previously pruned filters provides our approach with larger optimization space than fixing the filters to zero. Therefore, the network trained by our method has a larger model capacity to learn from the training data. (2) Less dependence on the pretrained model. Large capacity enables SFP to train from scratch and prune the model simultaneously. In contrast, previous filter pruning methods should be conducted on the basis of the pre-trained model to guarantee their performance. Empirically, SFP from scratch outperforms the previous filter pruning methods. Moreover, our approach has been demonstrated effective for many advanced CNN architectures. Notably, on ILSCRC-2012, SFP reduces more than 42% FLOPs on ResNet-101 with even 0.2% top-5 accuracy improvement, which has advanced the state-of-the-art. Code is publicly available on GitHub: https://github.com/he-y/softfilter-pruning

scholarly journals Handwritten Bangla Character Recognition Using the State-of-the-Art Deep Convolutional Neural Networks

2018 ◽  
Vol 2018 ◽  
pp. 1-13 ◽  
Author(s):  
Md Zahangir Alom ◽  
Paheding Sidike ◽  
Mahmudul Hasan ◽  
Tarek M. Taha ◽  
Vijayan K. Asari
Keyword(s):  
Neural Networks ◽  
Object Recognition ◽  
Convolutional Neural Networks ◽  
Character Recognition ◽  
State Of The Art ◽  
The State ◽  
Superior Performance ◽  
Deep Convolutional Neural Networks ◽  
Practical Applications ◽  
High Degree

In spite of advances in object recognition technology, handwritten Bangla character recognition (HBCR) remains largely unsolved due to the presence of many ambiguous handwritten characters and excessively cursive Bangla handwritings. Even many advanced existing methods do not lead to satisfactory performance in practice that related to HBCR. In this paper, a set of the state-of-the-art deep convolutional neural networks (DCNNs) is discussed and their performance on the application of HBCR is systematically evaluated. The main advantage of DCNN approaches is that they can extract discriminative features from raw data and represent them with a high degree of invariance to object distortions. The experimental results show the superior performance of DCNN models compared with the other popular object recognition approaches, which implies DCNN can be a good candidate for building an automatic HBCR system for practical applications.

scholarly journals Dealing with Lack of Training Data for Convolutional Neural Networks: The Case of Digital Pathology

Electronics ◽  
2019 ◽  
Vol 8 (3) ◽  
pp. 256
Author(s):  
Francesco Ponzio ◽  
Gianvito Urgese ◽  
Elisa Ficarra ◽  
Santa Di Cataldo
Keyword(s):  
Neural Networks ◽  
Transfer Learning ◽  
Convolutional Neural Networks ◽  
Digital Pathology ◽  
Training Data ◽  
Automated Classification ◽  
Computationally Efficient ◽  
Deep Convolutional Neural Networks ◽  
Cad Systems ◽  
Tissue Characteristics

Thanks to their capability to learn generalizable descriptors directly from images, deep Convolutional Neural Networks (CNNs) seem the ideal solution to most pattern recognition problems. On the other hand, to learn the image representation, CNNs need huge sets of annotated samples that are unfeasible in many every-day scenarios. This is the case, for example, of Computer-Aided Diagnosis (CAD) systems for digital pathology, where additional challenges are posed by the high variability of the cancerous tissue characteristics. In our experiments, state-of-the-art CNNs trained from scratch on histological images were less accurate and less robust to variability than a traditional machine learning framework, highlighting all the issues of fully training deep networks with limited data from real patients. To solve this problem, we designed and compared three transfer learning frameworks, leveraging CNNs pre-trained on non-medical images. This approach obtained very high accuracy, requiring much less computational resource for the training. Our findings demonstrate that transfer learning is a solution to the automated classification of histological samples and solves the problem of designing accurate and computationally-efficient CAD systems with limited training data.

scholarly journals Automatic classification of ovarian cancer types from cytological images using deep convolutional neural networks

2018 ◽  
Vol 38 (3) ◽  
Author(s):  
Miao Wu ◽  
Chuanbo Yan ◽  
Huiqiang Liu ◽  
Qian Liu
Keyword(s):  
Neural Networks ◽  
Ovarian Cancer ◽  
Convolutional Neural Networks ◽  
Image Data ◽  
Training Data ◽  
Serous Carcinoma ◽  
Deep Convolutional Neural Networks ◽  
Ovarian Cancers ◽  
Cancer Types

Ovarian cancer is one of the most common gynecologic malignancies. Accurate classification of ovarian cancer types (serous carcinoma, mucous carcinoma, endometrioid carcinoma, transparent cell carcinoma) is an essential part in the different diagnosis. Computer-aided diagnosis (CADx) can provide useful advice for pathologists to determine the diagnosis correctly. In our study, we employed a Deep Convolutional Neural Networks (DCNN) based on AlexNet to automatically classify the different types of ovarian cancers from cytological images. The DCNN consists of five convolutional layers, three max pooling layers, and two full reconnect layers. Then we trained the model by two group input data separately, one was original image data and the other one was augmented image data including image enhancement and image rotation. The testing results are obtained by the method of 10-fold cross-validation, showing that the accuracy of classification models has been improved from 72.76 to 78.20% by using augmented images as training data. The developed scheme was useful for classifying ovarian cancers from cytological images.

scholarly journals ADVERSARIAL DOMAIN ADAPTATION FOR THE CLASSIFICATION OF AERIAL IMAGES AND HEIGHT DATA USING CONVOLUTIONAL NEURAL NETWORKS

2019 ◽  
Vol IV-2/W7 ◽  
pp. 197-204 ◽  
Author(s):  
D. Wittich ◽  
F. Rottensteiner
Keyword(s):  
Neural Networks ◽  
Convolutional Neural Networks ◽  
Domain Adaptation ◽  
Feature Representation ◽  
Training Data ◽  
Aerial Images ◽  
Remotely Sensed Data ◽  
Target Domain ◽  
Deep Convolutional Neural Networks

<p><strong>Abstract.</strong> Domain adaptation (DA) can drastically decrease the amount of training data needed to obtain good classification models by leveraging available data from a source domain for the classification of a new (target) domains. In this paper, we address deep DA, i.e. DA with deep convolutional neural networks (CNN), a problem that has not been addressed frequently in remote sensing. We present a new method for semi-supervised DA for the task of pixel-based classification by a CNN. After proposing an encoder-decoder-based fully convolutional neural network (FCN), we adapt a method for adversarial discriminative DA to be applicable to the pixel-based classification of remotely sensed data based on this network. It tries to learn a feature representation that is domain invariant; domain-invariance is measured by a classifier’s incapability of predicting from which domain a sample was generated. We evaluate our FCN on the ISPRS labelling challenge, showing that it is close to the best-performing models. DA is evaluated on the basis of three domains. We compare different network configurations and perform the representation transfer at different layers of the network. We show that when using a proper layer for adaptation, our method achieves a positive transfer and thus an improved classification accuracy in the target domain for all evaluated combinations of source and target domains.</p>

scholarly journals Evolving Deep Convolutional Neural Networks by Variable-Length Particle Swarm Optimization for Image Classification

2021 ◽  
Author(s):  
Bin Wang ◽  
Y Sun ◽  
Bing Xue ◽  
Mengjie Zhang
Keyword(s):  
Neural Networks ◽  
Image Classification ◽  
Convolutional Neural Networks ◽  
Particle Swarm ◽  
The State ◽  
Variable Length ◽  
Classification Error ◽  
Classification Problems ◽  
Deep Convolutional Neural Networks ◽  
State Of Art

© 2018 IEEE. Convolutional neural networks (CNNs) are one of the most effective deep learning methods to solve image classification problems, but the best architecture of a CNN to solve a specific problem can be extremely complicated and hard to design. This paper focuses on utilising Particle Swarm Optimisation (PSO) to automatically search for the optimal architecture of CNNs without any manual work involved. In order to achieve the goal, three improvements are made based on traditional PSO. First, a novel encoding strategy inspired by computer networks which empowers particle vectors to easily encode CNN layers is proposed; Second, in order to allow the proposed method to learn variable-length CNN architectures, a Disabled layer is designed to hide some dimensions of the particle vector to achieve variable-length particles; Third, since the learning process on large data is slow, partial datasets are randomly picked for the evaluation to dramatically speed it up. The proposed algorithm is examined and compared with 12 existing algorithms including the state-of-art methods on three widely used image classification benchmark datasets. The experimental results show that the proposed algorithm is a strong competitor to the state-of-art algorithms in terms of classification error. This is the first work using PSO for automatically evolving the architectures of CNNs. © 2018 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.

scholarly journals DEEPCON: protein contact prediction using dilated convolutional neural networks with dropout

2019 ◽  
Vol 36 (2) ◽  
pp. 470-477 ◽  
Author(s):  
Badri Adhikari
Keyword(s):  
Neural Networks ◽  
Long Range ◽  
Convolutional Neural Networks ◽  
High Throughput Sequencing ◽  
State Of The Art ◽  
Deep Convolutional Neural Networks ◽  
Contact Prediction ◽  
Learning Techniques ◽  
Medium Range ◽  
Art Methods

Abstract Motivation Exciting new opportunities have arisen to solve the protein contact prediction problem from the progress in neural networks and the availability of a large number of homologous sequences through high-throughput sequencing. In this work, we study how deep convolutional neural networks (ConvNets) may be best designed and developed to solve this long-standing problem. Results With publicly available datasets, we designed and trained various ConvNet architectures. We tested several recent deep learning techniques including wide residual networks, dropouts and dilated convolutions. We studied the improvements in the precision of medium-range and long-range contacts, and compared the performance of our best architectures with the ones used in existing state-of-the-art methods. The proposed ConvNet architectures predict contacts with significantly more precision than the architectures used in several state-of-the-art methods. When trained using the DeepCov dataset consisting of 3456 proteins and tested on PSICOV dataset of 150 proteins, our architectures achieve up to 15% higher precision when L/2 long-range contacts are evaluated. Similarly, when trained using the DNCON2 dataset consisting of 1426 proteins and tested on 84 protein domains in the CASP12 dataset, our single network achieves 4.8% higher precision than the ensembled DNCON2 method when top L long-range contacts are evaluated. Availability and implementation DEEPCON is available at https://github.com/badriadhikari/DEEPCON/.

Hybrid pooling with wavelets for convolutional neural networks

2022 ◽  
pp. 1-10
Author(s):  
Daniel Trevino-Sanchez ◽  
Vicente Alarcon-Aquino
Keyword(s):  
Neural Networks ◽  
Convolutional Neural Networks ◽  
State Of The Art ◽  
Computational Cost ◽  
Relevant Information ◽  
Accuracy Improvement ◽  
Proposed Model ◽  
Benchmark Datasets ◽  
Augmentation Techniques ◽  
High Computational Cost

The need to detect and classify objects correctly is a constant challenge, being able to recognize them at different scales and scenarios, sometimes cropped or badly lit is not an easy task. Convolutional neural networks (CNN) have become a widely applied technique since they are completely trainable and suitable to extract features. However, the growing number of convolutional neural networks applications constantly pushes their accuracy improvement. Initially, those improvements involved the use of large datasets, augmentation techniques, and complex algorithms. These methods may have a high computational cost. Nevertheless, feature extraction is known to be the heart of the problem. As a result, other approaches combine different technologies to extract better features to improve the accuracy without the need of more powerful hardware resources. In this paper, we propose a hybrid pooling method that incorporates multiresolution analysis within the CNN layers to reduce the feature map size without losing details. To prevent relevant information from losing during the downsampling process an existing pooling method is combined with wavelet transform technique, keeping those details "alive" and enriching other stages of the CNN. Achieving better quality characteristics improves CNN accuracy. To validate this study, ten pooling methods, including the proposed model, are tested using four benchmark datasets. The results are compared with four of the evaluated methods, which are also considered as the state-of-the-art.

Sign in / Sign up

Export Citation Format

Share Document