scholarly journals Convolutional Neural Networks Progress: Architectural and Optimization Methods Survey

2021 ◽  
Vol 8 (2) ◽  
pp. 44-68
Author(s):  
Mohsen Abdel-Atty
Keyword(s):  
Neural Networks ◽  
Convolutional Neural Networks ◽  
Optimization Methods

scholarly journals Thermal-based early breast cancer detection using inception V3, inception V4 and modified inception MV4

2021 ◽  
Author(s):  
Mohammed Abdulla Salim Al Husaini ◽  
Mohamed Hadi Habaebi ◽  
Teddy Surya Gunawan ◽  
Md Rafiqul Islam ◽  
Elfatih A. A. Elsheikh ◽  
...  
Keyword(s):  
Breast Cancer ◽  
Neural Networks ◽  
Deep Learning ◽  
Convolutional Neural Networks ◽  
Early Stage ◽  
Computational Cost ◽  
Optimization Methods ◽  
Optimization Method ◽  
Deep Convolutional Neural Networks ◽  
Breast Thermography

AbstractBreast cancer is one of the most significant causes of death for women around the world. Breast thermography supported by deep convolutional neural networks is expected to contribute significantly to early detection and facilitate treatment at an early stage. The goal of this study is to investigate the behavior of different recent deep learning methods for identifying breast disorders. To evaluate our proposal, we built classifiers based on deep convolutional neural networks modelling inception V3, inception V4, and a modified version of the latter called inception MV4. MV4 was introduced to maintain the computational cost across all layers by making the resultant number of features and the number of pixel positions equal. DMR database was used for these deep learning models in classifying thermal images of healthy and sick patients. A set of epochs 3–30 were used in conjunction with learning rates 1 × 10–3, 1 × 10–4 and 1 × 10–5, Minibatch 10 and different optimization methods. The training results showed that inception V4 and MV4 with color images, a learning rate of 1 × 10–4, and SGDM optimization method, reached very high accuracy, verified through several experimental repetitions. With grayscale images, inception V3 outperforms V4 and MV4 by a considerable accuracy margin, for any optimization methods. In fact, the inception V3 (grayscale) performance is almost comparable to inception V4 and MV4 (color) performance but only after 20–30 epochs. inception MV4 achieved 7% faster classification response time compared to V4. The use of MV4 model is found to contribute to saving energy consumed and fluidity in arithmetic operations for the graphic processor. The results also indicate that increasing the number of layers may not necessarily be useful in improving the performance.

scholarly journals Assessing Hyper Parameter Optimization and Speedup for Convolutional Neural Networks

2020 ◽  
Vol 10 (2) ◽  
pp. 1-17
Author(s):  
Sajid Nazir ◽  
Shushma Patel ◽  
Dilip Patel
Keyword(s):  
Neural Networks ◽  
Convolutional Neural Networks ◽  
Semantic Information ◽  
Optimization Methods ◽  
Training Models ◽  
Graphical Processing Units ◽  
Processing Power ◽  
Complex Image ◽  
Graphical Processing ◽  
Image Datasets

The increased processing power of graphical processing units (GPUs) and the availability of large image datasets has fostered a renewed interest in extracting semantic information from images. Promising results for complex image categorization problems have been achieved using deep learning, with neural networks comprised of many layers. Convolutional neural networks (CNN) are one such architecture which provides more opportunities for image classification. Advances in CNN enable the development of training models using large labelled image datasets, but the hyper parameters need to be specified, which is challenging and complex due to the large number of parameters. A substantial amount of computational power and processing time is required to determine the optimal hyper parameters to define a model yielding good results. This article provides a survey of the hyper parameter search and optimization methods for CNN architectures.

scholarly journals Calibrated Stochastic Gradient Descent for Convolutional Neural Networks

2019 ◽  
Vol 33 ◽  
pp. 9348-9355
Author(s):  
Li’an Zhuo ◽  
Baochang Zhang ◽  
Chen Chen ◽  
Qixiang Ye ◽  
Jianzhuang Liu ◽  
...  
Keyword(s):  
Neural Networks ◽  
Convolutional Neural Networks ◽  
Gradient Descent ◽  
Optimization Methods ◽  
Stochastic Gradient ◽  
Stochastic Gradient Descent ◽  
Natural Image ◽  
Digit Recognition ◽  
Gradient Optimization ◽  
New Research

In stochastic gradient descent (SGD) and its variants, the optimized gradient estimators may be as expensive to compute as the true gradient in many scenarios. This paper introduces a calibrated stochastic gradient descent (CSGD) algorithm for deep neural network optimization. A theorem is developed to prove that an unbiased estimator for the network variables can be obtained in a probabilistic way based on the Lipschitz hypothesis. Our work is significantly distinct from existing gradient optimization methods, by providing a theoretical framework for unbiased variable estimation in the deep learning paradigm to optimize the model parameter calculation. In particular, we develop a generic gradient calibration layer which can be easily used to build convolutional neural networks (CNNs). Experimental results demonstrate that CNNs with our CSGD optimization scheme can improve the stateof-the-art performance for natural image classification, digit recognition, ImageNet object classification, and object detection tasks. This work opens new research directions for developing more efficient SGD updates and analyzing the backpropagation algorithm.

scholarly journals Evolutionary-Fuzzy-Integral-Based Convolutional Neural Networks for Facial Image Classification

Electronics ◽  
2019 ◽  
Vol 8 (9) ◽  
pp. 997 ◽  
Author(s):  
Lin ◽  
Lin ◽  
Sun ◽  
Wang
Keyword(s):  
Neural Networks ◽  
Convolutional Neural Networks ◽  
Network Architecture ◽  
Optimization Methods ◽  
Optimization Method ◽  
Fuzzy Integral ◽  
Network Architectures ◽  
Age And Gender ◽  
Density Values ◽  
And Gender

Various optimization methods and network architectures are used by convolutional neural networks (CNNs). Each optimization method and network architecture style have their own advantages and representation abilities. To make the most of these advantages, evolutionary-fuzzy-integral-based convolutional neural networks (EFI-CNNs) are proposed in this paper. The proposed EFI-CNNs were verified by way of face classification of age and gender. The trained CNNs’ outputs were set as inputs of a fuzzy integral. The classification results were operated using either Sugeno or Choquet output rules. The conventional fuzzy density values of the fuzzy integral were decided by heuristic experiments. In this paper, particle swarm optimization (PSO) was used to adaptively find optimal fuzzy density values. To combine the advantages of each CNN type, the evaluation of each CNN type in EFI-CNNs is necessary. Three CNN structures, AlexNet, very deep convolutional neural network (VGG16), and GoogLeNet, and three databases, computational intelligence application laboratory (CIA), Morph, and cross-age celebrity dataset (CACD2000), were used in experiments to classify age and gender. The experimental results show that the proposed method achieved 5.95% and 3.1% higher accuracy, respectively, in classifying age and gender.

CNN-based Classification of Degraded Images

2020 ◽  
Vol 2020 (10) ◽  
pp. 28-1-28-7 ◽  
Author(s):  
Kazuki Endo ◽  
Masayuki Tanaka ◽  
Masatoshi Okutomi
Keyword(s):  
Neural Networks ◽  
Image Restoration ◽  
Image Classification ◽  
Convolutional Neural Networks ◽  
Deep Convolutional Neural Networks ◽  
Alternative Approach ◽  
Degraded Image ◽  
Degraded Images ◽  
Straightforward Approach

Classification of degraded images is very important in practice because images are usually degraded by compression, noise, blurring, etc. Nevertheless, most of the research in image classification only focuses on clean images without any degradation. Some papers have already proposed deep convolutional neural networks composed of an image restoration network and a classification network to classify degraded images. This paper proposes an alternative approach in which we use a degraded image and an additional degradation parameter for classification. The proposed classification network has two inputs which are the degraded image and the degradation parameter. The estimation network of degradation parameters is also incorporated if degradation parameters of degraded images are unknown. The experimental results showed that the proposed method outperforms a straightforward approach where the classification network is trained with degraded images only.

Sign in / Sign up

Export Citation Format

Share Document