A Hybrid GA-PSO Method for Evolving Architecture and Short Connections of Deep Convolutional Neural Networks

Deep Convolutional Neural Networks

Network Architecture ◽

Learning Task ◽

Fixed Number ◽

Learning Rate ◽

Current Layer ◽

Training Process ◽

© 2019, Springer Nature Switzerland AG. Image classification is a difficult machine learning task, where Convolutional Neural Networks (CNNs) have been applied for over 20 years in order to solve the problem. In recent years, instead of the traditional way of only connecting the current layer with its next layer, shortcut connections have been proposed to connect the current layer with its forward layers apart from its next layer, which has been proved to be able to facilitate the training process of deep CNNs. However, there are various ways to build the shortcut connections, it is hard to manually design the best shortcut connections when solving a particular problem, especially given the design of the network architecture is already very challenging. In this paper, a hybrid evolutionary computation (EC) method is proposed to automatically evolve both the architecture of deep CNNs and the shortcut connections. Three major contributions of this work are: Firstly, a new encoding strategy is proposed to encode a CNN, where the architecture and the shortcut connections are encoded separately; Secondly, a hybrid two-level EC method, which combines particle swarm optimisation and genetic algorithms, is developed to search for the optimal CNNs; Lastly, an adjustable learning rate is introduced for the fitness evaluations, which provides a better learning rate for the training process given a fixed number of epochs. The proposed algorithm is evaluated on three widely used benchmark datasets of image classification and compared with 12 peer Non-EC based competitors and one EC based competitor. The experimental results demonstrate that the proposed method outperforms all of the peer competitors in terms of classification accuracy.

CNN-based Classification of Degraded Images

Electronic Imaging ◽

10.2352/issn.2470-1173.2020.10.ipas-028 ◽

2020 ◽

Vol 2020 (10) ◽

pp. 28-1-28-7 ◽

Cited By ~ 1

Author(s):

Kazuki Endo ◽

Masayuki Tanaka ◽

Masatoshi Okutomi

Keyword(s):

Neural Networks ◽

Image Restoration ◽

Image Classification ◽

Alternative Approach ◽

Degraded Image ◽

Degraded Images ◽

Straightforward Approach

Classification of degraded images is very important in practice because images are usually degraded by compression, noise, blurring, etc. Nevertheless, most of the research in image classification only focuses on clean images without any degradation. Some papers have already proposed deep convolutional neural networks composed of an image restoration network and a classification network to classify degraded images. This paper proposes an alternative approach in which we use a degraded image and an additional degradation parameter for classification. The proposed classification network has two inputs which are the degraded image and the degradation parameter. The estimation network of degradation parameters is also incorporated if degradation parameters of degraded images are unknown. The experimental results showed that the proposed method outperforms a straightforward approach where the classification network is trained with degraded images only.

Semi-supervised training of deep convolutional neural networks with heterogeneous data and few local annotations: an experiment on prostate histopathology image classification

Medical Image Analysis ◽

10.1016/j.media.2021.102165 ◽

2021 ◽

pp. 102165

Author(s):

Niccolò Marini ◽

Sebastian Otálora ◽

Henning Müller ◽

Manfredo Atzori

Keyword(s):

Neural Networks ◽

Image Classification ◽

Heterogeneous Data ◽

Supervised Training

Document Image Classification with Intra-Domain Transfer Learning and Stacked Generalization of Deep Convolutional Neural Networks

2018 24th International Conference on Pattern Recognition (ICPR) ◽

10.1109/icpr.2018.8545630 ◽

2018 ◽

Cited By ~ 9

Author(s):

Arindam Das ◽

Saikat Roy ◽

Ujjwal Bhattacharya ◽

Swapan K. Parui

Keyword(s):

Neural Networks ◽

Image Classification ◽

Transfer Learning ◽

Document Image ◽

Stacked Generalization ◽

Domain Transfer ◽

Domain Transfer Learning

Lecture Notes in Computer Science - Experimental IR Meets Multilinguality, Multimodality, and Interaction ◽

Medical Image Classification with Weighted Latent Semantic Tensors and Deep Convolutional Neural Networks

10.1007/978-3-319-98932-7_8 ◽

2018 ◽

pp. 89-100 ◽

Cited By ~ 1

Author(s):

Spyridon Stathopoulos ◽

Theodore Kalamboukis

Keyword(s):

Neural Networks ◽

Image Classification ◽

Medical Image ◽

Medical Image Classification

A Baseline for Multi-Label Image Classification Using an Ensemble of Deep Convolutional Neural Networks

2019 IEEE International Conference on Image Processing (ICIP) ◽

10.1109/icip.2019.8803793 ◽

2019 ◽

Cited By ~ 1

Author(s):

Qian Wang ◽

Ning Jia ◽

Toby P. Breckon

Keyword(s):

Neural Networks ◽

Image Classification ◽

Deep Convolutional Neural Networks

Understanding unconventional preprocessors in deep convolutional neural networks for face identification

SN Applied Sciences ◽

10.1007/s42452-019-1538-5 ◽

2019 ◽

Vol 1 (11) ◽

Author(s):

Chollette C. Olisah ◽

Lyndon Smith

Keyword(s):

Neural Networks ◽

Network Architecture ◽

Data Augmentation ◽

Color Space ◽

Back Propagation ◽

Face Identification ◽

Softmax Classifier ◽

Deep Network

Abstract Deep convolutional neural networks have achieved huge successes in application domains like object and face recognition. The performance gain is attributed to different facets of the network architecture such as: depth of the convolutional layers, activation function, pooling, batch normalization, forward and back propagation and many more. However, very little emphasis is made on the preprocessor’s module of the network. Therefore, in this paper, the network’s preprocessing module is varied across different preprocessing approaches while keeping constant other facets of the deep network architecture, to investigate the contribution preprocessing makes to the network. Commonly used preprocessors are the data augmentation and normalization and are termed conventional preprocessors. Others are termed the unconventional preprocessors, they are: color space converters; grey-level resolution preprocessors; full-based and plane-based image quantization, Gaussian blur, illumination normalization and insensitive feature preprocessors. To achieve fixed network parameters, CNNs with transfer learning is employed. The aim is to transfer knowledge from the high-level feature vectors of the Inception-V3 network to offline preprocessed LFW target data; and features is trained using the SoftMax classifier for face identification. The experiments show that the discriminative capability of the deep networks can be improved by preprocessing RGB data with some of the unconventional preprocessors before feeding it to the CNNs. However, for best performance, the right setup of preprocessed data with augmentation and/or normalization is required. Summarily, preprocessing data before it is fed to the deep network is found to increase the homogeneity of neighborhood pixels even at reduced bit depth which serves for better storage efficiency.