scholarly journals Evolution of Deep Convolutional Neural Networks Using Cartesian Genetic Programming

2020 ◽  
Vol 28 (1) ◽  
pp. 141-163 ◽  
Author(s):  
Masanori Suganuma ◽  
Masayuki Kobayashi ◽  
Shinichi Shirakawa ◽  
Tomoharu Nagao
Keyword(s):  
Genetic Programming ◽  
Expert Knowledge ◽  
Computational Cost ◽  
Computational Time ◽  
Network Architectures ◽  
Cartesian Genetic Programming ◽  
Deep Convolutional Neural Networks ◽  
High Performing ◽  
Network Training ◽  
Automatic Design Methods

The convolutional neural network (CNN), one of the deep learning models, has demonstrated outstanding performance in a variety of computer vision tasks. However, as the network architectures become deeper and more complex, designing CNN architectures requires more expert knowledge and trial and error. In this article, we attempt to automatically construct high-performing CNN architectures for a given task. Our method uses Cartesian genetic programming (CGP) to encode the CNN architectures, adopting highly functional modules such as a convolutional block and tensor concatenation, as the node functions in CGP. The CNN structure and connectivity, represented by the CGP, are optimized to maximize accuracy using the evolutionary algorithm. We also introduce simple techniques to accelerate the architecture search: rich initialization and early network training termination. We evaluated our method on the CIFAR-10 and CIFAR-100 datasets, achieving competitive performance with state-of-the-art models. Remarkably, our method can find competitive architectures with a reasonable computational cost compared to other automatic design methods that require considerably more computational time and machine resources.

scholarly journals An Efficient Methodology for Object Classification Using Light Weight Deep Convolutional Neural Networks

2019 ◽  
Vol 8 (2) ◽  
pp. 5965-5968
Keyword(s):  
Neural Networks ◽  
Object Detection ◽  
Computational Cost ◽  
Computational Time ◽  
Video Frame ◽  
Surveillance Systems ◽  
Light Weight ◽  
Deep Convolutional Neural Networks ◽  
Deep Convolution Neural Network ◽  
Improving Accuracy

In current era, deep convolution neural networks (DCNNs) have good break-through in processing images while reducing computational cost and increasing accuracy. Proposed approach focuses on object detection using classification with DCNN model. This model uses feature map for pre-processing the images and convolution layers helps to minimize the processing using deep learning perceptron’s. After that the proposed approach uses Light – Weight Deep Convolution Neural Network(LW_DCNN) Model which includes less number of convolution layers, Max Pooling layers with relevant parameters and Dense, flatten layers to train the data using Leaky ReLU function for improving accuracy. The proposed methodology LW_DCNN is highly efficient compared to traditional classification techniques and presenting simple and powerful model for object detection in Video Surveillance Systems. This model also tested on GPU systems and proved efficiency in less computational time. Obtained Results are clearly shows that model is more efficient in classifying the objects intern classifying the working condition of the overhead power polls insulators in real time video frame sequences.

scholarly journals Random Shifting for CNN: a Solution to Reduce Information Loss in Down-Sampling Layers

2017 ◽  
Author(s):  
Gangming Zhao ◽  
Jingdong Wang ◽  
Zhaoxiang Zhang
Keyword(s):  
Neural Networks ◽  
Computational Cost ◽  
Receptive Fields ◽  
Information Loss ◽  
Network Architectures ◽  
Training Process ◽  
Feature Maps ◽  
Improve Performance ◽  
Deep Convolutional Neural Networks ◽  
Random Strategy

Down-sampling is widely adopted in deep convolutional neural networks (DCNN) for reducing the number of network parameters while preserving the transformation invariance. However, it cannot utilize information effectively because it only adopts a fixed stride strategy, which may result in poor generalization ability and information loss. In this paper, we propose a novel random strategy to alleviate these problems by embedding random shifting in the down-sampling layers during the training process. Random shifting can be universally applied to diverse DCNN models to dynamically adjust receptive fields by shifting kernel centers on feature maps in different directions. Thus, it can generate more robust features in networks and further enhance the transformation invariance of down-sampling operators. In addition, random shifting cannot only be integrated in all down-sampling layers including strided convolutional layers and pooling layers, but also improve performance of DCNN with negligible additional computational cost. We evaluate our method in different tasks (e.g., image classification and segmentation) with various network architectures (i.e., AlexNet, FCN and DFN-MR). Experimental results demonstrate the effectiveness of our proposed method.

scholarly journals A Genetic Programming Approach to Designing Convolutional Neural Network Architectures

2018 ◽  
Author(s):  
Masanori Suganuma ◽  
Shinichi Shirakawa ◽  
Tomoharu Nagao
Keyword(s):  
Neural Network ◽  
Genetic Programming ◽  
Convolutional Neural Network ◽  
State Of The Art ◽  
Directed Acyclic Graphs ◽  
Validation Dataset ◽  
Programming Approach ◽  
Network Architectures ◽  
Cartesian Genetic Programming ◽  
Acyclic Graphs

We propose a method for designing convolutional neural network (CNN) architectures based on Cartesian genetic programming (CGP). In the proposed method, the architectures of CNNs are represented by directed acyclic graphs, in which each node represents highly-functional modules such as convolutional blocks and tensor operations, and each edge represents the connectivity of layers. The architecture is optimized to maximize the classification accuracy for a validation dataset by an evolutionary algorithm. We show that the proposed method can find competitive CNN architectures compared with state-of-the-art methods on the image classification task using CIFAR-10 and CIFAR-100 datasets.

Improving the Computational Cost for Copied Region Detection in Forensic Images

2016 ◽  
Vol 2 (1) ◽  
pp. 55
Author(s):  
Tu Huynh-Kha ◽  
Thuong Le-Tien ◽  
Synh Ha ◽  
Khoa Huynh-Van
Keyword(s):  
Wavelet Transform ◽  
Euclidean Distance ◽  
Research Work ◽  
Computational Cost ◽  
Correlation Coefficients ◽  
Zernike Moments ◽  
Computational Time ◽  
Discrete Wavelet ◽  
Feature Vectors ◽  
Region Detection

This research work develops a new method to detect the forgery in image by combining the Wavelet transform and modified Zernike Moments (MZMs) in which the features are defined from more pixels than in traditional Zernike Moments. The tested image is firstly converted to grayscale and applied one level Discrete Wavelet Transform (DWT) to reduce the size of image by a half in both sides. The approximation sub-band (LL), which is used for processing, is then divided into overlapping blocks and modified Zernike moments are calculated in each block as feature vectors. More pixels are considered, more sufficient features are extracted. Lexicographical sorting and correlation coefficients computation on feature vectors are next steps to find the similar blocks. The purpose of applying DWT to reduce the dimension of the image before using Zernike moments with updated coefficients is to improve the computational time and increase exactness in detection. Copied or duplicated parts will be detected as traces of copy-move forgery manipulation based on a threshold of correlation coefficients and confirmed exactly from the constraint of Euclidean distance. Comparisons results between proposed method and related ones prove the feasibility and efficiency of the proposed algorithm.

scholarly journals A divide-and-conquer algorithm for quantum state preparation

2021 ◽  
Vol 11 (1) ◽  
Author(s):  
Israel F. Araujo ◽  
Daniel K. Park ◽  
Francesco Petruccione ◽  
Adenilton J. da Silva
Keyword(s):  
Computational Cost ◽  
Quantum Circuit ◽  
Divide And Conquer ◽  
Computational Time ◽  
Quantum Computers ◽  
Dimensional Vector ◽  
Quantum Devices ◽  
Divide And Conquer Algorithm ◽  
Quantum State Preparation ◽  
Quantum Device

AbstractAdvantages in several fields of research and industry are expected with the rise of quantum computers. However, the computational cost to load classical data in quantum computers can impose restrictions on possible quantum speedups. Known algorithms to create arbitrary quantum states require quantum circuits with depth O(N) to load an N-dimensional vector. Here, we show that it is possible to load an N-dimensional vector with exponential time advantage using a quantum circuit with polylogarithmic depth and entangled information in ancillary qubits. Results show that we can efficiently load data in quantum devices using a divide-and-conquer strategy to exchange computational time for space. We demonstrate a proof of concept on a real quantum device and present two applications for quantum machine learning. We expect that this new loading strategy allows the quantum speedup of tasks that require to load a significant volume of information to quantum devices.

scholarly journals Computing Expectiles Using k-Nearest Neighbours Approach

Symmetry ◽  
2021 ◽  
Vol 13 (4) ◽  
pp. 645
Author(s):  
Muhammad Farooq ◽  
Sehrish Sarfraz ◽  
Christophe Chesneau ◽  
Mahmood Ul Hassan ◽  
Muhammad Ali Raza ◽  
...  
Keyword(s):  
Computational Cost ◽  
Real Life ◽  
Distance Measures ◽  
Computational Time ◽  
High Dimensional ◽  
Test Error ◽  
Nearest Neighbours ◽  
Comparable Performance ◽  
Asymmetric Least Squares ◽  
Low Computational Cost

Expectiles have gained considerable attention in recent years due to wide applications in many areas. In this study, the k-nearest neighbours approach, together with the asymmetric least squares loss function, called ex-kNN, is proposed for computing expectiles. Firstly, the effect of various distance measures on ex-kNN in terms of test error and computational time is evaluated. It is found that Canberra, Lorentzian, and Soergel distance measures lead to minimum test error, whereas Euclidean, Canberra, and Average of (L1,L∞) lead to a low computational cost. Secondly, the performance of ex-kNN is compared with existing packages er-boost and ex-svm for computing expectiles that are based on nine real life examples. Depending on the nature of data, the ex-kNN showed two to 10 times better performance than er-boost and comparable performance with ex-svm regarding test error. Computationally, the ex-kNN is found two to five times faster than ex-svm and much faster than er-boost, particularly, in the case of high dimensional data.

scholarly journals Concrete Crack Detection Based on Well-Known Feature Extractor Model and the YOLO_v2 Network

2021 ◽  
Vol 11 (2) ◽  
pp. 813
Author(s):  
Shuai Teng ◽  
Zongchao Liu ◽  
Gongfa Chen ◽  
Li Cheng
Keyword(s):  
Feature Extraction ◽  
Crack Detection ◽  
Computational Cost ◽  
Concrete Structures ◽  
Detection Algorithm ◽  
Computational Time ◽  
Image Size ◽  
Important Indicator ◽  
Feature Extractor ◽  
Model Training

This paper compares the crack detection performance (in terms of precision and computational cost) of the YOLO_v2 using 11 feature extractors, which provides a base for realizing fast and accurate crack detection on concrete structures. Cracks on concrete structures are an important indicator for assessing their durability and safety, and real-time crack detection is an essential task in structural maintenance. The object detection algorithm, especially the YOLO series network, has significant potential in crack detection, while the feature extractor is the most important component of the YOLO_v2. Hence, this paper employs 11 well-known CNN models as the feature extractor of the YOLO_v2 for crack detection. The results confirm that a different feature extractor model of the YOLO_v2 network leads to a different detection result, among which the AP value is 0.89, 0, and 0 for ‘resnet18’, ‘alexnet’, and ‘vgg16’, respectively meanwhile, the ‘googlenet’ (AP = 0.84) and ‘mobilenetv2’ (AP = 0.87) also demonstrate comparable AP values. In terms of computing speed, the ‘alexnet’ takes the least computational time, the ‘squeezenet’ and ‘resnet18’ are ranked second and third respectively; therefore, the ‘resnet18’ is the best feature extractor model in terms of precision and computational cost. Additionally, through the parametric study (influence on detection results of the training epoch, feature extraction layer, and testing image size), the associated parameters indeed have an impact on the detection results. It is demonstrated that: excellent crack detection results can be achieved by the YOLO_v2 detector, in which an appropriate feature extractor model, training epoch, feature extraction layer, and testing image size play an important role.

scholarly journals Evaluating effects of focal length and viewing angle in a comparison of recent face landmark and alignment methods

2021 ◽  
Vol 2021 (1) ◽  
Author(s):  
Xiang Li ◽  
Jianzheng Liu ◽  
Jessica Baron ◽  
Khoa Luu ◽  
Eric Patterson
Keyword(s):  
Focal Length ◽  
Ground Truth ◽  
Detection Methods ◽  
Viewing Angle ◽  
Deep Convolutional Neural Networks ◽  
Landmark Detection ◽  
High Performing ◽  
Detection Techniques ◽  
And Performance ◽  
Lens Focal Length

AbstractRecent attention to facial alignment and landmark detection methods, particularly with application of deep convolutional neural networks, have yielded notable improvements. Neither these neural-network nor more traditional methods, though, have been tested directly regarding performance differences due to camera-lens focal length nor camera viewing angle of subjects systematically across the viewing hemisphere. This work uses photo-realistic, synthesized facial images with varying parameters and corresponding ground-truth landmarks to enable comparison of alignment and landmark detection techniques relative to general performance, performance across focal length, and performance across viewing angle. Recently published high-performing methods along with traditional techniques are compared in regards to these aspects.

Sign in / Sign up

Export Citation Format

Share Document