Evolutionary and Heuristic Approaches for the Selection of Neural Network Architectures and Parameters1

1998 ◽  
Vol 2 (6) ◽  
pp. 214-220 ◽  
Author(s):  
Tim Whitfort ◽  
◽  
Chris Matthews ◽  
Belinda Choi ◽  
John McCullagh
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Operating Parameters ◽  
Network Architectures ◽  
Further Training ◽  
High Performing ◽  
Design Heuristics ◽  
Stage Process ◽  
Neural Network Architectures ◽  
Selection Of

Genetic algorithms (GAs) and neural networks have been combined in various ways in an effort to develop powerful tools for problem solving. This work presents a two-stage process for the specification of high-performing backpropagation neural networks for four commonly used real-world databases. In the first stage, GAs are used to evolve a set of potential neural network architectures and operating parameters. The best networks are then used for further training and testing, to determine an optimum starting seed and number of training passes. We also compare the evolved network architectures and operating parameters from stage I with some of the backpropagation neural network (BPNN) design heuristics previously reported in the literature. In order to test and compare networks specified in this way, an exhaustive set of experiments using the machine learning induction algorithm, C4.5 were also carried out on the four databases. These results, together with some previously published benchmark data and BPNN results, were then compared with those obtained from the networks developed using our method. Our results compare more than favorably with the C4.5, benchmark, and previously published BPNN results across the four databases.

Prediction of slip in cable-drum systems using structured neural networks

2013 ◽  
Vol 228 (3) ◽  
pp. 441-456 ◽  
Author(s):  
Ergin Kilic ◽  
Melik Dolen
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Network Size ◽  
Network Architectures ◽  
Network Development ◽  
Low Velocity ◽  
Error Band ◽  
Artificial Neural ◽  
Recurrent Type ◽  
Selection Of

This study focuses on the slip prediction in a cable-drum system using artificial neural networks for the prospect of developing linear motion sensing scheme for such mechanisms. Both feed-forward and recurrent-type artificial neural network architectures are considered to capture the slip dynamics of cable-drum mechanisms. In the article, the network development is presented in a progressive (step-by-step) fashion for the purpose of not only making the design process transparent to the readers but also highlighting the corresponding challenges associated with the design phase (i.e. selection of architecture, network size, training process parameters, etc.). Prediction performances of the devised networks are evaluated rigorously via an experimental study. Finally, a structured neural network, which embodies the network with the best prediction performance, is further developed to overcome the drift observed at low velocity. The study illustrates that the resulting structured neural network could predict the slip in the mechanism within an error band of 100 µm when an absolute reference is utilized.

scholarly journals Neural networks for classification of strokes in electrical impedance tomography on a 3D head model

2022 ◽  
Vol 4 (4) ◽  
pp. 1-22
Author(s):  
Valentina Candiani ◽  
◽  
Matteo Santacesaria ◽  
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Electrical Impedance Tomography ◽  
Electrical Impedance ◽  
Network Architectures ◽  
Impedance Tomography ◽  
Average Accuracy ◽  
Neural Network Architectures ◽  
Fully Connected

<abstract><p>We consider the problem of the detection of brain hemorrhages from three-dimensional (3D) electrical impedance tomography (EIT) measurements. This is a condition requiring urgent treatment for which EIT might provide a portable and quick diagnosis. We employ two neural network architectures - a fully connected and a convolutional one - for the classification of hemorrhagic and ischemic strokes. The networks are trained on a dataset with $ 40\, 000 $ samples of synthetic electrode measurements generated with the complete electrode model on realistic heads with a 3-layer structure. We consider changes in head anatomy and layers, electrode position, measurement noise and conductivity values. We then test the networks on several datasets of unseen EIT data, with more complex stroke modeling (different shapes and volumes), higher levels of noise and different amounts of electrode misplacement. On most test datasets we achieve $ \geq 90\% $ average accuracy with fully connected neural networks, while the convolutional ones display an average accuracy $ \geq 80\% $. Despite the use of simple neural network architectures, the results obtained are very promising and motivate the applications of EIT-based classification methods on real phantoms and ultimately on human patients.</p></abstract>

Swarm-Based Nature-Inspired Metaheuristics for Neural Network Optimization

2018 ◽  
pp. 23-53
Author(s):  
Swathi Jamjala Narayanan ◽  
Boominathan Perumal ◽  
Jayant G. Rohra
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Network Optimization ◽  
Network Architectures ◽  
Neural Network Optimization ◽  
Local Optima ◽  
Gradient Based ◽  
Neural Network Architectures ◽  
Nature Inspired Algorithms ◽  
Nature Inspired Metaheuristics

Nature-inspired algorithms have been productively applied to train neural network architectures. There exist other mechanisms like gradient descent, second order methods, Levenberg-Marquardt methods etc. to optimize the parameters of neural networks. Compared to gradient-based methods, nature-inspired algorithms are found to be less sensitive towards the initial weights set and also it is less likely to become trapped in local optima. Despite these benefits, some nature-inspired algorithms also suffer from stagnation when applied to neural networks. The other challenge when applying nature inspired techniques for neural networks would be in handling large dimensional and correlated weight space. Hence, there arises a need for scalable nature inspired algorithms for high dimensional neural network optimization. In this chapter, the characteristics of nature inspired techniques towards optimizing neural network architectures along with its applicability, advantages and limitations/challenges are studied.

IMPROVING GENERALIZATION OF NEURAL NETWORKS THROUGH PRUNING

1991 ◽  
Vol 01 (04) ◽  
pp. 317-326 ◽  
Author(s):  
Hans Henrik Thodberg
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Internal Representation ◽  
Network Architectures ◽  
Ockham’S Razor ◽  
Ockham's Razor ◽  
Neural Network Architectures

A technique for constructing neural network architectures with better ability to generalize is presented under the name Ockham's Razor: several networks are trained and then pruned by removing connections one by one and retraining. The networks which achieve fewest connections generalize best. The method is tested on a classification of bit strings (the contiguity problem): the optimal architecture emerges, resulting in perfect generalization. The internal representation of the network changes substantially during the retraining, and this distinguishes the method from previous pruning studies.

scholarly journals Classification of Fermi-LAT sources with deep learning using energy and time spectra

2021 ◽  
Vol 507 (3) ◽  
pp. 4061-4073
Author(s):  
Thorben Finke ◽  
Michael Krämer ◽  
Silvia Manconi
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Active Galactic Nuclei ◽  
Photon Energy ◽  
Deep Neural Network ◽  
Gamma Ray ◽  
Galactic Nuclei ◽  
Network Architectures ◽  
Neural Network Architectures

ABSTRACT Despite the growing number of gamma-ray sources detected by the Fermi-Large Area Telescope (LAT), about one-third of the sources in each survey remains of uncertain type. We present a new deep neural network approach for the classification of unidentified or unassociated gamma-ray sources in the last release of the Fermi-LAT catalogue (4FGL-DR2) obtained with 10 yr of data. In contrast to previous work, our method directly uses the measurements of the photon energy spectrum and time series as input for the classification, instead of specific, human-crafted features. Dense neural networks, and for the first time in the context of gamma-ray source classification recurrent neural networks, are studied in depth. We focus on the separation between extragalactic sources, i.e. active galactic nuclei, and Galactic pulsars, and on the further classification of pulsars into young and millisecond pulsars. Our neural network architectures provide powerful classifiers, with a performance that is comparable to previous analyses based on human-crafted features. Our benchmark neural network predicts that of the sources of uncertain type in the 4FGL-DR2 catalogue, 1050 are active galactic nuclei and 78 are Galactic pulsars, with both classes following the expected sky distribution and the clustering in the variability–curvature plane. We investigate the problem of sample selection bias by testing our architectures against a cross-match test data set using an older catalogue, and propose a feature selection algorithm using autoencoders. Our list of high-confidence candidate sources labelled by the neural networks provides a set of targets for further multiwavelength observations addressed to identify their nature. The deep neural network architectures we develop can be easily extended to include specific features, as well as multiwavelength data on the source photon energy and time spectra coming from different instruments.

scholarly journals Interaffection of Multiple Datasets with Neural Networks in Speech Emotion Recognition

2020 ◽  
Author(s):  
Ronnypetson Da Silva ◽  
Valter M. Filho ◽  
Mario Souza
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Emotion Recognition ◽  
Deep Neural Networks ◽  
Speech Emotion Recognition ◽  
Network Architectures ◽  
Shared Representations ◽  
Multiple Datasets ◽  
Neural Network Architectures

Many works that apply Deep Neural Networks (DNNs) to Speech Emotion Recognition (SER) use single datasets or train and evaluate the models separately when using multiple datasets. Those datasets are constructed with specific guidelines and the subjective nature of the labels for SER makes it difficult to obtain robust and general models. We investigate how DNNs learn shared representations for different datasets in both multi-task and unified setups. We also analyse how each dataset benefits from others in different combinations of datasets and popular neural network architectures. We show that the longstanding belief of more data resulting in more general models doesn’t always hold for SER, as different dataset and meta-parameter combinations hold the best result for each of the analysed datasets.

scholarly journals Cinemática Inversa com Redes Neurais Aplicadas em Robôs Manipuladores

2018 ◽  
Vol 1 (43) ◽  
pp. 49
Author(s):  
Gabriel Daltro Duarte ◽  
Claudio Pereira Mego Quinteros ◽  
Lincoln Machado Araújo
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Nonlinear Equations ◽  
Inverse Kinematics ◽  
Robot Manipulator ◽  
Kinematic Model ◽  
Online Training ◽  
Network Architectures ◽  
The Neural Network ◽  
Neural Network Architectures

Today’s world is going through what is known as the Fourth Industrial Revolution. Robots have been gaining more and more space in the industry and going beyond expectations. The use of robots in industry is related to the increasing production and the quality of the electronic products. For an accurate movement of a robot manipulator it is necessary to obtain its inverse kinematic model, however, obtaining this model requires the challenging solution of a set of nonlinear equations. For this system of nonlinear equations there is no generic solution method. In view of this matter, this research aims to solve the problem of the inverse kinematics of a robot manipulator using artificial neural networks without the need to model the robot’s direct kinematics. Two neural network training methodologies, called offline training and online training were used. The basic difference between these two is that in the offline methodology all training points are obtained before any training of the neural network occurs, whereas in online training the use of a training method is recurrent. As the robot moves, new training points are obtained and training processes with the new acquired points are executed, allowing a learning process of continuous inverse kinematics. To validate the proposed methodology a prototype of a manipulated planar robot with one degree of freedom was developed and several architectures of neural networks were tested to find the optimal architecture. The offline training methodology obtained very satisfactory results for most of the neural network architectures tested. The online training only achieved satisfactory results in neural network architectures with quantities of neurons much larger than the quantities used in the architectures used in offline training and still obtained inferior results. The neural networks trained in offline mode, when compared to the training networks in the online mode, presented a greater capacity of generalization and a smaller value of output error. The online training has only achieved satisfactory results in neural network architectures with quantities of neurons much larger than the quantities used in the trained architectures in offline mode.

scholarly journals HELLO: improved neural network architectures and methodologies for small variant calling

2021 ◽  
Vol 22 (1) ◽  
Author(s):  
Anand Ramachandran ◽  
Steven S. Lumetta ◽  
Eric W. Klee ◽  
Deming Chen
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Image Recognition ◽  
Deep Neural Network ◽  
Deep Neural Networks ◽  
Method Development ◽  
Variant Calling ◽  
Network Architectures ◽  
Sequencing Data ◽  
Neural Network Architectures

Abstract Background Modern Next Generation- and Third Generation- Sequencing methods such as Illumina and PacBio Circular Consensus Sequencing platforms provide accurate sequencing data. Parallel developments in Deep Learning have enabled the application of Deep Neural Networks to variant calling, surpassing the accuracy of classical approaches in many settings. DeepVariant, arguably the most popular among such methods, transforms the problem of variant calling into one of image recognition where a Deep Neural Network analyzes sequencing data that is formatted as images, achieving high accuracy. In this paper, we explore an alternative approach to designing Deep Neural Networks for variant calling, where we use meticulously designed Deep Neural Network architectures and customized variant inference functions that account for the underlying nature of sequencing data instead of converting the problem to one of image recognition. Results Results from 27 whole-genome variant calling experiments spanning Illumina, PacBio and hybrid Illumina-PacBio settings suggest that our method allows vastly smaller Deep Neural Networks to outperform the Inception-v3 architecture used in DeepVariant for indel and substitution-type variant calls. For example, our method reduces the number of indel call errors by up to 18%, 55% and 65% for Illumina, PacBio and hybrid Illumina-PacBio variant calling respectively, compared to a similarly trained DeepVariant pipeline. In these cases, our models are between 7 and 14 times smaller. Conclusions We believe that the improved accuracy and problem-specific customization of our models will enable more accurate pipelines and further method development in the field. HELLO is available at https://github.com/anands-repo/hello

Evaluation of deep learning approaches based on convolutional neural networks for corrosion detection

2017 ◽  
Vol 17 (5) ◽  
pp. 1110-1128 ◽  
Author(s):  
Deegan J Atha ◽  
Mohammad R Jahanshahi
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Convolutional Neural Network ◽  
Convolutional Neural Networks ◽  
State Of The Art ◽  
Inspection Time ◽  
Computational Time ◽  
Network Architectures ◽  
Corrosion Detection ◽  
Neural Network Architectures

Corrosion is a major defect in structural systems that has a significant economic impact and can pose safety risks if left untended. Currently, an inspector visually assesses the condition of a structure to identify corrosion. This approach is time-consuming, tedious, and subjective. Robotic systems, such as unmanned aerial vehicles, paired with computer vision algorithms have the potential to perform autonomous damage detection that can significantly decrease inspection time and lead to more frequent and objective inspections. This study evaluates the use of convolutional neural networks for corrosion detection. A convolutional neural network learns the appropriate classification features that in traditional algorithms were hand-engineered. Eliminating the need for dependence on prior knowledge and human effort in designing features is a major advantage of convolutional neural networks. This article presents different convolutional neural network–based approaches for corrosion assessment on metallic surfaces. The effect of different color spaces, sliding window sizes, and convolutional neural network architectures are discussed. To this end, the performance of two pretrained state-of-the-art convolutional neural network architectures as well as two proposed convolutional neural network architectures are evaluated, and it is shown that convolutional neural networks outperform state-of-the-art vision-based corrosion detection approaches that are developed based on texture and color analysis using a simple multilayered perceptron network. Furthermore, it is shown that one of the proposed convolutional neural networks significantly improves the computational time in contrast with state-of-the-art pretrained convolutional neural networks while maintaining comparable performance for corrosion detection.

Sign in / Sign up

Export Citation Format

Share Document