scholarly journals Parsimonious Optimization of Multitask Neural Network Hyperparameters

Molecules ◽  
2021 ◽  
Vol 26 (23) ◽  
pp. 7254
Author(s):  
Cecile Valsecchi ◽  
Viviana Consonni ◽  
Roberto Todeschini ◽  
Marco Emilio Orlandi ◽  
Fabio Gosetti ◽  
...  
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Genetic Algorithms ◽  
Network Performance ◽  
Random Search ◽  
Quantitative Structure Activity Relationship ◽  
Computational Time ◽  
Grid Search ◽  
Design Strategies ◽  
Computational Point

Neural networks are rapidly gaining popularity in chemical modeling and Quantitative Structure–Activity Relationship (QSAR) thanks to their ability to handle multitask problems. However, outcomes of neural networks depend on the tuning of several hyperparameters, whose small variations can often strongly affect their performance. Hence, optimization is a fundamental step in training neural networks although, in many cases, it can be very expensive from a computational point of view. In this study, we compared four of the most widely used approaches for tuning hyperparameters, namely, grid search, random search, tree-structured Parzen estimator, and genetic algorithms on three multitask QSAR datasets. We mainly focused on parsimonious optimization and thus not only on the performance of neural networks, but also the computational time that was taken into account. Furthermore, since the optimization approaches do not directly provide information about the influence of hyperparameters, we applied experimental design strategies to determine their effects on the neural network performance. We found that genetic algorithms, tree-structured Parzen estimator, and random search require on average 0.08% of the hours required by grid search; in addition, tree-structured Parzen estimator and genetic algorithms provide better results than random search.

scholarly journals Hyperparameters optimization for ResNet and Xception in the purpose of diagnosing COVID-19

2021 ◽  
pp. 1-17
Author(s):  
Hania H. Farag ◽  
Lamiaa A. A. Said ◽  
Mohamed R. M. Rizk ◽  
Magdy Abd ElAzim Ahmed
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Deep Learning ◽  
Convolutional Neural Network ◽  
Random Search ◽  
Learning Networks ◽  
Residual Network ◽  
Global Pandemic ◽  
Search Optimization

COVID-19 has been considered as a global pandemic. Recently, researchers are using deep learning networks for medical diseases’ diagnosis. Some of these researches focuses on optimizing deep learning neural networks for enhancing the network accuracy. Optimizing the Convolutional Neural Network includes testing various networks which are obtained through manually configuring their hyperparameters, then the configuration with the highest accuracy is implemented. Each time a different database is used, a different combination of the hyperparameters is required. This paper introduces two COVID-19 diagnosing systems using both Residual Network and Xception Network optimized by random search in the purpose of finding optimal models that give better diagnosis rates for COVID-19. The proposed systems showed that hyperparameters tuning for the ResNet and the Xception Net using random search optimization give more accurate results than other techniques with accuracies 99.27536% and 100 % respectively. We can conclude that hyperparameters tuning using random search optimization for either the tuned Residual Network or the tuned Xception Network gives better accuracies than other techniques diagnosing COVID-19.

Modularity in Artificial Neural Networks

2011 ◽  
pp. 1095-1101
Author(s):  
Ricardo Téllez ◽  
Cecilio Angulo
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Artificial Neural Networks ◽  
Complex Systems ◽  
Intelligent Systems ◽  
Point Of View ◽  
Main Concern ◽  
Complex Structures ◽  
Computational Point ◽  
Points Of View

The concept of modularity is a main concern for the generation of artificially intelligent systems. Modularity is an ubiquitous organization principle found everywhere in natural and artificial complex systems (Callebaut, 2005). Evidences from biological and philosophical points of view (Caelli and Wen, 1999) (Fodor, 1983), indicate that modularity is a requisite for complex intelligent behaviour. Besides, from an engineering point of view, modularity seems to be the only way for the construction of complex structures. Hence, whether complex neural programs for complex agents are desired, modularity is required. This article introduces the concepts of modularity and module from a computational point of view, and how they apply to the generation of neural programs based on modules. Two levels, strategic and tactical, at which modularity can be implemented, are identified. How they work and how they can be combined for the generation of a completely modular controller for a neural network based agent is presented.

scholarly journals The Use of Neural Networks and Genetic Algorithms for Design of Groundwater Remediation Schemes

1997 ◽  
Vol 1 (2) ◽  
pp. 345-356 ◽  
Author(s):  
Z. Rao ◽  
D. G. Jamieson
Keyword(s):  
Neural Network ◽  
Genetic Algorithm ◽  
Neural Networks ◽  
Artificial Neural Network ◽  
Genetic Algorithms ◽  
Hybrid Algorithm ◽  
Large Scale ◽  
Groundwater Remediation ◽  
Injection Wells ◽  
Representative Problem

Abstract. The increasing incidence of groundwater pollution has led to recognition of a need to develop objective techniques for designing reniediation schemes. This paper outlines one such possibility for determining how many abstraction/injection wells are required, where they should be located etc., having regard to minimising the overall cost. To that end, an artificial neural network is used in association with a 2-D or 3-D groundwater simulation model to determine the performance of different combinations of abstraction/injection wells. Thereafter, a genetic algorithm is used to identify which of these combinations offers the least-cost solution to achieve the prescribed residual levels of pollutant within whatever timescale is specified. The resultant hybrid algorithm has been shown to be effective for a simplified but nevertheless representative problem; based on the results presented, it is expected the methodology developed will be equally applicable to large-scale, real-world situations.

Attempt at Resilient Modulus Modeling Using Artificial Neural Networks

1996 ◽  
Vol 1540 (1) ◽  
pp. 1-6 ◽  
Author(s):  
Erol Tutumluer ◽  
Roger W. Meier
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Artificial Neural Networks ◽  
Network Performance ◽  
Resilient Modulus ◽  
Quality Of Data ◽  
Unsuccessful Attempt ◽  
Independent Test ◽  
Artificial Neural

The pitfalls inherent in the indiscriminate application of artificial neural networks to numerical modeling problems are illustrated. An example is used of an apparently successful (but ultimately unsuccessful) attempt at training a neural network constitutive model for computing the resilient modulus of gravels as a function of stress state and various material properties. Issues such as the quantity and quality of data needed to successfully train a neural network are explored, and the importance of an independent test set to verify network performance is examined.

scholarly journals Development of a graph convolutional neural network model for efficient prediction of protein-ligand binding affinities

PLoS ONE ◽  
2021 ◽  
Vol 16 (4) ◽  
pp. e0249404
Author(s):  
Jeongtae Son ◽  
Dongsup Kim
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Drug Discovery ◽  
Ligand Binding ◽  
Convolutional Neural Network ◽  
Convolutional Neural Networks ◽  
Data Augmentation ◽  
Docking Simulation ◽  
Computational Time ◽  
Binding Affinities

Prediction of protein-ligand interactions is a critical step during the initial phase of drug discovery. We propose a novel deep-learning-based prediction model based on a graph convolutional neural network, named GraphBAR, for protein-ligand binding affinity. Graph convolutional neural networks reduce the computational time and resources that are normally required by the traditional convolutional neural network models. In this technique, the structure of a protein-ligand complex is represented as a graph of multiple adjacency matrices whose entries are affected by distances, and a feature matrix that describes the molecular properties of the atoms. We evaluated the predictive power of GraphBAR for protein-ligand binding affinities by using PDBbind datasets and proved the efficiency of the graph convolution. Given the computational efficiency of graph convolutional neural networks, we also performed data augmentation to improve the model performance. We found that data augmentation with docking simulation data could improve the prediction accuracy although the improvement seems not to be significant. The high prediction performance and speed of GraphBAR suggest that such networks can serve as valuable tools in drug discovery.

scholarly journals NEAT Snake: a both evolutionary and neural network adaptation approach

2020 ◽  
Author(s):  
Alisson Steffens Henrique ◽  
Vinicius Almeida dos Santos ◽  
Rodrigo Lyra
Keyword(s):  
Neural Network ◽  
Genetic Algorithm ◽  
Neural Networks ◽  
Decision Making ◽  
Genetic Algorithms ◽  
Real Time ◽  
Relative Position ◽  
Network Adaptation ◽  
The Neural Network

There are several challenges when modeling artificial intelligencemethods for autonomous players on games (bots). NEAT is one ofthe models that, combining genetic algorithms and neural networks,seek to describe a bot behavior more intelligently. In NEAT, a neuralnetwork is used for decision making, taking relevant inputs fromthe environment and giving real-time decisions. In a more abstractway, a genetic algorithm is applied for the learning step of the neuralnetworks’ weights, layers, and parameters. This paper proposes theuse of relative position as the input of the neural network, basedon the hypothesis that the bot profit will be improved.

scholarly journals Model Classification Of Nominal Value And The Original Of IDR Money By Applying Evolutionary Neural Network

2020 ◽  
Vol 3 (2) ◽  
pp. 258-265
Author(s):  
Al-Khowarizmi Al-Khowarizmi
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Genetic Algorithms ◽  
Object Recognition ◽  
Success Rate ◽  
Model Classification ◽  
Evolutionary Neural Network ◽  
Evolutionary Neural Networks

Indonesian Rupiah (IDR) banknotes have unique characteristics that distinguish them from one another, both in the form of numbers, zeros and background images. This pattern of each type of banknote will be modeled in order to test the nominal value and authenticity of IDR, so as to be able to distinguish not only IDR banknotes but also other denominations. Evolutionary Neural Network is the development of the concept of evolution to get a neural network (NN) using genetic algorithms (GA). In this paper the application of evolutionary neural networks with less input is able to have a better success rate in object recognition, because the parameters for producing neural networks are far better

Performance of Multi-Layer Feedforward Neural Networks to Predict Liver Transplantation Outcome

1996 ◽  
Vol 35 (01) ◽  
pp. 12-18 ◽  
Author(s):  
M. Subotin ◽  
W. Marsh ◽  
J. McMichael ◽  
J. J. Fung ◽  
I. Dvorchik
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Liver Transplantation ◽  
Missing Data ◽  
Network Performance ◽  
Missing Values ◽  
Feedforward Neural Networks ◽  
Linear Scaling ◽  
Data Set ◽  
New Approach

AbstractA novel multisolutional clustering and quantization (MCO) algorithm has been developed that provides a flexible way to preprocess data. It was tested whether it would impact the neural network’s performance favorably and whether the employment of the proposed algorithm would enable neural networks to handle missing data. This was assessed by comparing the performance of neural networks using a well-documented data set to predict outcome following liver transplantation. This new approach to data preprocessing leads to a statistically significant improvement in network performance when compared to simple linear scaling. The obtained results also showed that coding missing data as zeroes in combination with the MCO algorithm, leads to a significant improvement in neural network performance on a data set containing missing values in 59.4% of cases when compared to replacement of missing values with either series means or medians.

Using a Hybrid System Composed of Neural Networks and Genetic Algorithms for Financial Forecasting

2015 ◽  
pp. 55-62
Author(s):  
Mihaela Dumitrescu
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Genetic Algorithms ◽  
Network Architecture ◽  
Back Propagation ◽  
Financial Forecasting ◽  
Forecast Horizon ◽  
Genetic Search ◽  
Adaptive Processes ◽  
Artificial Neural

The possibility of applying artificial neural networks in different areas determined the discovery of more complex structures. This chapter describes the characteristic aspects of using a back-propagation neural network algorithm in making financial forecasting improved by a different technology: genetic algorithms. These can help build an automatic artificial neural network by two adaptive processes: first, genetic search through the data entry window, the forecast horizon, network architecture space, and control parameters to select the best performers; second, back propagation learning in individual networks to evaluate the selected architectures. Thus, network performance population increases from generation to generation. This chapter also presents how genetic algorithms can be used both to find the best network architecture and to find the right combination of inputs, the best prediction horizon and the most effective weight. Finally, this study shows how the results obtained using these technologies can be applied to obtain decision support systems that can lead to increased performance in economic activity and financial projections.

Breast Cancer Diagnosis Using Optimized Attribute Division in Modular Neural Networks

2013 ◽  
pp. 34-47
Author(s):  
Rahul Kala ◽  
Anupam Shukla ◽  
Ritu Tiwari
Keyword(s):  
Breast Cancer ◽  
Neural Network ◽  
Neural Networks ◽  
Genetic Algorithms ◽  
Cancer Diagnosis ◽  
Breast Cancer Diagnosis ◽  
Optimal Distribution ◽  
Modular Neural Networks ◽  
Novel Concept ◽  
The Individual

The complexity of problems has led to a shift toward the use of modular neural networks in place of traditional neural networks. The number of inputs to neural networks must be kept within manageable limits to escape from the curse of dimensionality. Attribute division is a novel concept to reduce the problem dimensionality without losing information. In this paper, the authors use Genetic Algorithms to determine the optimal distribution of the parameters to the various modules of the modular neural network. The attribute set is divided into the various modules. Each module computes the output using its own list of attributes. The individual results are then integrated by an integrator. This framework is used for the diagnosis of breast cancer. Experimental results show that optimal distribution strategy exceeds the well-known methods for the diagnosis of the disease.

Sign in / Sign up

Export Citation Format

Share Document