How to Train Multilayer Perceptrons Efficiently With Large Data Sets

2011 ◽  
pp. 186-206
Author(s):  
Hyeyoung Park
Keyword(s):  
Neural Networks ◽  
Computational Cost ◽  
Large Data ◽  
Large Data Sets ◽  
Multilayer Perceptrons ◽  
Data Sets ◽  
Learning Method ◽  
Learning Speed ◽  
Gradient Learning ◽  
On Line Learning

Feed forward neural networks or multilayer perceptrons have been successfully applied to a number of difficult and diverse applications by using the gradient descent learning method known as the error backpropagation algorithm. However, it is known that the backpropagation method is extremely slow in many cases mainly due to plateaus. In data mining, the data set is usually large and the slow learning speed of neural networks is a critical defect. In this chapter, we present an efficient on-line learning method called adaptive natural gradient learning. It can solve the plateau problems, and can be successfully applied to the learning associated with large data sets. We compare the presented method with various popular learning algorithms with the aim of improving the learning speed and discuss briefly the merits and defects of each method so that one can get some guidance as to the choice of the proper method for a given application. In addition, we also give a number of technical tips, which can be easily implemented with low computational cost and can sometimes make a remarkable improvement in the learning speed.

scholarly journals Generation of geometric interpolations of building types with deep variational autoencoders

2020 ◽  
Vol 6 ◽  
Author(s):  
Jaime de Miguel Rodríguez ◽  
Maria Eugenia Villafañe ◽  
Luka Piškorec ◽  
Fernando Sancho Caparrini
Keyword(s):  
Machine Learning ◽  
Neural Networks ◽  
Large Data ◽  
Learning Model ◽  
Large Data Sets ◽  
Data Sets ◽  
Connectivity Map ◽  
Data Set ◽  
3D Objects ◽  
Machine Learning Model

Abstract This work presents a methodology for the generation of novel 3D objects resembling wireframes of building types. These result from the reconstruction of interpolated locations within the learnt distribution of variational autoencoders (VAEs), a deep generative machine learning model based on neural networks. The data set used features a scheme for geometry representation based on a ‘connectivity map’ that is especially suited to express the wireframe objects that compose it. Additionally, the input samples are generated through ‘parametric augmentation’, a strategy proposed in this study that creates coherent variations among data by enabling a set of parameters to alter representative features on a given building type. In the experiments that are described in this paper, more than 150 k input samples belonging to two building types have been processed during the training of a VAE model. The main contribution of this paper has been to explore parametric augmentation for the generation of large data sets of 3D geometries, showcasing its problems and limitations in the context of neural networks and VAEs. Results show that the generation of interpolated hybrid geometries is a challenging task. Despite the difficulty of the endeavour, promising advances are presented.

scholarly journals Artificial Intelligence Explained for Nonexperts

2020 ◽  
Vol 24 (01) ◽  
pp. 003-011 ◽  
Author(s):  
Narges Razavian ◽  
Florian Knoll ◽  
Krzysztof J. Geras
Keyword(s):  
Artificial Intelligence ◽  
Neural Networks ◽  
Clinical Practice ◽  
Medical Imaging ◽  
Deep Neural Networks ◽  
Large Data ◽  
Large Data Sets ◽  
Natural Images ◽  
Data Sets ◽  
Current State

AbstractArtificial intelligence (AI) has made stunning progress in the last decade, made possible largely due to the advances in training deep neural networks with large data sets. Many of these solutions, initially developed for natural images, speech, or text, are now becoming successful in medical imaging. In this article we briefly summarize in an accessible way the current state of the field of AI. Furthermore, we highlight the most promising approaches and describe the current challenges that will need to be solved to enable broad deployment of AI in clinical practice.

Modular Grammatical Evolution for the Generation of Artificial Neural Networks

2021 ◽  
pp. 1-36
Author(s):  
Khabat Soltanian ◽  
Ali Ebnenasir ◽  
Mohsen Afsharchi
Keyword(s):  
Neural Networks ◽  
State Of The Art ◽  
Solution Space ◽  
Single Layer ◽  
Large Data ◽  
Grammatical Evolution ◽  
Large Data Sets ◽  
Data Sets ◽  
Novel Method ◽  
Weak Locality

Abstract This paper presents a novel method, called Modular Grammatical Evolution (MGE), towards validating the hypothesis that restricting the solution space of NeuroEvolution to modular and simple neural networks enables the efficient generation of smaller and more structured neural networks while providing acceptable (and in some cases superior) accuracy on large data sets. MGE also enhances the state-of-the-art Grammatical Evolution (GE) methods in two directions. First, MGE's representation is modular in that each individual has a set of genes, and each gene is mapped to a neuron by grammatical rules. Second, the proposed representation mitigates two important drawbacks of GE, namely the low scalability and weak locality of representation, towards generating modular and multi-layer networks with a high number of neurons. We define and evaluate five different forms of structures with and without modularity using MGE and find single-layer modules with no coupling more productive. Our experiments demonstrate that modularity helps in finding better neural networks faster. We have validated the proposed method using ten well-known classification benchmarks with different sizes, feature counts, and output class counts. Our experimental results indicate that MGE provides superior accuracy with respect to existing NeuroEvolution methods and returns classifiers that are significantly simpler than other machine learning generated classifiers. Finally, we empirically demonstrate that MGE outperforms other GE methods in terms of locality and scalability properties.

Methodological support of the expert system in the problem of interaction of business ecosystems

2021 ◽  
Vol 4 ◽  
pp. 47-53
Author(s):  
K. V. Simonov ◽  
◽  
V. V. Kuimov ◽  
M. V. Kobalinsky ◽  
S. V. Kirillova ◽  
...  
Keyword(s):  
Neural Networks ◽  
Expert System ◽  
Business Models ◽  
Quantitative Description ◽  
Large Data ◽  
Large Data Sets ◽  
Data Sets ◽  
Effective Solution ◽  
Business Ecosystems ◽  
The Government

The paper discusses modern approaches and digital transformations in business models and interactions. In this regard for a quantitative description of interactions in ecosystems a variant of methodological support based on neural networks is proposed for fast nonlinear multiparametric regression of large data sets within the projected expert system. The possibility of effective solution of the problem of filling gaps in the observational data arrays and processing of not precisely specified information is shown. This approach is proposed for solving predictive problems in the problem of interaction of objects of interest in business ecosystems. The article was prepared within the framework of the Grant of the RFBR and the Government of the Krasnoyarsk Territory No. 20-410-242916 / 20 r_mk Krasnoyarsk.

Feed Forward Backpropagation Neural Networks and their Use in Predicting the Acute Toxicity of Chemicals to the Fathead Minnow

1997 ◽  
Vol 32 (3) ◽  
pp. 637-658 ◽  
Author(s):  
Klaus L.E. Kaiser ◽  
Stefan P. Niculescu ◽  
Gerrit Schüürmann
Keyword(s):  
Neural Networks ◽  
Fathead Minnow ◽  
Large Data ◽  
Large Data Sets ◽  
Data Sets ◽  
Explicit Equation ◽  
Data Set ◽  
Feed Forward ◽  
Backpropagation Neural Networks ◽  
Water Partition Coefficient

Abstract Various aspects connected to the use of feed forward backpropagation neural networks to build multivariate QSARs based on large data sets containing considerable amounts of important information are investigated. Based on such a model and a 419 compound data set, the explicit equation of one of the resulting multivariate QSARs for the computation of toxicity to the fathead minnow is presented as function of measured Microtox, logarithms of molecular weight and octanol/water partition coefficient, and 48 other functional group and discrete descriptors.

scholarly journals Advanced Interferometric Techniques for High-Resolution Bathymetry

2012 ◽  
Vol 46 (2) ◽  
pp. 9-31 ◽  
Author(s):  
Gerard Llort-Pujol ◽  
Christophe Sintes ◽  
Thierry Chonavel ◽  
Archie T. Morrison ◽  
Sylvie Daniel
Keyword(s):  
High Resolution ◽  
Computational Cost ◽  
Large Data ◽  
Large Data Sets ◽  
Data Sets ◽  
Music Algorithm ◽  
Multiple Signal Classification ◽  
Array Configuration ◽  
Complex Correlation ◽  
High Resolution Bathymetry

AbstractCurrent high-resolution sidescan and multibeam sonars produce very large data sets. However, conventional interferometry-based bathymetry algorithms underestimate the potential information of such soundings, generally because they use small baselines to avoid phase ambiguity. Moreover, these algorithms limit the triangulation capabilities of multibeam echosounders (MBES) to the detection of one sample per beam, i.e., the zero-phase instant. In this paper, we argue that the correlation between signals plays a very important role in the exploration of a remotely observed scene. In the case of multibeam sonars, capabilities can be improved by using the interferometric signal as a continuous quantity. This allows consideration of many more useful soundings per beam and enriches understanding of the environment. To this end, continuous interferometry detection is compared here, from a statistical perspective, first with conventional interferometry-based algorithms and then with high-resolution methods such as the Multiple Signal Classification (MUSIC) algorithm. We demonstrate that a well-designed interferometry algorithm based on a coherence error model and an optimal array configuration permits a reduction in the number of beam formings (and therefore the computational cost) and an improvement in target detection (such as ship mooring cables or masts). A possible interferometry processing algorithm based on the complex correlation between received signals is tested on both sidescan sonars and MBESs and shows promising results for detection of small in-water targets.

Sign in / Sign up

Export Citation Format

Share Document