Specification of Training Sets and the Number of Hidden Neurons for Multilayer Perceptrons

2001 ◽  
Vol 13 (12) ◽  
pp. 2673-2680 ◽  
Author(s):  
L. S. Camargo ◽  
T. Yoneyama
Keyword(s):  
Chebyshev Polynomials ◽  
Main Idea ◽  
Multilayer Perceptrons ◽  
Input Output ◽  
Real Functions ◽  
Bounded Functions ◽  
Hidden Layer ◽  
Hidden Neurons ◽  
Selection Of ◽  
Training Sets

This work concerns the selection of input-output pairs for improved training of multilayer perceptrons, in the context of approximation of univariate real functions. A criterion for the choice of the number of neurons in the hidden layer is also provided. The main idea is based on the fact that Chebyshev polynomials can provide approximations to bounded functions up to a prescribed tolerance, and, in turn, a polynomial of a certain order can be fitted with a three-layer perceptron with a prescribed number of hidden neurons. The results are applied to a sensor identification example.

Translated Multiplicative Neuron: An Extended Multiplicative Neuron that can Translate Decision Surfaces

2004 ◽  
Vol 8 (5) ◽  
pp. 460-468 ◽  
Author(s):  
Eduardo Masato Iyoda ◽  
◽  
Hajime Nobuhara ◽  
Kaoru Hirota
Keyword(s):  
Neural Networks ◽  
Neuron Model ◽  
Benchmark Problems ◽  
Multilayer Perceptrons ◽  
Input Space ◽  
Learning Rules ◽  
Hidden Layer ◽  
Almost All ◽  
Hidden Neurons ◽  
Better Than

A multiplicative neuron model called translated multiplicative neuron (πt-neuron) is proposed. Compared to the traditional π-neuron, the πt-neuron presents 2 advantages: (1) it can generate decision surfaces centered at any point of its input space; and (2) πt-neuron has a meaningful set of adjustable parameters. Learning rules for πt-neurons are derived using the error backpropagation procedure. It is shown that the XOR and N-bit parity problems can be perfectly solved using only 1 πt-neuron, with no need for hidden neurons. The πt-neuron is also evaluated in Hwang's regression benchmark problems, in which neural networks composed of πt-neurons in the hidden layer can perform better than conventional multilayer perceptrons (MLP) in almost all cases: Errors are reduced an average of 58% using about 33% fewer hidden neurons than MLP.

METHODS OF TRAINING AND CONSTRUCTING MULTILAYER PERCEPTRONS WITH ARBITRARY PATTERN SETS

1995 ◽  
Vol 06 (03) ◽  
pp. 233-247 ◽  
Author(s):  
XUN LIANG ◽  
SHAOWEI XIA
Keyword(s):  
Local Minimum ◽  
Back Propagation ◽  
Multilayer Perceptrons ◽  
Local Minima ◽  
Global Minima ◽  
Compensation Methods ◽  
Input Layer ◽  
Binary Input ◽  
Hidden Layer ◽  
Hidden Neurons

This paper presents two compensation methods for multilayer perceptrons (MLPs) which are very difficult to train by traditional Back Propagation (BP) methods. For MLPs trapped in local minima, compensating methods can correct the wrong outputs one by one using constructing techniques until all outputs are right, so that the MLPs can skip from the local minima to the global minima. A hidden neuron is added as compensation for a binary input three-layer perceptron trapped in a local minimum; and one or two hidden neurons are added as compensation for a real input three-layer perceptron. For a perceptron of more than three layers, the second hidden layer from behind will be temporarily treated as the input layer during compensation, hence the above methods can also be used. Examples are given.

scholarly journals Exploiting heterogeneity in operational neural networks by synaptic plasticity

2021 ◽  
Author(s):  
Serkan Kiranyaz ◽  
Junaid Malik ◽  
Habib Ben Abdallah ◽  
Turker Ince ◽  
Alexandros Iosifidis ◽  
...  
Keyword(s):  
Neural Networks ◽  
Synaptic Plasticity ◽  
Network Model ◽  
Neuron Model ◽  
Linear Operators ◽  
Training Data ◽  
Learning Performance ◽  
Minimal Network ◽  
Hidden Layer ◽  
Hidden Neurons

AbstractThe recently proposed network model, Operational Neural Networks (ONNs), can generalize the conventional Convolutional Neural Networks (CNNs) that are homogenous only with a linear neuron model. As a heterogenous network model, ONNs are based on a generalized neuron model that can encapsulate any set of non-linear operators to boost diversity and to learn highly complex and multi-modal functions or spaces with minimal network complexity and training data. However, the default search method to find optimal operators in ONNs, the so-called Greedy Iterative Search (GIS) method, usually takes several training sessions to find a single operator set per layer. This is not only computationally demanding, also the network heterogeneity is limited since the same set of operators will then be used for all neurons in each layer. To address this deficiency and exploit a superior level of heterogeneity, in this study the focus is drawn on searching the best-possible operator set(s) for the hidden neurons of the network based on the “Synaptic Plasticity” paradigm that poses the essential learning theory in biological neurons. During training, each operator set in the library can be evaluated by their synaptic plasticity level, ranked from the worst to the best, and an “elite” ONN can then be configured using the top-ranked operator sets found at each hidden layer. Experimental results over highly challenging problems demonstrate that the elite ONNs even with few neurons and layers can achieve a superior learning performance than GIS-based ONNs and as a result, the performance gap over the CNNs further widens.

Input Vector Identification and System Model Construction by Average Mutual Information

2000 ◽  
Author(s):  
Paul B. Deignan ◽  
Peter H. Meckl ◽  
Matthew A. Franchek ◽  
Salim A. Jaliwala ◽  
George G. Zhu
Keyword(s):  
Mutual Information ◽  
System Model ◽  
Input Output ◽  
Output Data ◽  
Average Mutual Information ◽  
Virtual Sensors ◽  
Input Signals ◽  
Model Independent ◽  
Intelligent Model ◽  
Selection Of

Abstract A methodology for the intelligent, model-independent selection of an appropriate set of input signals for the system identification of an unknown process is demonstrated. In modeling this process, it is shown that the terms of a simple nonlinear polynomial model may also be determined through the analysis of the average mutual information between inputs and the output. Average mutual information can be thought of as a nonlinear correlation coefficient and can be calculated from input/output data alone. The methodology described here is especially applicable to the development of virtual sensors.

scholarly journals Extreme Learning Machines on High Dimensional and Large Data Applications: A Survey

2015 ◽  
Vol 2015 ◽  
pp. 1-13 ◽  
Author(s):  
Jiuwen Cao ◽  
Zhiping Lin
Keyword(s):  
Linear System ◽  
Video Processing ◽  
Large Data ◽  
Feedforward Neural Networks ◽  
High Dimensional ◽  
Recent Developments ◽  
Learning Machine ◽  
Hidden Layer ◽  
Hidden Neurons ◽  
The Cost

Extreme learning machine (ELM) has been developed for single hidden layer feedforward neural networks (SLFNs). In ELM algorithm, the connections between the input layer and the hidden neurons are randomly assigned and remain unchanged during the learning process. The output connections are then tuned via minimizing the cost function through a linear system. The computational burden of ELM has been significantly reduced as the only cost is solving a linear system. The low computational complexity attracted a great deal of attention from the research community, especially for high dimensional and large data applications. This paper provides an up-to-date survey on the recent developments of ELM and its applications in high dimensional and large data. Comprehensive reviews on image processing, video processing, medical signal processing, and other popular large data applications with ELM are presented in the paper.

An Effective Solution to Regression Problem by RBF Neuron Network

2015 ◽  
Vol 6 (4) ◽  
pp. 57-74 ◽  
Author(s):  
Dang Thi Thu Hien ◽  
Hoang Xuan Huan ◽  
Le Xuan Minh Hoang
Keyword(s):  
Regression Function ◽  
Training Data ◽  
Neuron Network ◽  
Open Problems ◽  
Regression Problem ◽  
Rbf Network ◽  
Network Training ◽  
Hidden Layer ◽  
Definition Of ◽  
Selection Of

Radial Basis Function (RBF) neuron network is being applied widely in multivariate function regression. However, selection of neuron number for hidden layer and definition of suitable centre in order to produce a good regression network are still open problems which have been researched by many people. This article proposes to apply grid equally space nodes as the centre of hidden layer. Then, the authors use k-nearest neighbour method to define the value of regression function at the center and an interpolation RBF network training algorithm with equally spaced nodes to train the network. The experiments show the outstanding efficiency of regression function when the training data has Gauss white noise.

scholarly journals The proposal to transform an old limestone quarry into a botanical garden with a rainforest zone: a case study

2019 ◽  
Vol 28 (1) ◽  
pp. 131-142
Author(s):  
Daria Słonina ◽  
Grzegorz Kusza ◽  
Mateusz Mikołajów
Keyword(s):  
Main Idea ◽  
Botanical Garden ◽  
Original Research ◽  
Regional Culture ◽  
Great Role ◽  
Limestone Quarry ◽  
Landscape Type ◽  
Mining Areas ◽  
Selection Of

Nowadays, a significant part of cities is tackling the problems with post-mining areas. This manuscript is an original research which shows possibilities of their reclamation. The aim of the article is to present the proposal of developing the closed limestone quarry and creating a botanical garden. The proposed spatial solutions allow for creating a new, tourist and recreation space, maintaining the natural heritage. The work also assumed carrying out a dendrological inventory, in order to determine the existing dendrofl ora. The required spatial, nature and communication analyses, which illustrate the current condition of the area and defi ne further design works, have also been carried out. The main idea of the project was to maintain the particular biodiversity, combined with regional culture and its continuous development. This type of assumption aims not only at protection of endangered species. It also has a great role in shaping the awareness of natural environment of various social groups. The creation of a rainforest substitute in the Opole Botanical Garden was possible through selection of the existing afforestation, considering its adaptation as well as through liquidation and introduction of new trees, shrubs, perennial and climbing plants, which shall emphasise the tropical landscape type by their shapes, texture and colours. The project includes many elements, which reflect the general image of humid rainforests. The planned vegetation in connection with the appropriately selected architecture shall undoubtedly influence visitors’ senses, transferring them to the ‘wild’ and mysterious part of the world.

scholarly journals Frequency distribution of journalistic attention for scientific studies and scientific sources: An input – output analysis

2020 ◽  
Author(s):  
Markus Lehmkuhl ◽  
Nikolai Promies
Keyword(s):  
Power Law ◽  
Media Coverage ◽  
Social Impact ◽  
News Coverage ◽  
Input Output ◽  
Output Analysis ◽  
Input Output Analysis ◽  
Study Results ◽  
Scientific Results ◽  
Selection Of

Based on the decision-theoretical conditions underlying the selection of events for news coverage in science journalism, this article uses a novel input-output analysis to investigate which of the more than eight million scientific study results published between August 2014 and July 2018 have been selected by global journalism to a relevant degree. We are interested in two different structures in the media coverage of scientific results. Firstly, the structure of sources that journalists use, i.e. scientific journals, and secondly, the congruence of the journalistic selection of single results. Previous research suggests that the selection of sources and results follows a certain heavy-tailed distribution, a power law. Mathematically, this distribution can be described with a function of the form C*x-α. We argue that the exponent of such power law distributions can potentially be an indicator to describe selectivity in journalism on a high aggregation level. In our input-output analysis, we look for such patterns in the coverage of all scientific results published in the database Scopus over four years. To get an estimate of the coverage of these results, we use data from the altmetrics provider Altmetric, more precisely their Mainstream-Media-Score (MSM-Score). Based on exploratory analyses, we define papers with a score of 50 or above as Social Impact Papers (SIPs). Over our study period, we identified 5,833 SIPs published in 1,236 journals. We consider a power law fit with an exponent of about -2 to be plausible for the distribution of the source selection but cannot confirm the power law hypothesis for the distribution of the selection of single results. In this case, an exponentially truncated power law seems to be the better fit.

Sign in / Sign up

Export Citation Format

Share Document