Convergence of a modified gradient-based learning algorithm with penalty for single-hidden-layer feed-forward networks

2018 ◽  
Vol 32 (7) ◽  
pp. 2445-2456 ◽  
Author(s):  
Jian Wang ◽  
Bingjie Zhang ◽  
Zhaoyang Sang ◽  
Yusong Liu ◽  
Shujun Wu ◽  
...  
Keyword(s):  
Learning Algorithm ◽  
Feed Forward ◽  
Gradient Based ◽  
Hidden Layer

scholarly journals Unsupervised learning by competing hidden units

2019 ◽  
Vol 116 (16) ◽  
pp. 7723-7731 ◽  
Author(s):  
Dmitry Krotov ◽  
John J. Hopfield
Keyword(s):  
Learning Algorithm ◽  
Lower Layer ◽  
Learning Rule ◽  
Backpropagation Algorithm ◽  
Feedforward Networks ◽  
Feature Detectors ◽  
End To End ◽  
Hidden Layer ◽  
Full Network ◽  
Global Inhibition

It is widely believed that end-to-end training with the backpropagation algorithm is essential for learning good feature detectors in early layers of artificial neural networks, so that these detectors are useful for the task performed by the higher layers of that neural network. At the same time, the traditional form of backpropagation is biologically implausible. In the present paper we propose an unusual learning rule, which has a degree of biological plausibility and which is motivated by Hebb’s idea that change of the synapse strength should be local—i.e., should depend only on the activities of the pre- and postsynaptic neurons. We design a learning algorithm that utilizes global inhibition in the hidden layer and is capable of learning early feature detectors in a completely unsupervised way. These learned lower-layer feature detectors can be used to train higher-layer weights in a usual supervised way so that the performance of the full network is comparable to the performance of standard feedforward networks trained end-to-end with a backpropagation algorithm on simple tasks.

Study on Deep Structure of Extreme Learning Machine (DS-ELM) for Datasets with Noise

2014 ◽  
Vol 989-994 ◽  
pp. 3679-3682 ◽  
Author(s):  
Meng Meng Ma ◽  
Bo He
Keyword(s):  
Extreme Learning Machine ◽  
Simple Structure ◽  
Learning Algorithm ◽  
Deep Structure ◽  
Noisy Data ◽  
Machine Learning Algorithm ◽  
Feed Forward Neural Networks ◽  
Learning Machine ◽  
Hidden Layer ◽  
Auto Associative Neural Network

Extreme learning machine (ELM), a relatively novel machine learning algorithm for single hidden layer feed-forward neural networks (SLFNs), has been shown competitive performance in simple structure and superior training speed. To improve the effectiveness of ELM for dealing with noisy datasets, a deep structure of ELM, short for DS-ELM, is proposed in this paper. DS-ELM contains three level networks (actually contains three nets ): the first level network is trained by auto-associative neural network (AANN) aim to filter out noise as well as reduce dimension when necessary; the second level network is another AANN net aim to fix the input weights and bias of ELM; and the last level network is ELM. Experiments on four noisy datasets are carried out to examine the new proposed DS-ELM algorithm. And the results show that DS-ELM has higher performance than ELM when dealing with noisy data.

Roboterbasierte spanende Bearbeitung*/Robot-Based Milling Operation. Machine Learning Algorithm for a model-based feed-forward torque control

2019 ◽  
Vol 109 (05) ◽  
pp. 352-357
Author(s):  
C. Brecher ◽  
L. Gründel ◽  
L. Lienenlüke ◽  
S. Storms
Keyword(s):  
Machine Learning ◽  
Position Control ◽  
Learning Algorithm ◽  
Milling Process ◽  
Industrial Robots ◽  
Torque Control ◽  
Feed Forward ◽  
Control Loops ◽  
Model Based ◽  
Machine Learning Approach

Die Lageregelung von konventionellen Industrierobotern ist nicht auf den dynamischen Fräsprozess ausgelegt. Eine Möglichkeit, das Verhalten der Regelkreise zu optimieren, ist eine modellbasierte Momentenvorsteuerung, welche in dieser Arbeit aufgrund vieler Vorteile durch einen Machine-Learning-Ansatz erweitert wird. Hierzu wird die Umsetzung in Matlab und die simulative Evaluation erläutert, die im Anschluss das Potenzial dieses Konzeptes bestätigt.   The position control of conventional industrial robots is not designed for the dynamic milling process. One possibility to optimize the behavior of the control loops is a model-based feed-forward torque control which is supported by a machine learning approach due to many advantages. The implementation in Matlab and the simulative evaluation are explained, which subsequently confirms the potential of this concept.

scholarly journals A Gradient-Based Reinforcement Learning Algorithm for Multiple Cooperative Agents

IEEE Access ◽  
2018 ◽  
Vol 6 ◽  
pp. 70223-70235 ◽  
Author(s):  
Zhen Zhang ◽  
Dongqing Wang ◽  
Dongbin Zhao ◽  
Qiaoni Han ◽  
Tingting Song
Keyword(s):  
Reinforcement Learning ◽  
Learning Algorithm ◽  
Cooperative Agents ◽  
Gradient Based ◽  
Reinforcement Learning Algorithm

scholarly journals Fpga Implementation of Precise Convolutional Neural Network for Extreme Learning Machine

2020 ◽  
Vol 9 (8) ◽  
pp. 470-480
Keyword(s):  
Neural Network ◽  
Receptive Field ◽  
Convolutional Neural Network ◽  
Computation Time ◽  
Field Approach ◽  
Pixel Array ◽  
Gradient Based ◽  
Feed Forward Neural Networks ◽  
Learning Machine ◽  
Hidden Layer

Feed-forward neural networks can be trained based on a gradient-descent based backpropagation algorithm. But, these algorithms require more computation time. Extreme Learning Machines (ELM’s) are time-efficient, and they are less complicated than the conventional gradient-based algorithm. In previous years, an SRAM based convolutional neural network using a receptive – field Approach was proposed. This neural network was used as an encoder for the ELM algorithm and was implemented on FPGA. But, this neural network used an inaccurate 3-stage pipelined parallel adder. Hence, this neural network generates imprecise stimuli to the hidden layer neurons. This paper presents an implementation of precise convolutional neural network for encoding in the ELM algorithm based on the receptive - field approach at the hardware level. In the third stage of the pipelined parallel adder, instead of approximating the output by using one 2-input 15-bit adder, one 4-input 14-bit adder is used. Also, an additional weighted pixel array block is used. This weighted pixel array improves the accuracy of generating 128 weighted pixels. This neural network was simulated using ModelSim-Altera 10.1d and synthesized using Quartus II 13.0 sp1. This neural network is implemented on Cyclone V FPGA and used for pattern recognition applications. Although this design consumes slightly more hardware resources, this design is more accurate compared to previously existing encoders

Simulation of Compressor Transient Behavior Through Recurrent Neural Network Models

2005 ◽  
Vol 128 (3) ◽  
pp. 444-454 ◽  
Author(s):  
M. Venturini
Keyword(s):  
Measurement Errors ◽  
Learning Algorithm ◽  
Back Propagation ◽  
Network Models ◽  
Transient Behavior ◽  
Optimal Number ◽  
Measured Data ◽  
Outlet Temperature ◽  
Neural Network Models ◽  
Hidden Layer

In the paper, self-adapting models capable of reproducing time-dependent data with high computational speed are investigated. The considered models are recurrent feed-forward neural networks (RNNs) with one feedback loop in a recursive computational structure, trained by using a back-propagation learning algorithm. The data used for both training and testing the RNNs have been generated by means of a nonlinear physics-based model for compressor dynamic simulation, which was calibrated on a multistage axial-centrifugal small size compressor. The first step of the analysis is the selection of the compressor maneuver to be used for optimizing RNN training. The subsequent step consists in evaluating the most appropriate RNN structure (optimal number of neurons in the hidden layer and number of outputs) and RNN proper delay time. Then, the robustness of the model response towards measurement uncertainty is ascertained, by comparing the performance of RNNs trained on data uncorrupted or corrupted with measurement errors with respect to the simulation of data corrupted with measurement errors. Finally, the best RNN model is tested on field data taken on the axial-centrifugal compressor on which the physics-based model was calibrated, by comparing physics-based model and RNN predictions against measured data. The comparison between RNN predictions and measured data shows that the agreement can be considered acceptable for inlet pressure, outlet pressure and outlet temperature, while errors are significant for inlet mass flow rate.

scholarly journals Feed Forward Neural Network Modeling for Rainfall Prediction

2018 ◽  
Vol 73 ◽  
pp. 05017
Author(s):  
Yasin Hasbi ◽  
Warsito Budi ◽  
Santoso Rukun
Keyword(s):  
Neural Network ◽  
Autocorrelation Function ◽  
Seasonal Pattern ◽  
Optimization Method ◽  
Rainfall Data ◽  
Neural Network Modeling ◽  
Least Mean Square ◽  
Feed Forward Neural Network ◽  
Feed Forward ◽  
Hidden Layer

Prediction of rainfall data by using Feed Forward Neural Network (FFNN) model is proposed. FFNN is a class of neural network which has three layers for processing. In time series prediction, including in case of rainfall data, the input layer is the past values of the same series up to certain lag and the output layer is the current value. Beside a few lagged times, the seasonal pattern also considered as an important aspect of choosing the potential input. The autocorrelation function and partial autocorrelation function patterns are used as aid of selecting the input. In the second layer called hidden layer, the logistic sigmoid is used as activation function because of the monotonic and differentiable. Processing is done by the weighted summing of the input variables and transfer process in the hidden layer. Backpropagation algorithm is applied in the training process. Some gradient based optimization methods are used to obtain the connection weights of FFNN model. The prediction is the output resulting of the process in the last layer. In each optimization method, the looping process is performed several times in order to get the most suitable result in various composition of separating data. The best one is chosen by the least mean square error (MSE) criteria. The least of in-sample and out-sample predictions from the repeating results been the base of choosing the best optimization method. In this study, the model is applied in the ten-daily rainfall data of ZOM 136 Cokrotulung Klaten. Simulation results give a consecution that the more complex architecture is not guarantee the better prediction.

scholarly journals IMPLEMENTASI BACKPROPAGATION NEURAL NETWORK DALAM PRAKIRAAN CUACA DI DAERAH BALI SELATAN

2016 ◽  
Vol 5 (4) ◽  
pp. 126 ◽  
Author(s):  
I MADE DWI UDAYANA PUTRA ◽  
G. K. GANDHIADI ◽  
LUH PUTU IDA HARINI
Keyword(s):  
Neural Network ◽  
Artificial Neural Network ◽  
Learning Algorithm ◽  
Weather Forecast ◽  
Weather Forecasts ◽  
Forecast Models ◽  
Artificial Neural Network Method ◽  
Artificial Neural ◽  
Hidden Layer ◽  
Backpropagation Learning

Weather information has an important role in human life in various fields, such as agriculture, marine, and aviation. The accurate weather forecasts are needed in order to improve the performance of various fields. In this study, use artificial neural network method with backpropagation learning algorithm to create a model of weather forecasting in the area of ??South Bali. The aim of this study is to determine the effect of the number of neurons in the hidden layer and to determine the level of accuracy of the method of artificial neural network with backpropagation learning algorithm in weather forecast models. Weather forecast models in this study use input of the factors that influence the weather, namely air temperature, dew point, wind speed, visibility, and barometric pressure.The results of testing the network with a different number of neurons in the hidden layer of artificial neural network method with backpropagation learning algorithms show that the increase in the number of neurons in the hidden layer is not directly proportional to the value of the accuracy of the weather forecasts, the increase in the number of neurons in the hidden layer does not necessarily increase or decrease value accuracy of weather forecasts we obtain the best accuracy rate of 51.6129% on a network model with three neurons in the hidden layer.

Sign in / Sign up

Export Citation Format

Share Document