scholarly journals Convergence Analysis of An Improved Extreme Learning Machine Based on Gradient Descent Method

2016 ◽  
Vol 8 (1) ◽  
pp. 5-15
Author(s):  
Liu Yusong ◽  
Su Zhixun ◽  
Zhang Bingjie ◽  
Gong Xiaoling ◽  
Sang Zhaoyang
Keyword(s):  
Extreme Learning Machine ◽  
Learning Algorithm ◽  
Error Function ◽  
Gradient Descent Method ◽  
Testing Time ◽  
Training Procedure ◽  
Feed Forward Neural Network ◽  
Learning Machine ◽  
Theoretical Results ◽  
Hidden Nodes

Abstract Extreme learning machine (ELM) is an efficient algorithm, but it requires more hidden nodes than the BP algorithms to reach the matched performance. Recently, an efficient learning algorithm, the upper-layer-solution-unaware algorithm (USUA), is proposed for the single-hidden layer feed-forward neural network. It needs less number of hidden nodes and testing time than ELM. In this paper, we mainly give the theoretical analysis for USUA. Theoretical results show that the error function monotonously decreases in the training procedure, the gradient of the error function with respect to weights tends to zero (the weak convergence), and the weight sequence goes to a fixed point (the strong convergence) when the iterations approach positive infinity. An illustrated simulation has been implemented on the MNIST database of handwritten digits which effectively verifies the theoretical results..

scholarly journals Rank based Pseudoinverse Computation in Extreme Learning Machine for large Datasets

2019 ◽  
Vol 8 (10) ◽  
pp. 1341-1346
Keyword(s):  
Extreme Learning Machine ◽  
Learning Algorithm ◽  
Matrix Decomposition ◽  
Least Square ◽  
Feed Forward Neural Network ◽  
Worst Case ◽  
Training Time ◽  
Learning Machine ◽  
Hidden Layer ◽  
Hidden Nodes

Extreme Learning Machine (ELM) is an efficient and effective least-square-based learning algorithm for classification, regression problems based on single hidden layer feed-forward neural network (SLFN). It has been shown in the literature that it has faster convergence and good generalization ability for moderate datasets. But, there is great deal of challenge involved in computing the pseudoinverse when there are large numbers of hidden nodes or for large number of instances to train complex pattern recognition problems. To address this problem, a few approaches such as EM-ELM, DF-ELM have been proposed in the literature. In this paper, a new rank-based matrix decomposition of the hidden layer matrix is introduced to have the optimal training time and reduce the computational complexity for a large number of hidden nodes in the hidden layer. The results show that it has constant training time which is closer towards the minimal training time and very far from worst-case training time of the DF-ELM algorithm that has been shown efficient in the recent literature.

scholarly journals Theory and Numerical Analysis of Extreme Learning Machine and Its Application for Different Degrees of Defect Recognition of Hoisting Wire Rope

2018 ◽  
Vol 2018 ◽  
pp. 1-13 ◽  
Author(s):  
Zhike Zhao ◽  
Xiaoguang Zhang
Keyword(s):  
Extreme Learning Machine ◽  
Performance Test ◽  
Research Problem ◽  
Wire Rope ◽  
Nondestructive Method ◽  
Experimental Results ◽  
Feed Forward Neural Network ◽  
Learning Machine ◽  
Hidden Layer ◽  
Hidden Nodes

An improved classification approach is proposed to solve the hot research problem of some complex multiclassification samples based on extreme learning machine (ELM). ELM was proposed based on the single-hidden layer feed-forward neural network (SLFNN). ELM is characterized by the easier parameter selection rules, the faster converge speed, the less human intervention, and so on. In order to further improve the classification precision of ELM, an improved generation method of the network structure of ELM is developed by dynamically adjusting the number of hidden nodes. The number change of the hidden nodes can serve as the computational updated step length of the ELM algorithm. In this paper, the improved algorithm can be called the variable step incremental extreme learning machine (VSI-ELM). In order to verify the effect of the hidden layer nodes on the performance of ELM, an open-source machine learning database (University of California, Irvine (UCI)) is provided by the performance test data sets. The regression and classification experiments are used to study the performance of the VSI-ELM model, respectively. The experimental results show that the VSI-ELM algorithm is valid. The classification of different degrees of broken wires is now still a problem in the nondestructive testing of hoisting wire rope. The magnetic flux leakage (MFL) method of wire rope is an efficient nondestructive method which plays an important role in safety evaluation. Identifying the proposed VSI-ELM model is effective and reliable for actually applying data, and it is used to identify the classification problem of different types of samples from MFL signals. The final experimental results show that the VSI-ELM algorithm is of faster classification speed and higher classification accuracy of different broken wires.

Comparison of the Extreme Learning Machine with the BP Neural Network for Short-Term Prediction of Wind Power

2012 ◽  
Vol 608-609 ◽  
pp. 564-568 ◽  
Author(s):  
Yi Hui Zhang ◽  
He Wang ◽  
Zhi Jian Hu ◽  
Meng Lin Zhang ◽  
Xiao Lu Gong ◽  
...  
Keyword(s):  
Neural Network ◽  
Wind Power ◽  
Extreme Learning Machine ◽  
Bp Neural Network ◽  
Learning Algorithm ◽  
Power Prediction ◽  
Feed Forward Neural Network ◽  
Wind Power Prediction ◽  
Learning Machine ◽  
Hidden Layer

Extreme learning machine (ELM) is a new and effective single-hidden layer feed forward neural network learning algorithm. Extreme learning machine only needs to set the number of hidden layer nodes of the network, and there is no need to adjust the neural network input weights and the hidden units bias, and it generates the only optimum solution, so it has the advantage of fast learning and good generalization ability. And the back propagation (BP) neural network is the most maturely applied. This paper has introduced the extreme learning machine into the wind power prediction. By comparing the wind power prediction method using the BP neural network. Study shows that the extreme learning machine has better prediction accuracy and shorter model training time.

scholarly journals Smoothing L0 Regularization for Extreme Learning Machine

2020 ◽  
Vol 2020 ◽  
pp. 1-10 ◽  
Author(s):  
Qinwei Fan ◽  
Ting Liu
Keyword(s):  
Extreme Learning Machine ◽  
Numerical Experiments ◽  
Learning Algorithm ◽  
Human Intervention ◽  
Feedforward Networks ◽  
Learning Machine ◽  
Hidden Layer ◽  
Efficient Learning ◽  
Hidden Nodes ◽  
Novel Algorithm

Extreme learning machine (ELM) has been put forward for single hidden layer feedforward networks. Because of its powerful modeling ability and it needs less human intervention, the ELM algorithm has been used widely in both regression and classification experiments. However, in order to achieve required accuracy, it needs many more hidden nodes than is typically needed by the conventional neural networks. This paper considers a new efficient learning algorithm for ELM with smoothing L0 regularization. A novel algorithm updates weights in the direction along which the overall square error is reduced the most and then this new algorithm can sparse network structure very efficiently. The numerical experiments show that the ELM algorithm with smoothing L0 regularization has less hidden nodes but better generalization performance than original ELM and ELM with L1 regularization algorithms.

scholarly journals Road Traffic Prediction Model Using Extreme Learning Machine: The Case Study of Tangier, Morocco

Information ◽  
2020 ◽  
Vol 11 (12) ◽  
pp. 542
Author(s):  
Mouna Jiber ◽  
Abdelilah Mbarek ◽  
Ali Yahyaouy ◽  
My Abdelouahed Sabri ◽  
Jaouad Boumhidi
Keyword(s):  
Extreme Learning Machine ◽  
Intelligent Transportation Systems ◽  
High Speed ◽  
Road Traffic ◽  
Learning Algorithm ◽  
Route Planning ◽  
Transportation Systems ◽  
Feed Forward Neural Network ◽  
Data Set ◽  
Learning Machine

An efficient and credible approach to road traffic management and prediction is a crucial aspect in the Intelligent Transportation Systems (ITS). It can strongly influence the development of road structures and projects. It is also essential for route planning and traffic regulations. In this paper, we propose a hybrid model that combines extreme learning machine (ELM) and ensemble-based techniques to predict the future hourly traffic of a road section in Tangier, a city in the north of Morocco. The model was applied to a real-world historical data set extracted from fixed sensors over a 5-years period. Our approach is based on a type of Single hidden Layer Feed-forward Neural Network (SLFN) known for being a high-speed machine learning algorithm. The model was, then, compared to other well-known algorithms in the prediction literature. Experimental results demonstrated that, according to the most commonly used criteria of error measurements (RMSE, MAE, and MAPE), our model is performing better in terms of prediction accuracy. The use of Akaike’s Information Criterion technique (AIC) has also shown that the proposed model has a higher performance.

Study on Deep Structure of Extreme Learning Machine (DS-ELM) for Datasets with Noise

2014 ◽  
Vol 989-994 ◽  
pp. 3679-3682 ◽  
Author(s):  
Meng Meng Ma ◽  
Bo He
Keyword(s):  
Extreme Learning Machine ◽  
Simple Structure ◽  
Learning Algorithm ◽  
Deep Structure ◽  
Noisy Data ◽  
Machine Learning Algorithm ◽  
Feed Forward Neural Networks ◽  
Learning Machine ◽  
Hidden Layer ◽  
Auto Associative Neural Network

Extreme learning machine (ELM), a relatively novel machine learning algorithm for single hidden layer feed-forward neural networks (SLFNs), has been shown competitive performance in simple structure and superior training speed. To improve the effectiveness of ELM for dealing with noisy datasets, a deep structure of ELM, short for DS-ELM, is proposed in this paper. DS-ELM contains three level networks (actually contains three nets ): the first level network is trained by auto-associative neural network (AANN) aim to filter out noise as well as reduce dimension when necessary; the second level network is another AANN net aim to fix the input weights and bias of ELM; and the last level network is ELM. Experiments on four noisy datasets are carried out to examine the new proposed DS-ELM algorithm. And the results show that DS-ELM has higher performance than ELM when dealing with noisy data.

Prediction of Software Defect Using Ensemble Learning Based Improved Sparrow Search Algorithm To Optimize Extreme Learning Machine

2021 ◽  
Author(s):  
Yu Tang ◽  
Qi Dai ◽  
Mengyuan Yang ◽  
Lifang Chen
Keyword(s):  
Extreme Learning Machine ◽  
Ensemble Learning ◽  
Learning Algorithm ◽  
Search Algorithm ◽  
Defect Prediction ◽  
Software Defect Prediction ◽  
Prediction Ability ◽  
Software Defect ◽  
Ensemble Learning Algorithm ◽  
Learning Machine

Abstract For the traditional ensemble learning algorithm of software defect prediction, the base predictor exists the problem that too many parameters are difficult to optimize, resulting in the optimized performance of the model unable to be obtained. An ensemble learning algorithm for software defect prediction that is proposed by using the improved sparrow search algorithm to optimize the extreme learning machine, which divided into three parts. Firstly, the improved sparrow search algorithm (ISSA) is proposed to improve the optimization ability and convergence speed, and the performance of the improved sparrow search algorithm is tested by using eight benchmark test functions. Secondly, ISSA is used to optimize extreme learning machine (ISSA-ELM) to improve the prediction ability. Finally, the optimized ensemble learning algorithm (ISSA-ELM-Bagging) is presented in the Bagging algorithm which improve the prediction performance of ELM in software defect datasets. Experiments are carried out in six groups of software defect datasets. The experimental results show that ISSA-ELM-Bagging ensemble learning algorithm is significantly better than the other four comparison algorithms under the six evaluation indexes of Precision, Recall, F-measure, MCC, Accuracy and G-mean, which has better stability and generalization ability.

scholarly journals Short Time Solar Power Forecasting Using Persistence Extreme Learning Machine Approach

2021 ◽  
Vol 294 ◽  
pp. 01002
Author(s):  
Xiaoyan Xiang ◽  
Yao Sun ◽  
Xiaofei Deng
Keyword(s):  
Solar Energy ◽  
Extreme Learning Machine ◽  
Power Output ◽  
Smart Grids ◽  
Large Scale ◽  
Solar Power ◽  
Learning Algorithm ◽  
Power Performance ◽  
Energy Prediction ◽  
Learning Machine

Solar energy in nature is irregular, so photovoltaic (PV) power performance is intermittent, and highly dependent on solar radiation, temperature and other meteorological parameters. Accurately predicting solar power to ensure the economic operation of micro-grids (MG) and smart grids is an important challenge to improve the large-scale application of PV to traditional power systems. In this paper, a hybrid machine learning algorithm is proposed to predict solar power accurately, and Persistence Extreme Learning Machine(P-ELM) algorithm is used to train the system. The input parameters are the temperature, sunshine and solar power output at the time of i, and the output parameters are the temperature, sunshine and solar power output at the time i+1. The proposed method can realize the prediction of solar power output 20 minutes in advance. Mean absolute error (MAE) and root-mean-square error (RMSE) are used to characterize the performance of P-ELM algorithm, and compared with ELM algorithm. The results show that the accuracy of P-ELM algorithm is better in short-term prediction, and P-ELM algorithm is very suitable for real-time solar energy prediction accuracy and reliability.

scholarly journals Forecasting Gold Prices Based on Extreme Learning Machine

2016 ◽  
Vol 11 (3) ◽  
pp. 372 ◽  
Author(s):  
Kumar Chandar Sivalingam ◽  
Sumathi Mahendran ◽  
Sivanandam Natarajan
Keyword(s):  
Extreme Learning Machine ◽  
Learning Algorithm ◽  
Back Propagation ◽  
Learning Ability ◽  
Stock Index ◽  
Feed Forward Back Propagation ◽  
Crude Oil Prices ◽  
The Future ◽  
Good Learning ◽  
Learning Machine

<p>In recent years, the investors pay major attention to invest in gold market ecause of huge profits in the future. Gold is the only commodity which maintains ts value even in the economic and financial crisis. Also, the gold prices are closely elated with other commodities. The future gold price prediction becomes the warning ystem for the investors due to unforeseen risk in the market. Hence, an accurate gold rice forecasting is required to foresee the business trends. This paper concentrates on orecasting the future gold prices from four commodities like historical data’s of gold rices, silver prices, Crude oil prices, Standard and Poor’s 500 stock index (S&amp;P500) ndex and foreign exchange rate. The period used for the study is from 1st January 000 to 31st April 2014. In this paper, a learning algorithm for single hidden layered eed forward neural networks called Extreme Learning Machine (ELM) is used which as good learning ability. Also, this study compares the five models namely Feed orward networks without feedback, Feed forward back propagation networks, Radial asis function, ELMAN networks and ELM learning model. The results prove that he ELM learning performs better than the other methods.</p>

Sign in / Sign up

Export Citation Format

Share Document