SVM–ELM: Pruning of Extreme Learning Machine with Support Vector Machines for Regression

2016 ◽  
Vol 25 (4) ◽  
pp. 555-566 ◽  
Author(s):  
Saif F. Mahmood ◽  
Mohammad H. Marhaban ◽  
Fakhrul Z. Rokhani ◽  
Khairulmizam Samsudin ◽  
Olasimbo Ayodeji Arigbabu
Keyword(s):  
Extreme Learning Machine ◽  
Optimal Number ◽  
Training Data ◽  
Computational Time ◽  
Support Vector ◽  
Benchmark Datasets ◽  
Clustering And Classification ◽  
Learning Machine ◽  
Hidden Nodes ◽  
Selection Of

AbstractExtreme Learning Machine provides very competitive performance to other related classical predictive models for solving problems such as regression, clustering, and classification. An ELM possesses the advantage of faster computational time in both training and testing. However, one of the main challenges of an ELM is the selection of the optimal number of hidden nodes. This paper presents a new approach to node selection of an ELM based on a 1-norm support vector machine (SVM). In this method, the targets of SVM yi ∈{+1, –1} are derived using the mean or median of ELM training errors as a threshold for separating the training data, which are projected to SVM dimensions. We present an integrated architecture that exploits the sparseness in solution of SVM to prune out the inactive hidden nodes in ELM. Several experiments are conducted on real-world benchmark datasets, and the results attained attest to the efficiency of the proposed method.

scholarly journals Fault Diagnosis for Engine Based on Single-Stage Extreme Learning Machine

2016 ◽  
Vol 2016 ◽  
pp. 1-10 ◽  
Author(s):  
Fei Gao ◽  
Jiangang Lv
Keyword(s):  
Fault Diagnosis ◽  
Extreme Learning Machine ◽  
Feature Space ◽  
Single Stage ◽  
Training Data ◽  
Learning Machine ◽  
Hidden Layer ◽  
Signal Processing Methods ◽  
Hidden Nodes ◽  
Selection Of

Single-Stage Extreme Learning Machine (SS-ELM) is presented to dispose of the mechanical fault diagnosis in this paper. Based on it, the traditional mapping type of extreme learning machine (ELM) has been changed and the eigenvectors extracted from signal processing methods are directly regarded as outputs of the network’s hidden layer. Then the uncertainty that training data transformed from the input space to the ELM feature space with the ELM mapping and problem of the selection of the hidden nodes are avoided effectively. The experiment results of diesel engine fault diagnosis show good performance of the SS-ELM algorithm.

Constructive hidden nodes selection of extreme learning machine for regression

2010 ◽  
Vol 73 (16-18) ◽  
pp. 3191-3199 ◽  
Author(s):  
Yuan Lan ◽  
Yeng Chai Soh ◽  
Guang-Bin Huang
Keyword(s):  
Extreme Learning Machine ◽  
Learning Machine ◽  
Hidden Nodes ◽  
Selection Of

scholarly journals Development of a Kernel Extreme Learning Machine Model for Capacity Selection of Distributed Generation Considering the Characteristics of Electric Vehicles

2019 ◽  
Vol 9 (12) ◽  
pp. 2401
Author(s):  
Zhongdong Yin ◽  
Jingjing Tu ◽  
Yonghai Xu
Keyword(s):  
Extreme Learning Machine ◽  
Distributed Generation ◽  
Electric Vehicles ◽  
Distribution Network ◽  
Support Vector ◽  
Kernel Extreme Learning Machine ◽  
Learning Machine ◽  
The Stability ◽  
Capacity Selection ◽  
Selection Of

The large-scale access of distributed generation (DG) and the continuous increase in the demand of electric vehicle (EV) charging will result in fundamental changes in the planning and operating characteristics of the distribution network. Therefore, studying the capacity selection of the distributed generation, such as wind and photovoltaic (PV), and considering the charging characteristic of electric vehicles, is of great significance to the stability and economic operation of the distribution network. By using the network node voltage, the distributed generation output and the electric vehicles’ charging power as training data, we propose a capacity selection model based on the kernel extreme learning machine (KELM). The model accuracy is evaluated by using the root mean square error (RMSE). The stability of the network is evaluated by voltage stability evaluation index (Ivse). The IEEE33 node distributed system is used as simulation example, and gives results calculated by the kernel extreme learning machine that satisfy the minimum network loss and total investment cost. Finally, the results are compared with support vector machine (SVM), particle swarm optimization algorithm (PSO) and genetic algorithm (GA), to verify the feasibility and effectiveness of the proposed model and method.

scholarly journals BD-ELM: A Regularized Extreme Learning Machine Using Biased DropConnect and Biased Dropout

2020 ◽  
Vol 2020 ◽  
pp. 1-7
Author(s):  
Jie Lai ◽  
Xiaodan Wang ◽  
Rui Li ◽  
Yafei Song ◽  
Lei Lei
Keyword(s):  
Extreme Learning Machine ◽  
Regularization Method ◽  
Network Performance ◽  
Structural Complexity ◽  
Benchmark Datasets ◽  
Fixed Constant ◽  
The Difference ◽  
Learning Machine ◽  
Hidden Layer ◽  
Hidden Nodes

In order to prevent the overfitting and improve the generalization performance of Extreme Learning Machine (ELM), a new regularization method, Biased DropConnect, and a new regularized ELM using the Biased DropConnect and Biased Dropout (BD-ELM) are both proposed in this paper. Like the Biased Dropout to hidden nodes, the Biased DropConnect can utilize the difference of connection weights to keep more information of network after dropping. The regular Dropout and DropConnect set the connection weights and output of the hidden layer to 0 with a single fixed probability. But the Biased DropConnect and Biased Dropout divide the connection weights and hidden nodes into high and low groups by threshold, and set different groups to 0 with different probabilities. Connection weights with high value and hidden nodes with a high-activated value, which make more contribution to network performance, will be kept by a lower drop probability, while the weights and hidden nodes with a low value will be given a higher drop probability to keep the drop probability of the whole network to a fixed constant. Using Biased DropConnect and Biased Dropout regularization, in BD-ELM, the sparsity of parameters is enhanced and the structural complexity is reduced. Experiments on various benchmark datasets show that Biased DropConnect and Biased Dropout can effectively address the overfitting, and BD-ELM can provide higher classification accuracy than ELM, R-ELM, and Drop-ELM.

scholarly journals An Extreme Learning Machine Based on the Mixed Kernel Function of Triangular Kernel and Generalized Hermite Dirichlet Kernel

2016 ◽  
Vol 2016 ◽  
pp. 1-11
Author(s):  
Senyue Zhang ◽  
Wenan Tan
Keyword(s):  
Extreme Learning Machine ◽  
Kernel Function ◽  
Binary Classification ◽  
Dirichlet Kernel ◽  
Computational Time ◽  
Support Vector ◽  
Structure Information ◽  
Kernel Parameter ◽  
Hermitian Kernel ◽  
Learning Machine

According to the characteristics that the kernel function of extreme learning machine (ELM) and its performance have a strong correlation, a novel extreme learning machine based on a generalized triangle Hermitian kernel function was proposed in this paper. First, the generalized triangle Hermitian kernel function was constructed by using the product of triangular kernel and generalized Hermite Dirichlet kernel, and the proposed kernel function was proved as a valid kernel function of extreme learning machine. Then, the learning methodology of the extreme learning machine based on the proposed kernel function was presented. The biggest advantage of the proposed kernel is its kernel parameter values only chosen in the natural numbers, which thus can greatly shorten the computational time of parameter optimization and retain more of its sample data structure information. Experiments were performed on a number of binary classification, multiclassification, and regression datasets from the UCI benchmark repository. The experiment results demonstrated that the robustness and generalization performance of the proposed method are outperformed compared to other extreme learning machines with different kernels. Furthermore, the learning speed of proposed method is faster than support vector machine (SVM) methods.

scholarly journals Deep Convolutional Neural Network Based Extreme Learning Machine Image Classification

2021 ◽  
pp. 30-38
Author(s):  
G. D. Praveenkumar ◽  
Dr. R. Nagaraj
Keyword(s):  
Neural Network ◽  
Convolutional Neural Network ◽  
Extreme Learning Machine ◽  
Deep Convolutional Neural Network ◽  
Computational Time ◽  
Machine Model ◽  
Proposed Model ◽  
Benchmark Datasets ◽  
Learning Machine ◽  
Elm Classifier

In this paper, we introduce a new deep convolutional neural network based extreme learning machine model for the classification task in order to improve the network's performance. The proposed model has two stages: first, the input images are fed into a convolutional neural network layer to extract deep-learned attributes, and then the input is classified using an ELM classifier. The proposed model achieves good recognition accuracy while reducing computational time on both the MNIST and CIFAR-10 benchmark datasets.

Intrusion Detection Using Deep Belief Network and Extreme Learning Machine

2015 ◽  
Vol 3 (2) ◽  
pp. 35-56 ◽  
Author(s):  
Zahangir Alom ◽  
Venkata Ramesh Bontupalli ◽  
Tarek M. Taha
Keyword(s):  
Intrusion Detection ◽  
Extreme Learning Machine ◽  
Training Data ◽  
Support Vector ◽  
Security Threats ◽  
Deep Belief Networks ◽  
Detection Systems ◽  
Testing Accuracy ◽  
Learning Machine ◽  
The Impact

Security threats for computer networks have increased dramatically over the last decade, becoming bolder and more brazen. There is a strong need for effective Intrusion Detection Systems (IDS) that are designed to interpret intrusion attempts in incoming network traffic intelligently. In this paper, the authors explored the capabilities of Deep Belief Networks (DBN) – one of the most influential deep learning approach – in performing intrusion detection after training with the NSL-KDD dataset. Additionally, they examined the impact of using Extreme Learning Machine (ELM) and Regularized ELM on the same dataset to evaluate the performance against DBN and Support Vector Machine (SVM) approaches. The trained system identifies any type of unknown attack in the dataset examined. In addition to detecting attacks, the proposed system also classifies them into five groups. The implementation with DBN and SVM give a testing accuracy of about 97.5% and 88.33% respectively with 40% of training data selected from the NSL-KDD dataset. On the other hand, the experimental results show around 98.20% and 98.26% testing accuracy respectively for ELM and RELM after reducing the data dimensions from 41 to 9 essential features with 40% training data. ELM and RELM perform better in terms of testing accuracy upon comparison with DBN and SVM.

DIESEL ENGINE MODELLING USING EXTREME LEARNING MACHINE UNDER SCARCE AND EXPONENTIAL DATA SETS

2013 ◽  
Vol 21 (supp02) ◽  
pp. 87-98 ◽  
Author(s):  
PAK KIN WONG ◽  
CHI MAN VONG ◽  
CHUN SHUN CHEUNG ◽  
KA IN WONG
Keyword(s):  
Support Vector Machine ◽  
Diesel Engine ◽  
Extreme Learning Machine ◽  
Prediction Accuracy ◽  
Logarithmic Transformation ◽  
Training Data ◽  
Support Vector ◽  
Data Sets ◽  
Sample Data ◽  
Learning Machine

To predict the performance of a diesel engine, current practice relies on the use of black-box identification where numerous experiments must be carried out in order to obtain numerical values for model training. Although many diesel engine models based on artificial neural networks (ANNs) have already been developed, they have many drawbacks such as local minima, user burden on selection of optimal network structure, large training data size and poor generalization performance, making themselves difficult to be put into practice. This paper proposes to use extreme learning machine (ELM), which can overcome most of the aforementioned drawbacks, to model the emission characteristics and the brake-specific fuel consumption of the diesel engine under scarce and exponential sample data sets. The resulting ELM model is compared with those developed using popular ANNs such as radial basis function neural network (RBFNN) and advanced techniques such as support vector machine (SVM) and its variants, namely least squares support vector machine (LS-SVM) and relevance vector machine (RVM). Furthermore, some emission outputs of diesel engines suffer from the problem of exponentiality (i.e., the output y grows up exponentially along input x) that will deteriorate the prediction accuracy. A logarithmic transformation is therefore applied to preprocess and post-process the sample data sets in order to improve the prediction accuracy of the model. Evaluation results show that ELM with the logarithmic transformation is better than SVM, LS-SVM, RVM and RBFNN with/without the logarithmic transformation, regardless the model accuracy and training time.

scholarly journals Data-Driven Approach for Rainfall-Runoff Modelling Using Equilibrium Optimizer Coupled Extreme Learning Machine and Deep Neural Network

2021 ◽  
Vol 11 (13) ◽  
pp. 6238
Author(s):  
Bishwajit Roy ◽  
Maheshwari Prasad Singh ◽  
Mosbeh R. Kaloop ◽  
Deepak Kumar ◽  
Jong-Wan Hu ◽  
...  
Keyword(s):  
Neural Network ◽  
Extreme Learning Machine ◽  
Deep Neural Network ◽  
Optimal Number ◽  
Gradient Boosting ◽  
Support Vector ◽  
Discrete Wavelet ◽  
Rainfall Runoff ◽  
Hydrological Variable ◽  
Learning Machine

Rainfall-runoff (R-R) modelling is used to study the runoff generation of a catchment. The quantity or rate of change measure of the hydrological variable, called runoff, is important for environmental scientists to accomplish water-related planning and design. This paper proposes (i) an integrated model namely EO-ELM (an integration of equilibrium optimizer (EO) and extreme learning machine (ELM)) and (ii) a deep neural network (DNN) for one day-ahead R-R modelling. The proposed R-R models are validated at two different benchmark stations of the catchments, namely river Teifi at Glanteifi and river Fal at Tregony in the UK. Firstly, a partial autocorrelation function (PACF) is used for optimal number of lag inputs to deploy the proposed models. Six other well-known machine learning models, called ELM, kernel ELM (KELM), and particle swarm optimization-based ELM (PSO-ELM), support vector regression (SVR), artificial neural network (ANN) and gradient boosting machine (GBM) are utilized to validate the two proposed models in terms of prediction efficiency. Furthermore, to increase the performance of the proposed models, paper utilizes a discrete wavelet-based data pre-processing technique is applied in rainfall and runoff data. The performance of wavelet-based EO-ELM and DNN are compared with wavelet-based ELM (WELM), KELM (WKELM), PSO-ELM (WPSO-ELM), SVR (WSVR), ANN (WANN) and GBM (WGBM). An uncertainty analysis and two-tailed t-test are carried out to ensure the trustworthiness and efficacy of the proposed models. The experimental results for two different time series datasets show that the EO-ELM performs better in an optimal number of lags than the others. In the case of wavelet-based daily R-R modelling, proposed models performed better and showed robustness compared to other models used. Therefore, this paper shows the efficient applicability of EO-ELM and DNN in R-R modelling that may be used in the hydrological modelling field.

Sign in / Sign up

Export Citation Format

Share Document