scholarly journals A Fast Algorithm for Multi-Class Learning from Label Proportions

Electronics ◽  
2019 ◽  
Vol 8 (6) ◽  
pp. 609 ◽  
Author(s):  
Fan Zhang ◽  
Jiabin Liu ◽  
Bo Wang ◽  
Zhiquan Qi ◽  
Yong Shi
Keyword(s):  
Extreme Learning Machine ◽  
Fast Algorithm ◽  
Efficient Solutions ◽  
Training Data ◽  
Fast Learning ◽  
Learning Speed ◽  
Output Matrix ◽  
Learning Machine ◽  
Learning From Label Proportions ◽  
Hidden Layer

Learning from label proportions (LLP) is a new kind of learning problem which has attracted wide interest in machine learning. Different from the well-known supervised learning, the training data of LLP is in the form of bags and only the proportion of each class in each bag is available. Actually, many modern applications can be successfully abstracted to this problem such as modeling voting behaviors and spam filtering. However, time-consuming training is still a challenge for LLP, which becomes a bottleneck especially when addressing large bags and bag sizes. In this paper, we propose a fast algorithm called multi-class learning from label proportions by extreme learning machine (LLP-ELM), which takes advantage of an extreme learning machine with fast learning speed to solve multi-class learning from label proportions. Firstly, we reshape the hidden layer output matrix and the training data target matrix of an extreme learning machine to adapt to the proportion information instead of the real labels. Secondly, a robust loss function with a regularization term is formulated and two efficient solutions are provided to different cases. Finally, various experiments demonstrate the significant speed-up of the proposed model with better accuracies on different datasets compared with several state-of-the-art methods.

FUSION OF EXTREME LEARNING MACHINE WITH FUZZY INTEGRAL

2013 ◽  
Vol 21 (supp02) ◽  
pp. 23-34 ◽  
Author(s):  
JUNHAI ZHAI ◽  
HONGYU XU ◽  
YAN LI
Keyword(s):  
Extreme Learning Machine ◽  
Learning Algorithm ◽  
Fuzzy Integral ◽  
Fast Learning ◽  
Original Dataset ◽  
Learning Speed ◽  
Feed Forward Neural Networks ◽  
Three Stages ◽  
Learning Machine ◽  
Hidden Layer

Extreme learning machine (ELM) is an efficient and practical learning algorithm used for training single hidden layer feed-forward neural networks (SLFNs). ELM can provide good generalization performance at extremely fast learning speed. However, ELM suffers from instability and over-fitting, especially on relatively large datasets. Based on probabilistic SLFNs, an approach of fusion of extreme learning machine (F-ELM) with fuzzy integral is proposed in this paper. The proposed algorithm consists of three stages. Firstly, the bootstrap technique is employed to generate several subsets of original dataset. Secondly, probabilistic SLFNs are trained with ELM algorithm on each subset. Finally, the trained probabilistic SLFNs are fused with fuzzy integral. The experimental results show that the proposed approach can alleviate to some extent the problems mentioned above, and can increase the prediction accuracy.

scholarly journals An Improved Extreme Learning Machine Based on Full Rank Cholesky Factorization

2018 ◽  
Vol 246 ◽  
pp. 03018
Author(s):  
Zuozhi Liu ◽  
JinJian Wu ◽  
Jianpeng Wang
Keyword(s):  
Extreme Learning Machine ◽  
Learning Algorithm ◽  
Computational Cost ◽  
Full Rank ◽  
Cholesky Factorization ◽  
Fast Learning ◽  
New Novel ◽  
Learning Speed ◽  
Learning Machine ◽  
Hidden Layer

Extreme learning machine (ELM) is a new novel learning algorithm for generalized single-hidden layer feedforward networks (SLFNs). Although it shows fast learning speed in many areas, there is still room for improvement in computational cost. To address this issue, this paper proposes an improved ELM (FRCFELM) which employs the full rank Cholesky factorization to compute output weights instead of traditional SVD. In addition, this paper proves in theory that the proposed FRCF-ELM has lower computational complexity. Experimental results over some benchmark applications indicate that the proposed FRCF-ELM learns faster than original ELM algorithm while preserving good generalization performance.

HONNs with Extreme Learning Machine to Handle Incomplete Datasets

2013 ◽  
pp. 276-292
Author(s):  
Shuxiang Xu
Keyword(s):  
Neural Networks ◽  
Missing Data ◽  
Extreme Learning Machine ◽  
Real Life ◽  
Bankruptcy Prediction ◽  
Share Prices ◽  
Fast Learning ◽  
Business Cases ◽  
Learning Machine ◽  
Hidden Layer

An Extreme Learning Machine (ELM) randomly chooses hidden neurons and analytically determines the output weights (Huang, et al., 2005, 2006, 2008). With the ELM algorithm, only the connection weights between hidden layer and output layer are adjusted. The ELM algorithm tends to generalize better at a very fast learning speed: it can learn thousands of times faster than conventionally popular learning algorithms (Huang, et al., 2006). Artificial Neural Networks (ANNs) have been widely used as powerful information processing models and adopted in applications such as bankruptcy prediction, predicting costs, forecasting revenue, forecasting share prices and exchange rates, processing documents, and many more. Higher Order Neural Networks (HONNs) are ANNs in which the net input to a computational neuron is a weighted sum of products of its inputs. Real life data are not usually perfect. They contain wrong, incomplete, or vague data. Hence, it is usual to find missing data in many information sources used. Missing data is a common problem in statistical analysis (Little & Rubin, 1987). This chapter uses the Extreme Learning Machine (ELM) algorithm for HONN models and applies it in several significant business cases, which involve missing datasets. The experimental results demonstrate that HONN models with the ELM algorithm offer significant advantages over standard HONN models, such as faster training, as well as improved generalization abilities.

scholarly journals A Robust AdaBoost.RT Based Ensemble Extreme Learning Machine

2015 ◽  
Vol 2015 ◽  
pp. 1-12 ◽  
Author(s):  
Pengbo Zhang ◽  
Zhixin Yang
Keyword(s):  
Extreme Learning Machine ◽  
Learning Algorithm ◽  
Approximation Accuracy ◽  
Weak Learner ◽  
Fast Learning ◽  
Learning Speed ◽  
Regression Problems ◽  
Learning Machine ◽  
The Stability ◽  
Stability And Accuracy

Extreme learning machine (ELM) has been well recognized as an effective learning algorithm with extremely fast learning speed and high generalization performance. However, to deal with the regression applications involving big data, the stability and accuracy of ELM shall be further enhanced. In this paper, a new hybrid machine learning method called robust AdaBoost.RT based ensemble ELM (RAE-ELM) for regression problems is proposed, which combined ELM with the novel robust AdaBoost.RT algorithm to achieve better approximation accuracy than using only single ELM network. The robust threshold for each weak learner will be adaptive according to the weak learner’s performance on the corresponding problem dataset. Therefore, RAE-ELM could output the final hypotheses in optimally weighted ensemble of weak learners. On the other hand, ELM is a quick learner with high regression performance, which makes it a good candidate of “weak” learners. We prove that the empirical error of the RAE-ELM is within a significantly superior bound. The experimental verification has shown that the proposed RAE-ELM outperforms other state-of-the-art algorithms on many real-world regression problems.

scholarly journals One-Class Classification with Extreme Learning Machine

2015 ◽  
Vol 2015 ◽  
pp. 1-11 ◽  
Author(s):  
Qian Leng ◽  
Honggang Qi ◽  
Jun Miao ◽  
Wentao Zhu ◽  
Guiping Su
Keyword(s):  
Neural Network ◽  
Extreme Learning Machine ◽  
Classification Problem ◽  
Traditional Learning ◽  
Learning Speed ◽  
One Class Classifier ◽  
Learning Machine ◽  
Hidden Layer ◽  
One Class Classification ◽  
Network Approaches

One-class classification problem has been investigated thoroughly for past decades. Among one of the most effective neural network approaches for one-class classification, autoencoder has been successfully applied for many applications. However, this classifier relies on traditional learning algorithms such as backpropagation to train the network, which is quite time-consuming. To tackle the slow learning speed in autoencoder neural network, we propose a simple and efficient one-class classifier based on extreme learning machine (ELM). The essence of ELM is that the hidden layer need not be tuned and the output weights can be analytically determined, which leads to much faster learning speed. The experimental evaluation conducted on several real-world benchmarks shows that the ELM based one-class classifier can learn hundreds of times faster than autoencoder and it is competitive over a variety of one-class classification methods.

Multiple-Instance Learning via an RBF Kernel-Based Extreme Learning Machine

2017 ◽  
Vol 26 (1) ◽  
pp. 185-195 ◽  
Author(s):  
Jie Wang ◽  
Liangjian Cai ◽  
Xin Zhao
Keyword(s):  
Extreme Learning Machine ◽  
State Of The Art ◽  
Multiple Instance Learning ◽  
Training Data ◽  
Data Sets ◽  
Rbf Kernel ◽  
Kernel Extreme Learning Machine ◽  
Learning Machine ◽  
Hidden Layer ◽  
Instance Space

AbstractAs we are usually confronted with a large instance space for real-word data sets, it is significant to develop a useful and efficient multiple-instance learning (MIL) algorithm. MIL, where training data are prepared in the form of labeled bags rather than labeled instances, is a variant of supervised learning. This paper presents a novel MIL algorithm for an extreme learning machine called MI-ELM. A radial basis kernel extreme learning machine is adapted to approach the MIL problem using Hausdorff distance to measure the distance between the bags. The clusters in the hidden layer are composed of bags that are randomly generated. Because we do not need to tune the parameters for the hidden layer, MI-ELM can learn very fast. The experimental results on classifications and multiple-instance regression data sets demonstrate that the MI-ELM is useful and efficient as compared to the state-of-the-art algorithms.

OMP-ELM: Orthogonal Matching Pursuit-Based Extreme Learning Machine for Regression

2015 ◽  
Vol 24 (1) ◽  
pp. 135-143 ◽  
Author(s):  
Omer F. Alcin ◽  
Abdulkadir Sengur ◽  
Jiang Qian ◽  
Melih C. Ince
Keyword(s):  
Extreme Learning Machine ◽  
Matching Pursuit ◽  
Computational Cost ◽  
Machine Intelligence ◽  
Orthogonal Matching Pursuit ◽  
Training Data ◽  
Singularity Problem ◽  
Data Set ◽  
Learning Machine ◽  
Hidden Layer

AbstractExtreme learning machine (ELM) is a recent scheme for single hidden layer feed forward networks (SLFNs). It has attracted much interest in the machine intelligence and pattern recognition fields with numerous real-world applications. The ELM structure has several advantages, such as its adaptability to various problems with a rapid learning rate and low computational cost. However, it has shortcomings in the following aspects. First, it suffers from the irrelevant variables in the input data set. Second, choosing the optimal number of neurons in the hidden layer is not well defined. In case the hidden nodes are greater than the training data, the ELM may encounter the singularity problem, and its solution may become unstable. To overcome these limitations, several methods have been proposed within the regularization framework. In this article, we considered a greedy method for sparse approximation of the output weight vector of the ELM network. More specifically, the orthogonal matching pursuit (OMP) algorithm is embedded to the ELM. This new technique is named OMP-ELM. OMP-ELM has several advantages over regularized ELM methods, such as lower complexity and immunity to the singularity problem. Experimental works on nine commonly used regression problems indicate that the investigated OMP-ELM method confirms these advantages. Moreover, OMP-ELM is compared with the ELM method, the regularized ELM scheme, and artificial neural networks.

scholarly journals A Novel Improved ELM Algorithm for a Real Industrial Application

2014 ◽  
Vol 2014 ◽  
pp. 1-7 ◽  
Author(s):  
Hai-Gang Zhang ◽  
Sen Zhang ◽  
Yi-Xin Yin
Keyword(s):  
Neural Network ◽  
Feedforward Neural Networks ◽  
Generalization Performance ◽  
Stator Current ◽  
Fast Learning ◽  
Learning Speed ◽  
Network Methods ◽  
Learning Machine ◽  
Hidden Layer ◽  
Current Monitoring

It is well known that the feedforward neural networks meet numbers of difficulties in the applications because of its slow learning speed. The extreme learning machine (ELM) is a new single hidden layer feedforward neural network method aiming at improving the training speed. Nowadays ELM algorithm has received wide application with its good generalization performance under fast learning speed. However, there are still several problems needed to be solved in ELM. In this paper, a new improved ELM algorithm named R-ELM is proposed to handle the multicollinear problem appearing in calculation of the ELM algorithm. The proposed algorithm is employed in bearing fault detection using stator current monitoring. Simulative results show that R-ELM algorithm has better stability and generalization performance compared with the original ELM and the other neural network methods.

scholarly journals Fault Diagnosis for Engine Based on Single-Stage Extreme Learning Machine

2016 ◽  
Vol 2016 ◽  
pp. 1-10 ◽  
Author(s):  
Fei Gao ◽  
Jiangang Lv
Keyword(s):  
Fault Diagnosis ◽  
Extreme Learning Machine ◽  
Feature Space ◽  
Single Stage ◽  
Training Data ◽  
Learning Machine ◽  
Hidden Layer ◽  
Signal Processing Methods ◽  
Hidden Nodes ◽  
Selection Of

Single-Stage Extreme Learning Machine (SS-ELM) is presented to dispose of the mechanical fault diagnosis in this paper. Based on it, the traditional mapping type of extreme learning machine (ELM) has been changed and the eigenvectors extracted from signal processing methods are directly regarded as outputs of the network’s hidden layer. Then the uncertainty that training data transformed from the input space to the ELM feature space with the ELM mapping and problem of the selection of the hidden nodes are avoided effectively. The experiment results of diesel engine fault diagnosis show good performance of the SS-ELM algorithm.

A Combined Algorithm Based on ELM-RBF and Genetic Algorithm

2014 ◽  
Vol 1049-1050 ◽  
pp. 1292-1296
Author(s):  
Qing Feng Xia
Keyword(s):  
Optimization Problem ◽  
Rbf Neural Network ◽  
Local Minima ◽  
Rbf Network ◽  
Fast Learning ◽  
Learning Speed ◽  
Solve Optimization Problem ◽  
Learning Machine ◽  
Hidden Layer ◽  
Combined Algorithm

Extreme Learning Machine-Radial Basis Function (ELM-RBF) not only inherit RBF’s merit of not suffering from local minima, but also ELM’s merit of fast learning speed, Nevertheless, it is still a research hot area of how to improve the generalization ability of ELM-RBF network. Genetic Algorithms (GA) to solve optimization problem has its unique advantage. Considered on these, the paper adopted GA to optimize ELM-RBF neural network hidden layer neurons center and biases value. Experiments data results indicated that our proposed combined algorithm has better generalization performance than classical ELM-RBF, it achieved the basic anticipated task of design.

Sign in / Sign up

Export Citation Format

Share Document