A Heterogeneous AdaBoost Ensemble Based Extreme Learning Machines for Imbalanced Data

2019 ◽  
Vol 13 (3) ◽  
pp. 19-35 ◽  
Author(s):  
Adnan Omer Abuassba ◽  
Dezheng Zhang ◽  
Xiong Luo
Keyword(s):  
Real World ◽  
Image Quality Assessment ◽  
Learning Algorithm ◽  
Imbalanced Data ◽  
Learning Performance ◽  
Feed Forward Neural Network ◽  
Unseen Data ◽  
Benchmark Datasets ◽  
Learning Machine ◽  
Hidden Layer

Extreme learning machine (ELM) is an effective learning algorithm for the single hidden layer feed-forward neural network (SLFN). It is diversified in the form of kernels or feature mapping functions, while achieving a good learning performance. It is agile in learning and often has good performance, including kernel ELM and Regularized ELM. Dealing with imbalanced data has been a long-term focus for the learning algorithms to achieve satisfactory analytical results. It is obvious that the unbalanced class distribution imposes very challenging obstacles to implement learning tasks in real-world applications, including online visual tracking and image quality assessment. This article addresses this issue through advanced diverse AdaBoost based ELM ensemble (AELME) for imbalanced binary and multiclass data classification. This article aims to improve classification accuracy of the imbalanced data. In the proposed method, the ensemble is developed while splitting the trained data into corresponding subsets. And different algorithms of enhanced ELM, including regularized ELM and kernel ELM, are used as base learners, so that an active learner is constructed from a group of relatively weak base learners. Furthermore, AELME is implemented by training a randomly selected ELM classifier on a subset, chosen by random re-sampling. Then, the labels of unseen data could be predicted using the weighting approach. AELME is validated through classification on real-world benchmark datasets.

scholarly journals Improving Classification Performance through an Advanced Ensemble Based Heterogeneous Extreme Learning Machines

2017 ◽  
Vol 2017 ◽  
pp. 1-11 ◽  
Author(s):  
Adnan O. M. Abuassba ◽  
Dezheng Zhang ◽  
Xiong Luo ◽  
Ahmad Shaheryar ◽  
Hazrat Ali
Keyword(s):  
Learning Algorithm ◽  
Majority Vote ◽  
Classification Performance ◽  
Training Data ◽  
Generalization Performance ◽  
Fast Learning ◽  
Unseen Data ◽  
Benchmark Datasets ◽  
Learning Machine ◽  
Hidden Layer

Extreme Learning Machine (ELM) is a fast-learning algorithm for a single-hidden layer feedforward neural network (SLFN). It often has good generalization performance. However, there are chances that it might overfit the training data due to having more hidden nodes than needed. To address the generalization performance, we use a heterogeneous ensemble approach. We propose an Advanced ELM Ensemble (AELME) for classification, which includes Regularized-ELM, L2-norm-optimized ELM (ELML2), and Kernel-ELM. The ensemble is constructed by training a randomly chosen ELM classifier on a subset of training data selected through random resampling. The proposed AELM-Ensemble is evolved by employing an objective function of increasing diversity and accuracy among the final ensemble. Finally, the class label of unseen data is predicted using majority vote approach. Splitting the training data into subsets and incorporation of heterogeneous ELM classifiers result in higher prediction accuracy, better generalization, and a lower number of base classifiers, as compared to other models (Adaboost, Bagging, Dynamic ELM ensemble, data splitting ELM ensemble, and ELM ensemble). The validity of AELME is confirmed through classification on several real-world benchmark datasets.

Comparison of the Extreme Learning Machine with the BP Neural Network for Short-Term Prediction of Wind Power

2012 ◽  
Vol 608-609 ◽  
pp. 564-568 ◽  
Author(s):  
Yi Hui Zhang ◽  
He Wang ◽  
Zhi Jian Hu ◽  
Meng Lin Zhang ◽  
Xiao Lu Gong ◽  
...  
Keyword(s):  
Neural Network ◽  
Wind Power ◽  
Extreme Learning Machine ◽  
Bp Neural Network ◽  
Learning Algorithm ◽  
Power Prediction ◽  
Feed Forward Neural Network ◽  
Wind Power Prediction ◽  
Learning Machine ◽  
Hidden Layer

Extreme learning machine (ELM) is a new and effective single-hidden layer feed forward neural network learning algorithm. Extreme learning machine only needs to set the number of hidden layer nodes of the network, and there is no need to adjust the neural network input weights and the hidden units bias, and it generates the only optimum solution, so it has the advantage of fast learning and good generalization ability. And the back propagation (BP) neural network is the most maturely applied. This paper has introduced the extreme learning machine into the wind power prediction. By comparing the wind power prediction method using the BP neural network. Study shows that the extreme learning machine has better prediction accuracy and shorter model training time.

scholarly journals Rank based Pseudoinverse Computation in Extreme Learning Machine for large Datasets

2019 ◽  
Vol 8 (10) ◽  
pp. 1341-1346
Keyword(s):  
Extreme Learning Machine ◽  
Learning Algorithm ◽  
Matrix Decomposition ◽  
Least Square ◽  
Feed Forward Neural Network ◽  
Worst Case ◽  
Training Time ◽  
Learning Machine ◽  
Hidden Layer ◽  
Hidden Nodes

Extreme Learning Machine (ELM) is an efficient and effective least-square-based learning algorithm for classification, regression problems based on single hidden layer feed-forward neural network (SLFN). It has been shown in the literature that it has faster convergence and good generalization ability for moderate datasets. But, there is great deal of challenge involved in computing the pseudoinverse when there are large numbers of hidden nodes or for large number of instances to train complex pattern recognition problems. To address this problem, a few approaches such as EM-ELM, DF-ELM have been proposed in the literature. In this paper, a new rank-based matrix decomposition of the hidden layer matrix is introduced to have the optimal training time and reduce the computational complexity for a large number of hidden nodes in the hidden layer. The results show that it has constant training time which is closer towards the minimal training time and very far from worst-case training time of the DF-ELM algorithm that has been shown efficient in the recent literature.

Study on Deep Structure of Extreme Learning Machine (DS-ELM) for Datasets with Noise

2014 ◽  
Vol 989-994 ◽  
pp. 3679-3682 ◽  
Author(s):  
Meng Meng Ma ◽  
Bo He
Keyword(s):  
Extreme Learning Machine ◽  
Simple Structure ◽  
Learning Algorithm ◽  
Deep Structure ◽  
Noisy Data ◽  
Machine Learning Algorithm ◽  
Feed Forward Neural Networks ◽  
Learning Machine ◽  
Hidden Layer ◽  
Auto Associative Neural Network

Extreme learning machine (ELM), a relatively novel machine learning algorithm for single hidden layer feed-forward neural networks (SLFNs), has been shown competitive performance in simple structure and superior training speed. To improve the effectiveness of ELM for dealing with noisy datasets, a deep structure of ELM, short for DS-ELM, is proposed in this paper. DS-ELM contains three level networks (actually contains three nets ): the first level network is trained by auto-associative neural network (AANN) aim to filter out noise as well as reduce dimension when necessary; the second level network is another AANN net aim to fix the input weights and bias of ELM; and the last level network is ELM. Experiments on four noisy datasets are carried out to examine the new proposed DS-ELM algorithm. And the results show that DS-ELM has higher performance than ELM when dealing with noisy data.

A Measure Optimized Cost-Sensitive Learning Framework for Imbalanced Data Classification

2014 ◽  
pp. 48-75 ◽  
Author(s):  
Peng Cao ◽  
Osmar Zaiane ◽  
Dazhe Zhao
Keyword(s):  
Real World ◽  
Class Imbalance ◽  
Imbalanced Data ◽  
Support Vector ◽  
Feature Subset ◽  
Cost Sensitive Learning ◽  
Intrinsic Parameters ◽  
Real World Problem ◽  
Benchmark Datasets ◽  
Feed Forward Neural Networks

Class imbalance is one of the challenging problems for machine-learning in many real-world applications. Many methods have been proposed to address and attempt to solve the problem, including sampling and cost-sensitive learning. The latter has attracted significant attention in recent years to solve the problem, but it is difficult to determine the precise misclassification costs in practice. There are also other factors that influence the performance of the classification including the input feature subset and the intrinsic parameters of the classifier. This chapter presents an effective wrapper framework incorporating the evaluation measure (AUC and G-mean) into the objective function of cost sensitive learning directly to improve the performance of classification by simultaneously optimizing the best pair of feature subset, intrinsic parameters, and misclassification cost parameter. The optimization is based on Particle Swarm Optimization (PSO). The authors use two different common methods, support vector machine and feed forward neural networks, to evaluate the proposed framework. Experimental results on various standard benchmark datasets with different ratios of imbalance and a real-world problem show that the proposed method is effective in comparison with commonly used sampling techniques.

scholarly journals Theory and Numerical Analysis of Extreme Learning Machine and Its Application for Different Degrees of Defect Recognition of Hoisting Wire Rope

2018 ◽  
Vol 2018 ◽  
pp. 1-13 ◽  
Author(s):  
Zhike Zhao ◽  
Xiaoguang Zhang
Keyword(s):  
Extreme Learning Machine ◽  
Performance Test ◽  
Research Problem ◽  
Wire Rope ◽  
Nondestructive Method ◽  
Experimental Results ◽  
Feed Forward Neural Network ◽  
Learning Machine ◽  
Hidden Layer ◽  
Hidden Nodes

An improved classification approach is proposed to solve the hot research problem of some complex multiclassification samples based on extreme learning machine (ELM). ELM was proposed based on the single-hidden layer feed-forward neural network (SLFNN). ELM is characterized by the easier parameter selection rules, the faster converge speed, the less human intervention, and so on. In order to further improve the classification precision of ELM, an improved generation method of the network structure of ELM is developed by dynamically adjusting the number of hidden nodes. The number change of the hidden nodes can serve as the computational updated step length of the ELM algorithm. In this paper, the improved algorithm can be called the variable step incremental extreme learning machine (VSI-ELM). In order to verify the effect of the hidden layer nodes on the performance of ELM, an open-source machine learning database (University of California, Irvine (UCI)) is provided by the performance test data sets. The regression and classification experiments are used to study the performance of the VSI-ELM model, respectively. The experimental results show that the VSI-ELM algorithm is valid. The classification of different degrees of broken wires is now still a problem in the nondestructive testing of hoisting wire rope. The magnetic flux leakage (MFL) method of wire rope is an efficient nondestructive method which plays an important role in safety evaluation. Identifying the proposed VSI-ELM model is effective and reliable for actually applying data, and it is used to identify the classification problem of different types of samples from MFL signals. The final experimental results show that the VSI-ELM algorithm is of faster classification speed and higher classification accuracy of different broken wires.

FUSION OF EXTREME LEARNING MACHINE WITH FUZZY INTEGRAL

2013 ◽  
Vol 21 (supp02) ◽  
pp. 23-34 ◽  
Author(s):  
JUNHAI ZHAI ◽  
HONGYU XU ◽  
YAN LI
Keyword(s):  
Extreme Learning Machine ◽  
Learning Algorithm ◽  
Fuzzy Integral ◽  
Fast Learning ◽  
Original Dataset ◽  
Learning Speed ◽  
Feed Forward Neural Networks ◽  
Three Stages ◽  
Learning Machine ◽  
Hidden Layer

Extreme learning machine (ELM) is an efficient and practical learning algorithm used for training single hidden layer feed-forward neural networks (SLFNs). ELM can provide good generalization performance at extremely fast learning speed. However, ELM suffers from instability and over-fitting, especially on relatively large datasets. Based on probabilistic SLFNs, an approach of fusion of extreme learning machine (F-ELM) with fuzzy integral is proposed in this paper. The proposed algorithm consists of three stages. Firstly, the bootstrap technique is employed to generate several subsets of original dataset. Secondly, probabilistic SLFNs are trained with ELM algorithm on each subset. Finally, the trained probabilistic SLFNs are fused with fuzzy integral. The experimental results show that the proposed approach can alleviate to some extent the problems mentioned above, and can increase the prediction accuracy.

scholarly journals BD-ELM: A Regularized Extreme Learning Machine Using Biased DropConnect and Biased Dropout

2020 ◽  
Vol 2020 ◽  
pp. 1-7
Author(s):  
Jie Lai ◽  
Xiaodan Wang ◽  
Rui Li ◽  
Yafei Song ◽  
Lei Lei
Keyword(s):  
Extreme Learning Machine ◽  
Regularization Method ◽  
Network Performance ◽  
Structural Complexity ◽  
Benchmark Datasets ◽  
Fixed Constant ◽  
The Difference ◽  
Learning Machine ◽  
Hidden Layer ◽  
Hidden Nodes

In order to prevent the overfitting and improve the generalization performance of Extreme Learning Machine (ELM), a new regularization method, Biased DropConnect, and a new regularized ELM using the Biased DropConnect and Biased Dropout (BD-ELM) are both proposed in this paper. Like the Biased Dropout to hidden nodes, the Biased DropConnect can utilize the difference of connection weights to keep more information of network after dropping. The regular Dropout and DropConnect set the connection weights and output of the hidden layer to 0 with a single fixed probability. But the Biased DropConnect and Biased Dropout divide the connection weights and hidden nodes into high and low groups by threshold, and set different groups to 0 with different probabilities. Connection weights with high value and hidden nodes with a high-activated value, which make more contribution to network performance, will be kept by a lower drop probability, while the weights and hidden nodes with a low value will be given a higher drop probability to keep the drop probability of the whole network to a fixed constant. Using Biased DropConnect and Biased Dropout regularization, in BD-ELM, the sparsity of parameters is enhanced and the structural complexity is reduced. Experiments on various benchmark datasets show that Biased DropConnect and Biased Dropout can effectively address the overfitting, and BD-ELM can provide higher classification accuracy than ELM, R-ELM, and Drop-ELM.

scholarly journals Convergence Analysis of An Improved Extreme Learning Machine Based on Gradient Descent Method

2016 ◽  
Vol 8 (1) ◽  
pp. 5-15
Author(s):  
Liu Yusong ◽  
Su Zhixun ◽  
Zhang Bingjie ◽  
Gong Xiaoling ◽  
Sang Zhaoyang
Keyword(s):  
Extreme Learning Machine ◽  
Learning Algorithm ◽  
Error Function ◽  
Gradient Descent Method ◽  
Testing Time ◽  
Training Procedure ◽  
Feed Forward Neural Network ◽  
Learning Machine ◽  
Theoretical Results ◽  
Hidden Nodes

Abstract Extreme learning machine (ELM) is an efficient algorithm, but it requires more hidden nodes than the BP algorithms to reach the matched performance. Recently, an efficient learning algorithm, the upper-layer-solution-unaware algorithm (USUA), is proposed for the single-hidden layer feed-forward neural network. It needs less number of hidden nodes and testing time than ELM. In this paper, we mainly give the theoretical analysis for USUA. Theoretical results show that the error function monotonously decreases in the training procedure, the gradient of the error function with respect to weights tends to zero (the weak convergence), and the weight sequence goes to a fixed point (the strong convergence) when the iterations approach positive infinity. An illustrated simulation has been implemented on the MNIST database of handwritten digits which effectively verifies the theoretical results..

scholarly journals Improved Extreme Learning Machine and Its Application in Image Quality Assessment

2014 ◽  
Vol 2014 ◽  
pp. 1-7 ◽  
Author(s):  
Li Mao ◽  
Lidong Zhang ◽  
Xingyang Liu ◽  
Chaofeng Li ◽  
Hong Yang
Keyword(s):  
Image Quality ◽  
Quality Assessment ◽  
Extreme Learning Machine ◽  
Image Quality Assessment ◽  
Risk Minimization ◽  
Minimization Principle ◽  
Empirical Risk ◽  
Weighted Extreme Learning Machine ◽  
Learning Machine ◽  
Hidden Layer

Extreme learning machine (ELM) is a new class of single-hidden layer feedforward neural network (SLFN), which is simple in theory and fast in implementation. Zong et al. propose a weighted extreme learning machine for learning data with imbalanced class distribution, which maintains the advantages from original ELM. However, the current reported ELM and its improved version are only based on the empirical risk minimization principle, which may suffer from overfitting. To solve the overfitting troubles, in this paper, we incorporate the structural risk minimization principle into the (weighted) ELM, and propose a modified (weighted) extreme learning machine (M-ELM and M-WELM). Experimental results show that our proposed M-WELM outperforms the current reported extreme learning machine algorithm in image quality assessment.

Sign in / Sign up

Export Citation Format

Share Document