scholarly journals Incremental Multiple Hidden Layers Regularized Extreme Learning Machine Based on Forced Positive-Definite Cholesky Factorization

2019 ◽  
Vol 2019 ◽  
pp. 1-15
Author(s):  
Jingyi Liu ◽  
Ba Tuan Le
Keyword(s):  
Prediction Model ◽  
Extreme Learning Machine ◽  
Network Structure ◽  
Prediction Accuracy ◽  
Computational Cost ◽  
Positive Definite ◽  
Cholesky Factorization ◽  
Optimal Network ◽  
Learning Machine ◽  
Hidden Layer

The theory and implementation of extreme learning machine (ELM) prove that it is a simple, efficient, and accurate machine learning method. Compared with other single hidden layer feedforward neural network algorithms, ELM is characterized by simpler parameter selection rules, faster convergence speed, and less human intervention. The multiple hidden layer regularized extreme learning machine (MRELM) inherits these advantages of ELM and has higher prediction accuracy. In the MRELM model, the number of hidden layers is randomly initiated and fixed, and there is no iterative tuning process. However, the optimal number of hidden layers is the key factor to determine the generalization ability of MRELM. Given this situation, it is obviously unreasonable to determine this number by trial and random initialization. In this paper, an incremental MRELM training algorithm (FC-IMRELM) based on forced positive-definite Cholesky factorization is put forward to solve the network structure design problem of MRELM. First, an MRELM-based prediction model with one hidden layer is constructed, and then a new hidden layer is added to the prediction model in each training step until the generalization performance of the prediction model reaches its peak value. Thus, the optimal network structure of the prediction model is determined. In the training procedure, forced positive-definite Cholesky factorization is used to calculate the output weights of MRELM, which avoids the calculation of the inverse matrix and Moore-Penrose generalized inverse of matrix involved in the training process of hidden layer parameters. Therefore, FC-IMRELM prediction model can effectively reduce the computational cost brought by the process of increasing the number of hidden layers. Experiments on classification and regression problems indicate that the algorithm can be effectively used to determine the optimal network structure of MRELM, and the prediction model training by the algorithm has excellent performance in prediction accuracy and computational cost.

scholarly journals An Improved Extreme Learning Machine Based on Full Rank Cholesky Factorization

2018 ◽  
Vol 246 ◽  
pp. 03018
Author(s):  
Zuozhi Liu ◽  
JinJian Wu ◽  
Jianpeng Wang
Keyword(s):  
Extreme Learning Machine ◽  
Learning Algorithm ◽  
Computational Cost ◽  
Full Rank ◽  
Cholesky Factorization ◽  
Fast Learning ◽  
New Novel ◽  
Learning Speed ◽  
Learning Machine ◽  
Hidden Layer

Extreme learning machine (ELM) is a new novel learning algorithm for generalized single-hidden layer feedforward networks (SLFNs). Although it shows fast learning speed in many areas, there is still room for improvement in computational cost. To address this issue, this paper proposes an improved ELM (FRCFELM) which employs the full rank Cholesky factorization to compute output weights instead of traditional SVD. In addition, this paper proves in theory that the proposed FRCF-ELM has lower computational complexity. Experimental results over some benchmark applications indicate that the proposed FRCF-ELM learns faster than original ELM algorithm while preserving good generalization performance.

OMP-ELM: Orthogonal Matching Pursuit-Based Extreme Learning Machine for Regression

2015 ◽  
Vol 24 (1) ◽  
pp. 135-143 ◽  
Author(s):  
Omer F. Alcin ◽  
Abdulkadir Sengur ◽  
Jiang Qian ◽  
Melih C. Ince
Keyword(s):  
Extreme Learning Machine ◽  
Matching Pursuit ◽  
Computational Cost ◽  
Machine Intelligence ◽  
Orthogonal Matching Pursuit ◽  
Training Data ◽  
Singularity Problem ◽  
Data Set ◽  
Learning Machine ◽  
Hidden Layer

AbstractExtreme learning machine (ELM) is a recent scheme for single hidden layer feed forward networks (SLFNs). It has attracted much interest in the machine intelligence and pattern recognition fields with numerous real-world applications. The ELM structure has several advantages, such as its adaptability to various problems with a rapid learning rate and low computational cost. However, it has shortcomings in the following aspects. First, it suffers from the irrelevant variables in the input data set. Second, choosing the optimal number of neurons in the hidden layer is not well defined. In case the hidden nodes are greater than the training data, the ELM may encounter the singularity problem, and its solution may become unstable. To overcome these limitations, several methods have been proposed within the regularization framework. In this article, we considered a greedy method for sparse approximation of the output weight vector of the ELM network. More specifically, the orthogonal matching pursuit (OMP) algorithm is embedded to the ELM. This new technique is named OMP-ELM. OMP-ELM has several advantages over regularized ELM methods, such as lower complexity and immunity to the singularity problem. Experimental works on nine commonly used regression problems indicate that the investigated OMP-ELM method confirms these advantages. Moreover, OMP-ELM is compared with the ELM method, the regularized ELM scheme, and artificial neural networks.

scholarly journals Aero Engine Fault Diagnosis Using an Optimized Extreme Learning Machine

2016 ◽  
Vol 2016 ◽  
pp. 1-10 ◽  
Author(s):  
Xinyi Yang ◽  
Shan Pang ◽  
Wei Shen ◽  
Xuesen Lin ◽  
Keyi Jiang ◽  
...  
Keyword(s):  
Gas Turbine ◽  
Extreme Learning Machine ◽  
Validation Data ◽  
Data Set ◽  
Swarm Optimization ◽  
Turbine Engine ◽  
Optimal Network ◽  
Aero Engine ◽  
Learning Machine ◽  
Hidden Layer

A new extreme learning machine optimized by quantum-behaved particle swarm optimization (QPSO) is developed in this paper. It uses QPSO to select optimal network parameters including the number of hidden layer neurons according to both the root mean square error on validation data set and the norm of output weights. The proposed Q-ELM was applied to real-world classification applications and a gas turbine fan engine diagnostic problem and was compared with two other optimized ELM methods and original ELM, SVM, and BP method. Results show that the proposed Q-ELM is a more reliable and suitable method than conventional neural network and other ELM methods for the defect diagnosis of the gas turbine engine.

A Real-Time Visual Tracking Approach Using Sparse Autoencoder and Extreme Learning Machine

2015 ◽  
Vol 03 (04) ◽  
pp. 267-275
Author(s):  
Liang Dai ◽  
Yuesheng Zhu ◽  
Guibo Luo ◽  
Chao He ◽  
Hanchi Lin
Keyword(s):  
Real Time ◽  
Extreme Learning Machine ◽  
Visual Tracking ◽  
Computational Cost ◽  
Real Time Tracking ◽  
Sparse Autoencoder ◽  
Selection For ◽  
Learning Machine ◽  
Hidden Layer ◽  
Online Process

Visual tracking algorithm based on deep learning is one of the state-of-the-art tracking approaches. However, its computational cost is high. To reduce the computational burden, in this paper, A real-time tracking approach is proposed by using three modules: a single hidden layer neural network based on sparse autoencoder, a feature selection for simplifying the network and an online process based on extreme learning machine. Our experimental results have demonstrated that the proposed algorithm has good performance of robust and real-time.

Ultra-short-term wind speed prediction based on variational mode decomposition and optimized extreme learning machine

2021 ◽  
pp. 0309524X2110385
Author(s):  
Lian Lian ◽  
Kan He
Keyword(s):  
Wind Speed ◽  
Prediction Model ◽  
Extreme Learning Machine ◽  
Prediction Accuracy ◽  
Good Prediction ◽  
Variational Mode Decomposition ◽  
Short Term ◽  
Mode Decomposition ◽  
Whale Optimization ◽  
Learning Machine

The main purpose of this paper is to improve the prediction accuracy of ultra-short-term wind speed. It is difficult to predict the ultra-short-term wind speed because of its unstable, non-stationary and non-linear. Aiming at the unstable and non-stationary characteristics of ultra-short-term wind speed, the variational mode decomposition algorithm is introduced to decompose the ultra-short-term wind speed data, and a series of stable and stationary components with different frequencies are obtained. The extreme learning machine with good prediction performance and real-time performance is selected as the prediction model of decomposed components. In order to solve the problem of random setting of input weights and bias of extreme learning machine, whale optimization algorithm is used to optimize extreme learning machine to improve the regression performance. The performance of the developed prediciton model is verified by real ultra-short-term wind speed sample data. Five prediction models are selected as the comparison model. Through the comparison between the predicted value and the actual value, the prediction error and its histogram distribution, eight performance indicators, and Pearson’s test correlation coefficient, the results show that the proposed prediction model has high prediction accuracy.

scholarly journals Cholesky Factorization Based Online Sequential Multiple Kernel Extreme Learning Machine Algorithm for a Cement Clinker Free Lime Content Prediction Model

Processes ◽  
2021 ◽  
Vol 9 (9) ◽  
pp. 1540
Author(s):  
Pengcheng Zhao ◽  
Ying Chen ◽  
Zhibiao Zhao
Keyword(s):  
Prediction Model ◽  
Real Time ◽  
Extreme Learning Machine ◽  
Cholesky Factorization ◽  
Cement Clinker ◽  
Free Lime ◽  
Multiple Kernel ◽  
Kernel Extreme Learning Machine ◽  
Online Prediction ◽  
Learning Machine

Aiming at the difficulty in real-time measuring and the long offline measurement cycle for the content of cement clinker free lime (fCaO), it is very important to build an online prediction model for fCaO content. In this work, on the basis of Cholesky factorization, the online sequential multiple kernel extreme learning machine algorithm (COS-MKELM) is proposed. The LDLT form Cholesky factorization of the matrix is introduced to avoid the large operation amount of inverse matrix calculation. In addition, the stored initial information is utilized to realize online model identification. Then, three regression datasets are used to test the performance of the COS-MKELM algorithm. Finally, an online prediction model for fCaO content is built based on COS-MKELM. Experimental results demonstrate that the fCaO content model improves the performance in terms of learning efficiency, regression accuracy, and generalization ability. In addition, the online prediction model can be corrected in real-time when the production conditions of cement clinker change.

Two-Motor Synchronous Decoupling Control Based on Improved Incremental Regularized Extreme Learning Machine

2013 ◽  
Vol 765-767 ◽  
pp. 1854-1857
Author(s):  
Feng Wang ◽  
Jin Lin Ding ◽  
Hong Sun
Keyword(s):  
Neural Network ◽  
Extreme Learning Machine ◽  
Learning Algorithm ◽  
Decoupling Control ◽  
Loop Control ◽  
Fast Learning ◽  
Optimal Network ◽  
Learning Machine ◽  
Hidden Layer ◽  
Hidden Neurons

Neural network generalized inverse (NNGI) can realize two-motor synchronous decoupling control, but traditional neural network (NN) exists many shortcomings, Regular extreme learning machine (RELM) has fast learning and good generalization ability, which is an ideal approach to approximate inverse system. But it is difficult to accurately give the reasonable number of hidden neurons. Improved incremental RELM(IIRELM) is prospected on the basis of analyzing RELM learning algorithm, which can automatically determine optimal network structure through gradually adding new hidden-layer neurons, and prediction model based on IIRELM is applied in two-motor closed-loop control based on NNGI, the decoupling control between velocity and tension is realized. The experimental results proved that the system has excellent performance.

A novel hybrid model using the rotation forest-based differential evolution online sequential extreme learning machine for illumination correction of dyed fabrics

2018 ◽  
Vol 89 (7) ◽  
pp. 1180-1197 ◽  
Author(s):  
Zhiyu Zhou ◽  
Xu Gao ◽  
Jianxin Zhang ◽  
Zefei Zhu ◽  
Xudong Hu
Keyword(s):  
Differential Evolution ◽  
Extreme Learning Machine ◽  
Prediction Accuracy ◽  
Differential Evolution Algorithm ◽  
Image Features ◽  
Support Vector ◽  
Rotation Forest ◽  
Generalization Ability ◽  
Learning Machine ◽  
Hidden Layer

This study proposes an ensemble differential evolution online sequential extreme learning machine (DE-OSELM) for textile image illumination correction based on the rotation forest framework. The DE-OSELM solves the inaccuracy and long training time problems associated with traditional illumination correction algorithms. First, the Grey–Edge framework is used to extract the low-dimensional and efficient image features as online sequential extreme learning machine (OSELM) input vectors to improve the training and learning speed of the OSELM. Since the input weight and hidden-layer bias of OSELMs are randomly obtained, the OSELM algorithm has poor prediction accuracy and low robustness. To overcome this shortcoming, a differential evolution algorithm that has the advantages of good global search ability and robustness is used to optimize the input weight and hidden-layer bias of the DE-OSELM. To further improve the generalization ability and robustness of the illumination correction model, the rotation forest algorithm is used as the ensemble framework, and the DE-OSELM is used as the base learner to replace the regression tree algorithm in the original rotation forest algorithm. Then, the obtained multiple different DE-OSELM learners are aggregated to establish the prediction model. The experimental results show that compared with the textile color correction algorithm based on the support vector regression and extreme learning machine algorithms, the ensemble illumination correction method achieves high prediction accuracy, strong robustness, and good generalization ability.

scholarly journals Aero Engine Component Fault Diagnosis Using Multi-Hidden-Layer Extreme Learning Machine with Optimized Structure

2016 ◽  
Vol 2016 ◽  
pp. 1-11 ◽  
Author(s):  
Shan Pang ◽  
Xinyi Yang ◽  
Xiaofeng Zhang
Keyword(s):  
Fault Diagnosis ◽  
Gas Turbine ◽  
Extreme Learning Machine ◽  
Network Structure ◽  
Gas Turbine Engine ◽  
Data Set ◽  
Turbine Engine ◽  
Engine Component ◽  
Learning Machine ◽  
Hidden Layer

A new aero gas turbine engine gas path component fault diagnosis method based on multi-hidden-layer extreme learning machine with optimized structure (OM-ELM) was proposed. OM-ELM employs quantum-behaved particle swarm optimization to automatically obtain the optimal network structure according to both the root mean square error on training data set and the norm of output weights. The proposed method is applied to handwritten recognition data set and a gas turbine engine diagnostic application and is compared with basic ELM, multi-hidden-layer ELM, and two state-of-the-art deep learning algorithms: deep belief network and the stacked denoising autoencoder. Results show that, with optimized network structure, OM-ELM obtains better test accuracy in both applications and is more robust to sensor noise. Meanwhile it controls the model complexity and needs far less hidden nodes than multi-hidden-layer ELM, thus saving computer memory and making it more efficient to implement. All these advantages make our method an effective and reliable tool for engine component fault diagnosis tool.

Artificial bee colony algorithm–optimized error minimized extreme learning machine and its application in short-term wind speed prediction

2018 ◽  
Vol 43 (3) ◽  
pp. 263-276 ◽  
Author(s):  
Zhongda Tian ◽  
Gang Wang ◽  
Shujiang Li ◽  
Yanhong Wang ◽  
Xiangdong Wang
Keyword(s):  
Wind Speed ◽  
Prediction Model ◽  
Extreme Learning Machine ◽  
Artificial Bee Colony Algorithm ◽  
Artificial Bee Colony ◽  
Short Term ◽  
Machine Model ◽  
Bee Colony ◽  
Learning Machine ◽  
Hidden Layer

In order to improve the prediction accuracy of short-term wind speed, a short-term wind speed prediction model based on artificial bee colony algorithm optimized error minimized extreme learning machine model is proposed. The extreme learning machine has the advantages of fast learning speed and strong generalization ability. But many useless neurons of incremental extreme learning machine have little influences on the final output and, at the same time, reduce the efficiency of the algorithm. The optimal parameters of the hidden layer nodes will make network output error of incremental extreme learning machine decrease with fast speed. Based on the error minimized extreme learning machine, artificial bee colony algorithm is introduced to optimize the parameters of the hidden layer nodes, decrease the number of useless neurons, reduce training and prediction error, achieve the goal of reducing the network complexity, and improve the efficiency of the algorithm. The error minimized extreme learning machine prediction model is constructed with the obtained optimal parameters. The stability and convergence property of artificial bee colony algorithm optimized error minimized extreme learning machine model are proved. The practical short-term wind speed time series is used as the research object and to verify the validity of the prediction model. Multi-step prediction simulation of short-term wind speed is carried out. Compared with other prediction models, simulation results show that the prediction model proposed in this article reduces the training time of the prediction model and decreases the number of hidden layer nodes. The prediction model has higher prediction accuracy and reliability performance, meanwhile improves the performance indicators.

Sign in / Sign up

Export Citation Format

Share Document