scholarly journals An Advanced Pruning Method in the Architecture of Extreme Learning Machines Using L1-Regularization and Bootstrapping

Electronics ◽  
2020 ◽  
Vol 9 (5) ◽  
pp. 811
Author(s):  
Paulo Vitor de Campos Souza ◽  
Luiz Carlos Bambirra Torres ◽  
Gustavo Rodrigues Lacerda Silva ◽  
Antonio de Padua Braga ◽  
Edwin Lughofer
Keyword(s):  
Time Series Prediction ◽  
Model Errors ◽  
Extreme Learning Machines ◽  
Data Set ◽  
Learning Machines ◽  
Pruning Method ◽  
Intermediate Layers ◽  
The Neural Network ◽  
Model Response ◽  
Hidden Layer

Extreme learning machines (ELMs) are efficient for classification, regression, and time series prediction, as well as being a clear solution to backpropagation structures to determine values in intermediate layers of the learning model. One of the problems that an ELM may face is due to a large number of neurons in the hidden layer, making the expert model a specific data set. With a large number of neurons in the hidden layer, overfitting is more likely and thus unnecessary information can deterioriate the performance of the neural network. To solve this problem, a pruning method is proposed, called Pruning ELM Using Bootstrapped Lasso BR-ELM, which is based on regularization and resampling techniques, to select the most representative neurons for the model response. This method is based on an ensembled variant of Lasso (achieved through bootstrap replications) and aims to shrink the output weight parameters of the neurons to 0 as many and as much as possible. According to a subset of candidate regressors having significant coefficient values (greater than 0), it is possible to select the best neurons in the hidden layer of the ELM. Finally, pattern classification tests and benchmark regression tests of complex real-world problems are performed by comparing the proposed approach to other pruning models for ELMs. It can be seen that statistically BR-ELM can outperform several related state-of-the-art methods in terms of classification accuracies and model errors (while performing equally to Pruning-ELM P-ELM), and this with a significantly reduced number of finally selected neurons.

A new pruning method for extreme learning machines via genetic algorithms

2016 ◽  
Vol 44 ◽  
pp. 101-107 ◽  
Author(s):  
Alisson S.C. Alencar ◽  
Ajalmar R. Rocha Neto ◽  
João Paulo P. Gomes
Keyword(s):  
Genetic Algorithms ◽  
Extreme Learning Machines ◽  
Learning Machines ◽  
Pruning Method

EFFECTS OF NONSINGULAR PREPROCESSING ON FEEDFORWARD NETWORK TRAINING

2005 ◽  
Vol 19 (02) ◽  
pp. 217-247 ◽  
Author(s):  
CHANGHUA YU ◽  
MICHAEL T. MANRY ◽  
JIANG LI
Keyword(s):  
Back Propagation ◽  
Original Data ◽  
Data Sets ◽  
Training Algorithm ◽  
Feedforward Network ◽  
Data Set ◽  
Network Training ◽  
The Neural Network ◽  
Hidden Layer ◽  
Theoretical Analyses

In the neural network literature, many preprocessing techniques, such as feature de-correlation, input unbiasing and normalization, are suggested to accelerate multilayer perceptron training. In this paper, we show that a network trained with an original data set and one trained with a linear transformation of the original data will go through the same training dynamics, as long as they start from equivalent states. Thus preprocessing techniques may not be helpful and are merely equivalent to using a different weight set to initialize the network. Theoretical analyses of such preprocessing approaches are given for conjugate gradient, back propagation and the Newton method. In addition, an efficient Newton-like training algorithm is proposed for hidden layer training. Experiments on various data sets confirm the theoretical analyses and verify the improvement of the new algorithm.

scholarly journals Online Regularized and Kernelized Extreme Learning Machines with Forgetting Mechanism

2014 ◽  
Vol 2014 ◽  
pp. 1-11 ◽  
Author(s):  
Xinran Zhou ◽  
Zijian Liu ◽  
Congxu Zhu
Keyword(s):  
Extreme Learning Machine ◽  
Feedforward Neural Networks ◽  
Matrix Inversion ◽  
Time Varying ◽  
Extreme Learning Machines ◽  
Learning Machines ◽  
Matrix Expansion ◽  
Learning Machine ◽  
Hidden Layer ◽  
Forgetting Mechanism

To apply the single hidden-layer feedforward neural networks (SLFN) to identify time-varying system, online regularized extreme learning machine (ELM) with forgetting mechanism (FORELM) and online kernelized ELM with forgetting mechanism (FOKELM) are presented in this paper. The FORELM updates the output weights of SLFN recursively by using Sherman-Morrison formula, and it combines advantages of online sequential ELM with forgetting mechanism (FOS-ELM) and regularized online sequential ELM (ReOS-ELM); that is, it can capture the latest properties of identified system by studying a certain number of the newest samples and also can avoid issue of ill-conditioned matrix inversion by regularization. The FOKELM tackles the problem of matrix expansion of kernel based incremental ELM (KB-IELM) by deleting the oldest sample according to the block matrix inverse formula when samples occur continually. The experimental results show that the proposed FORELM and FOKELM have better stability than FOS-ELM and have higher accuracy than ReOS-ELM in nonstationary environments; moreover, FORELM and FOKELM have time efficiencies superiority over dynamic regression extreme learning machine (DR-ELM) under certain conditions.

scholarly journals Prediction of Area and Production of Groundnut Using Box-Jenkins Arima and Neural Network Approach

2020 ◽  
Author(s):  
S. T. Pavana Kumar ◽  
Ferdinand B. Lyngdoh
Keyword(s):  
Neural Network ◽  
Moving Average ◽  
Arima Model ◽  
Network Models ◽  
Arima Models ◽  
Neural Network Models ◽  
Neural Network Approach ◽  
Data Set ◽  
The Neural Network ◽  
Hidden Layer

Selection of parameters for Auto Regressive Integrated Moving Average (ARIMA) model in the prediction process is one of the most important tasks. In the present study, groundnut data was utlised to decide appropriate p, d, q parameters for ARIMA model for the prediction purpose. Firstly, the models were fit to data without splitting into training and validation/testing sets and evaluated for their efficiency in predicting the area and production of groundnut over the years. Meanwhile, models are compared among other fitted ARIMA models with different p, d, q parameters based on decision criteria’s viz., ME, RMSE, MAPE, AIC, BIC and R-Square. The ARIMA model with parameters p-2 d-1-2, q-1-2 are found adequate in predicting the area as well as production of groundnut. The model ARIMA (2, 2, 2) and ARIMA (2,1,1) predicted the area of groundnut crop with minimum error estimates and residual characteristics (ei). The models were fit into split data i.e., training and test data set, but these models’ prediction power (R-Square) declined during testing. In case of predicting the area, ARIMA (2,2,2) was consistent over the split data but it was not consistent while predicting the production over years. Feed-forward neural networks with single hidden layer were fit to complete, training and split data. The neural network models provided better estimates compared to Box-Jenkins ARIMA models. The data was analysed using R-Studio.

scholarly journals Adaptive Ensemble Models of Extreme Learning Machines for Time Series Prediction

2009 ◽  
pp. 305-314 ◽  
Author(s):  
Mark van Heeswijk ◽  
Yoan Miche ◽  
Tiina Lindh-Knuutila ◽  
Peter A. J. Hilbers ◽  
Timo Honkela ◽  
...  
Keyword(s):  
Time Series ◽  
Time Series Prediction ◽  
Extreme Learning Machines ◽  
Ensemble Models ◽  
Learning Machines

scholarly journals Optimizing Convolutional Neural Network Parameters for Better Image Classification

2020 ◽  
Author(s):  
Manik Dhingra ◽  
Sarthak Rawat ◽  
Jinan Fiaidhi
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Image Classification ◽  
Web Service ◽  
Recognition Task ◽  
Extreme Learning Machines ◽  
Data Set ◽  
Learning Machines ◽  
Fully Connected ◽  
Fully Connected Networks

The work presented here works on getting higher performances for image recognition task using convolutional neural networks on the MNIST handwritten digits data-set. A range of techniques are compared for improvements with respect to time and accuracy, such as using one-shot Extreme Learning Machines (ELM) in place of the iteratively tuned fully-connected networks for classification, using transfer learning for faster convergence of image classification, and improving the size of data-set and making robust models by image augmentation. The final implementation is hosted on cloud as a web-service for better visualization of the prediction results.

Sign in / Sign up

Export Citation Format

Share Document