Analytical Calculation of Hidden Layer Biases in Extreme Learning Machines with Sinusoidal Activation Function

2019 ◽  
Author(s):  
Matheus Rocha Barbosa ◽  
Luis Gustavo Mota Souza
Keyword(s):  
Analytical Calculation ◽  
Activation Function ◽  
Extreme Learning Machines ◽  
Learning Machines ◽  
Hidden Layer

scholarly journals Online Regularized and Kernelized Extreme Learning Machines with Forgetting Mechanism

2014 ◽  
Vol 2014 ◽  
pp. 1-11 ◽  
Author(s):  
Xinran Zhou ◽  
Zijian Liu ◽  
Congxu Zhu
Keyword(s):  
Extreme Learning Machine ◽  
Feedforward Neural Networks ◽  
Matrix Inversion ◽  
Time Varying ◽  
Extreme Learning Machines ◽  
Learning Machines ◽  
Matrix Expansion ◽  
Learning Machine ◽  
Hidden Layer ◽  
Forgetting Mechanism

To apply the single hidden-layer feedforward neural networks (SLFN) to identify time-varying system, online regularized extreme learning machine (ELM) with forgetting mechanism (FORELM) and online kernelized ELM with forgetting mechanism (FOKELM) are presented in this paper. The FORELM updates the output weights of SLFN recursively by using Sherman-Morrison formula, and it combines advantages of online sequential ELM with forgetting mechanism (FOS-ELM) and regularized online sequential ELM (ReOS-ELM); that is, it can capture the latest properties of identified system by studying a certain number of the newest samples and also can avoid issue of ill-conditioned matrix inversion by regularization. The FOKELM tackles the problem of matrix expansion of kernel based incremental ELM (KB-IELM) by deleting the oldest sample according to the block matrix inverse formula when samples occur continually. The experimental results show that the proposed FORELM and FOKELM have better stability than FOS-ELM and have higher accuracy than ReOS-ELM in nonstationary environments; moreover, FORELM and FOKELM have time efficiencies superiority over dynamic regression extreme learning machine (DR-ELM) under certain conditions.

scholarly journals Seizure Detection Based on Adaptive Feature Extraction by Applying Extreme Learning Machines

2021 ◽  
Vol 38 (2) ◽  
pp. 331-340
Author(s):  
Muhammet Baykara ◽  
Awf Abdulrahman
Keyword(s):  
Feature Extraction ◽  
Seizure Detection ◽  
Activation Function ◽  
Error Rates ◽  
Extreme Learning Machines ◽  
Eeg Signals ◽  
Learning Machines ◽  
Brain Signals ◽  
Stockwell Transform ◽  
The Brain

Epilepsy is one of the most common chronic disorder which negatively affects the patients' life. The functionality of the brain can be obtained from brain signals and it is vital to analyze and examine the brain signals in seizure detection processes. In this study, we performed machine learning-based and signal processing methods to detect epileptic signals. To do that, we examined three different EEG signals (healthy, ictal, and interictal) with two different classes (healthy ones and epileptic ones). Our proposed method consists of three stages which are preprocessing, feature extraction, and classification. In the preprocessing phase, EEG signals normalized to scale all samples into [0,1] range. After Stockwell Transform was applied and chaotic features and Parseval's Energy collected from each EEG signal. In the last part, EEG signals were classified with ELM (Extreme Learning Machines) with different parameters. Our study shows the best classification accuracy obtained from the Sigmoid activation function with the number of 100 hidden neurons. The highlights of this study are: Stockwell Transform is used; Entropy values are selected based on the adaptive process. Threshold values are determined according to the error rates; ELM classifier algorithm is applied.

scholarly journals An Advanced Pruning Method in the Architecture of Extreme Learning Machines Using L1-Regularization and Bootstrapping

Electronics ◽  
2020 ◽  
Vol 9 (5) ◽  
pp. 811
Author(s):  
Paulo Vitor de Campos Souza ◽  
Luiz Carlos Bambirra Torres ◽  
Gustavo Rodrigues Lacerda Silva ◽  
Antonio de Padua Braga ◽  
Edwin Lughofer
Keyword(s):  
Time Series Prediction ◽  
Model Errors ◽  
Extreme Learning Machines ◽  
Data Set ◽  
Learning Machines ◽  
Pruning Method ◽  
Intermediate Layers ◽  
The Neural Network ◽  
Model Response ◽  
Hidden Layer

Extreme learning machines (ELMs) are efficient for classification, regression, and time series prediction, as well as being a clear solution to backpropagation structures to determine values in intermediate layers of the learning model. One of the problems that an ELM may face is due to a large number of neurons in the hidden layer, making the expert model a specific data set. With a large number of neurons in the hidden layer, overfitting is more likely and thus unnecessary information can deterioriate the performance of the neural network. To solve this problem, a pruning method is proposed, called Pruning ELM Using Bootstrapped Lasso BR-ELM, which is based on regularization and resampling techniques, to select the most representative neurons for the model response. This method is based on an ensembled variant of Lasso (achieved through bootstrap replications) and aims to shrink the output weight parameters of the neurons to 0 as many and as much as possible. According to a subset of candidate regressors having significant coefficient values (greater than 0), it is possible to select the best neurons in the hidden layer of the ELM. Finally, pattern classification tests and benchmark regression tests of complex real-world problems are performed by comparing the proposed approach to other pruning models for ELMs. It can be seen that statistically BR-ELM can outperform several related state-of-the-art methods in terms of classification accuracies and model errors (while performing equally to Pruning-ELM P-ELM), and this with a significantly reduced number of finally selected neurons.

scholarly journals Wind Turbine Prognosis Models Based on SCADA Data and Extreme Learning Machines

2021 ◽  
Vol 11 (2) ◽  
pp. 590
Author(s):  
Pere Marti-Puig ◽  
Alejandro Blanco-M. ◽  
Moisès Serra-Serra ◽  
Jordi Solé-Casals
Keyword(s):  
Wind Turbine ◽  
Wind Turbines ◽  
Feedforward Neural Networks ◽  
Model Parameters ◽  
Extreme Learning Machines ◽  
Target Variable ◽  
Learning Machines ◽  
Realistic Option ◽  
Hidden Layer ◽  
Search Approach

In this paper, a method to build models to monitor and evaluate the health status of wind turbines using Single-hidden Layer Feedforward Neural networks (SLFN) is presented. The models are trained using the Extreme Learning Machines (ELM) strategy. The data used is obtained from the SCADA systems, easily available in modern wind turbines. The ELM technique requires very low computational costs for the training of the models, and thus allows for the integration of a grid-search approach with parallelized instances to find out the optimal model parameters. These models can be built both individually, considering the turbines separately, or as an aggregate for the whole wind plant. The followed strategy consists in predicting a target variable using the rest of the variables of the system/subsystem, computing the error deviation from the real target variable and finally comparing high error values with a selection of alarm events for that system, therefore validating the performance of the model. The experimental results indicate that this methodology leads to the detection of mismatches in the stages of the system’s failure, thus making it possible to schedule the maintenance operation before a critical failure occurs. The simplicity of the ELM systems and the ease with which the parameters can be adjusted make it a realistic option to be implemented in wind turbine models to work in real time.

Analysis of Non-Linear Activation Functions for Classification Tasks Using Convolutional Neural Networks

2019 ◽  
Vol 12 (3) ◽  
pp. 156-161 ◽  
Author(s):  
Aman Dureja ◽  
Payal Pahwa
Keyword(s):  
Neural Networks ◽  
Deep Neural Networks ◽  
Activation Function ◽  
Primary Objective ◽  
Experimental Comparison ◽  
Activation Functions ◽  
Practical Applications ◽  
Network Activation ◽  
Non Linear ◽  
Hidden Layer

Background: In making the deep neural network, activation functions play an important role. But the choice of activation functions also affects the network in term of optimization and to retrieve the better results. Several activation functions have been introduced in machine learning for many practical applications. But which activation function should use at hidden layer of deep neural networks was not identified. Objective: The primary objective of this analysis was to describe which activation function must be used at hidden layers for deep neural networks to solve complex non-linear problems. Methods: The configuration for this comparative model was used by using the datasets of 2 classes (Cat/Dog). The number of Convolutional layer used in this network was 3 and the pooling layer was also introduced after each layer of CNN layer. The total of the dataset was divided into the two parts. The first 8000 images were mainly used for training the network and the next 2000 images were used for testing the network. Results: The experimental comparison was done by analyzing the network by taking different activation functions on each layer of CNN network. The validation error and accuracy on Cat/Dog dataset were analyzed using activation functions (ReLU, Tanh, Selu, PRelu, Elu) at number of hidden layers. Overall the Relu gave best performance with the validation loss at 25th Epoch 0.3912 and validation accuracy at 25th Epoch 0.8320. Conclusion: It is found that a CNN model with ReLU hidden layers (3 hidden layers here) gives best results and improve overall performance better in term of accuracy and speed. These advantages of ReLU in CNN at number of hidden layers are helpful to effectively and fast retrieval of images from the databases.

scholarly journals Hardware implementation of radial-basis neural networks with Gaussian activation functions on FPGA

2021 ◽  
Author(s):  
Volodymyr Shymkovych ◽  
Sergii Telenyk ◽  
Petro Kravets
Keyword(s):  
Neural Networks ◽  
Hardware Implementation ◽  
Gaussian Function ◽  
Activation Function ◽  
Rbf Neural Networks ◽  
Activation Functions ◽  
Rbf Network ◽  
Combination Scheme ◽  
Radial Basis ◽  
Hidden Layer

AbstractThis article introduces a method for realizing the Gaussian activation function of radial-basis (RBF) neural networks with their hardware implementation on field-programmable gaits area (FPGAs). The results of modeling of the Gaussian function on FPGA chips of different families have been presented. RBF neural networks of various topologies have been synthesized and investigated. The hardware component implemented by this algorithm is an RBF neural network with four neurons of the latent layer and one neuron with a sigmoid activation function on an FPGA using 16-bit numbers with a fixed point, which took 1193 logic matrix gate (LUTs—LookUpTable). Each hidden layer neuron of the RBF network is designed on an FPGA as a separate computing unit. The speed as a total delay of the combination scheme of the block RBF network was 101.579 ns. The implementation of the Gaussian activation functions of the hidden layer of the RBF network occupies 106 LUTs, and the speed of the Gaussian activation functions is 29.33 ns. The absolute error is ± 0.005. The Spartan 3 family of chips for modeling has been used to get these results. Modeling on chips of other series has been also introduced in the article. RBF neural networks of various topologies have been synthesized and investigated. Hardware implementation of RBF neural networks with such speed allows them to be used in real-time control systems for high-speed objects.

Sign in / Sign up

Export Citation Format

Share Document