scholarly journals FnnmOS-ELM: A Flexible Neural Network Mixed Online Sequential Elm

2019 ◽  
Vol 9 (18) ◽  
pp. 3772
Author(s):  
Xiali Li ◽  
Shuai He ◽  
Junzhi Yu ◽  
Licheng Wu ◽  
Zhao Yue
Keyword(s):  
Neural Network ◽  
Network Performance ◽  
Classification Performance ◽  
Feature Representation ◽  
Mixed Structure ◽  
Training Time ◽  
Learning Speed ◽  
Learning Machine ◽  
Fully Connected ◽  
The Relationship

The learning speed of online sequential extreme learning machine (OS-ELM) algorithms is much higher than that of convolutional neural networks (CNNs) or recurrent neural network (RNNs) on regression and simple classification datasets. However, the general feature extraction of OS-ELM makes it difficult to conveniently and effectively perform classification on some large and complex datasets, e.g., CIFAR. In this paper, we propose a flexible OS-ELM-mixed neural network, termed as fnnmOS-ELM. In this mixed structure, the OS-ELM can replace a part of fully connected layers in CNNs or RNNs. Our framework not only exploits the strong feature representation of CNNs or RNNs, but also performs at a fast speed in terms of classification. Additionally, it avoids the problem of long training time and large parameter size of CNNs or RNNs to some extent. Further, we propose a method for optimizing network performance by splicing OS-ELM after CNN or RNN structures. Iris, IMDb, CIFAR-10, and CIFAR-100 datasets are employed to verify the performance of the fnnmOS-ELM. The relationship between hyper-parameters and the performance of the fnnmOS-ELM is explored, which sheds light on the optimization of network performance. Finally, the experimental results demonstrate that the fnnmOS-ELM has a stronger feature representation and higher classification performance than contemporary methods.

scholarly journals Label Rectification Learning through Kernel Extreme Learning Machine

2021 ◽  
Vol 2021 ◽  
pp. 1-6
Author(s):  
Qiang Cai ◽  
Fenghai Li ◽  
Yifan Chen ◽  
Haisheng Li ◽  
Jian Cao ◽  
...  
Keyword(s):  
Neural Network ◽  
Convolutional Neural Network ◽  
Image Classification ◽  
Extreme Learning Machine ◽  
Classification Performance ◽  
Considerable Progress ◽  
Strong Representation ◽  
Kernel Extreme Learning Machine ◽  
Classification Tasks ◽  
Learning Machine

Along with the strong representation of the convolutional neural network (CNN), image classification tasks have achieved considerable progress. However, majority of works focus on designing complicated and redundant architectures for extracting informative features to improve classification performance. In this study, we concentrate on rectifying the incomplete outputs of CNN. To be concrete, we propose an innovative image classification method based on Label Rectification Learning (LRL) through kernel extreme learning machine (KELM). It mainly consists of two steps: (1) preclassification, extracting incomplete labels through a pretrained CNN, and (2) label rectification, rectifying the generated incomplete labels by the KELM to obtain the rectified labels. Experiments conducted on publicly available datasets demonstrate the effectiveness of our method. Notably, our method is extensible which can be easily integrated with off-the-shelf networks for improving performance.

A Study of the Influence of Data Complexity and Similarity on Soft Biometrics Classification Performance in a Transfer Learning Scenario

2021 ◽  
Vol 18 (2) ◽  
pp. 56-65
Author(s):  
Marcelo Romero ◽  
◽  
Matheus Gutoski ◽  
Leandro Takeshi Hattori ◽  
Manassés Ribeiro ◽  
...  
Keyword(s):  
Transfer Learning ◽  
Classification Performance ◽  
Fine Tuning ◽  
Similarity Metrics ◽  
Soft Biometrics ◽  
Final Layer ◽  
The One ◽  
Fully Connected ◽  
The Relationship ◽  
Fine Tune

Transfer learning is a paradigm that consists in training and testing classifiers with datasets drawn from distinct distributions. This technique allows to solve a particular problem using a model that was trained for another purpose. In the recent years, this practice has become very popular due to the increase of public available pre-trained models that can be fine-tuned to be applied in different scenarios. However, the relationship between the datasets used for training the model and the test data is usually not addressed, specially where the fine-tuning process is done only for the fully connected layers of a Convolutional Neural Network with pre-trained weights. This work presents a study regarding the relationship between the datasets used in a transfer learning process in terms of the performance achieved by models complexities and similarities. For this purpose, we fine-tune the final layer of Convolutional Neural Networks with pre-trained weights using diverse soft biometrics datasets. An evaluation of the performances of the models, when tested with datasets that are different from the one used for training the model, is presented. Complexity and similarity metrics are also used to perform the evaluation.

scholarly journals Single-Option P300-BCI Performance Is Affected by Visual Stimulation Conditions

Sensors ◽  
2020 ◽  
Vol 20 (24) ◽  
pp. 7198
Author(s):  
Juan David Chailloux Peguero ◽  
Omar Mendoza-Montoya ◽  
Javier M. Antelis
Keyword(s):  
Visual Stimulation ◽  
Event Related Potentials ◽  
Classification Performance ◽  
Single Trial ◽  
Experimental Protocol ◽  
Training Time ◽  
Machine Learning Model ◽  
Visual Frame ◽  
Related Potentials ◽  
The Relationship

The P300 paradigm is one of the most promising techniques for its robustness and reliability in Brain-Computer Interface (BCI) applications, but it is not exempt from shortcomings. The present work studied single-trial classification effectiveness in distinguishing between target and non-target responses considering two conditions of visual stimulation and the variation of the number of symbols presented to the user in a single-option visual frame. In addition, we also investigated the relationship between the classification results of target and non-target events when training and testing the machine-learning model with datasets containing different stimulation conditions and different number of symbols. To this end, we designed a P300 experimental protocol considering, as conditions of stimulation: the color highlighting or the superimposing of a cartoon face and from four to nine options. These experiments were carried out with 19 healthy subjects in 3 sessions. The results showed that the Event-Related Potentials (ERP) responses and the classification accuracy are stronger with cartoon faces as stimulus type and similar irrespective of the amount of options. In addition, the classification performance is reduced when using datasets with different type of stimulus, but it is similar when using datasets with different the number of symbols. These results have a special connotation for the design of systems, in which it is intended to elicit higher levels of evoked potentials and, at the same time, optimize training time.

scholarly journals A Customized Deep Neural Network Approach to Investigate Travel Mode Choice with Interpretable Utility Information

2020 ◽  
Vol 2020 ◽  
pp. 1-11
Author(s):  
Zhengchao Zhang ◽  
Congyuan Ji ◽  
Yineng Wang ◽  
Yanni Yang
Keyword(s):  
Neural Network ◽  
Learning Community ◽  
Mode Choice ◽  
Feature Representation ◽  
Classification Problems ◽  
Neural Network Approach ◽  
Nonlinear Fitting ◽  
Linear Utility ◽  
Utility Models ◽  
Fully Connected

Discrete choice modeling of travel modes is an essential part of traffic planning and management. Thus far, this field has been dominated by multinomial logit (MNL) models with a linear utility specification. However, deep neural networks (DNNs), owing to their powerful capacity of nonlinear fitting, are now rapidly replacing these models. This is because, by using DNNs, mode choice can be assimilated with the classification problems within the machine learning community. This article proposes a newly designed DNN framework for traffic mode choice in the style of two hidden layers. First, a local-connected layer automatically extracts an effective utility specification from the available data, and then, a fully connected layer augments the feature representation. Validated by a practical city-wide multimodal traffic dataset in Beijing, our model significantly outperforms the random utility models and simple fully connected neural network in terms of the prediction accuracy. Besides the comparison of the predictive power, we also present the interpretability of the proposed model.

scholarly journals One-Class Classification with Extreme Learning Machine

2015 ◽  
Vol 2015 ◽  
pp. 1-11 ◽  
Author(s):  
Qian Leng ◽  
Honggang Qi ◽  
Jun Miao ◽  
Wentao Zhu ◽  
Guiping Su
Keyword(s):  
Neural Network ◽  
Extreme Learning Machine ◽  
Classification Problem ◽  
Traditional Learning ◽  
Learning Speed ◽  
One Class Classifier ◽  
Learning Machine ◽  
Hidden Layer ◽  
One Class Classification ◽  
Network Approaches

One-class classification problem has been investigated thoroughly for past decades. Among one of the most effective neural network approaches for one-class classification, autoencoder has been successfully applied for many applications. However, this classifier relies on traditional learning algorithms such as backpropagation to train the network, which is quite time-consuming. To tackle the slow learning speed in autoencoder neural network, we propose a simple and efficient one-class classifier based on extreme learning machine (ELM). The essence of ELM is that the hidden layer need not be tuned and the output weights can be analytically determined, which leads to much faster learning speed. The experimental evaluation conducted on several real-world benchmarks shows that the ELM based one-class classifier can learn hundreds of times faster than autoencoder and it is competitive over a variety of one-class classification methods.

The Design of Image Compression with BP Neural Network Based on the Dynamic Adjusting Hidden Layer Nodes

2012 ◽  
Vol 433-440 ◽  
pp. 3797-3801 ◽  
Author(s):  
Jing Li ◽  
Xin Hui Wu ◽  
Chang Hai Qin ◽  
Jing Zhao
Keyword(s):  
Neural Network ◽  
Image Compression ◽  
Bp Neural Network ◽  
Training Image ◽  
Compression Algorithms ◽  
Training Time ◽  
Training Images ◽  
Learning Speed ◽  
Dispersion Degree ◽  
Hidden Layer

Aiming at the image compression algorithms with the used BP neural network ,they have inherent defects of poor universality and long training time, a model of the dynamic adjusting hidden layer nodes of BP neural network is designed. According to the training image, using the correlation coefficient and dispersion degree of the same hidden layer’s nodes, we cut and delete some no nodes, this algorithm not only can improve learning speed effectively but also has certain generalization ability, and can complete the task of no- training images compression through experiments.

scholarly journals Can Pre-Trained Convolutional Neural Networks be directly used as a Feature Extractor for Video-based Neonatal Sleep and Wake Classification?

2020 ◽  
Author(s):  
Muhammad Awais ◽  
Xi Long ◽  
Bin Yin ◽  
Chen chen ◽  
Saeed Akbarzadeh ◽  
...  
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Convolutional Neural Networks ◽  
Video Recording ◽  
Principal Component ◽  
Classification Performance ◽  
Support Vector ◽  
Video Frames ◽  
Feature Extractor ◽  
Fully Connected

Abstract Objective: In this paper, we propose to evaluate the use of a pre-trained convolutional neural networks (CNNs) as a features extractor followed by the Principal Component Analysis (PCA) to find the best discriminant features to perform classification using support vector machine (SVM) algorithm for neonatal sleep and wake states using Fluke® facial video frames. Using pre-trained CNNs as feature extractor would hugely reduce the effort of collecting new neonatal data for training a neural network which could be computationally very expensive. The features are extracted after fully connected layers (FCL’s), where we compare several pre-trained CNNs, e.g., VGG16, VGG19, InceptionV3, GoogLeNet, ResNet, and AlexNet. Results: From around 2-h Fluke® video recording of seven neonate, we achieved a modest classification performance with an accuracy, sensitivity, and specificity of 65.3%, 69.8%, 61.0%, respectively with AlexNet using Fluke® (RGB) video frames. This indicates that using a pre-trained model as a feature extractor could not fully suffice for highly reliable sleep and wake classification in neonates. Therefore, in future a dedicated neural network trained on neonatal data or a transfer learning approach is required.

scholarly journals Kinerja Metode CNN untuk Klasifikasi Pneumonia dengan Variasi Ukuran Citra Input

2021 ◽  
Vol 8 (3) ◽  
pp. 533
Author(s):  
Budi Nugroho ◽  
Eva Yulia Puspaningrum
Keyword(s):  
Neural Network ◽  
Convolutional Neural Network ◽  
Extreme Learning Machine ◽  
Classification Performance ◽  
Input Image ◽  
X Rays ◽  
Image Size ◽  
X Ray ◽  
The Difference ◽  
Learning Machine

<p class="Abstrak">Saat ini banyak dikembangkan proses pendeteksian pneumonia berdasarkan citra paru-paru dari hasil foto rontgen (x-ray), sebagaimana juga dilakukan pada penelitian ini. Metode yang digunakan adalah <em>Convolutional Neural Network</em> (CNN) dengan arsitektur yang berbeda dengan sejumlah penelitian sebelumnya. Selain itu, penelitian ini juga memodifikasi model CNN dimana metode <em>Extreme Learning Machine</em> (ELM) digunakan pada bagian klasifikasi, yang kemudian disebut CNN-ELM. Dataset untuk uji coba menggunakan kumpulan citra paru-paru hasil foto rontgen pada Kaggle yang terdiri atas 1.583 citra normal dan 4.237 citra pneumonia. Citra asal pada dataset kaggle ini bervariasi, tetapi hampir semua diatas ukuran 1000x1000 piksel. Ukuran citra yang besar ini dapat membuat pemrosesan klasifikasi kurang efektif, sehingga mesin CNN biasanya memodifikasi ukuran citra menjadi lebih kecil. Pada penelitian ini, pengujian dilakukan dengan variasi ukuran citra input, untuk mengetahui pengaruhnya terhadap kinerja mesin pengklasifikasi. Hasil uji coba menunjukkan bahwa ukuran citra input berpengaruh besar terhadap kinerja klasifikasi pneumonia, baik klasifikasi yang menggunakan metode CNN maupun CNN-ELM. Pada ukuran citra input 200x200, metode CNN dan CNN-ELM menunjukkan kinerja paling tinggi. Jika kinerja kedua metode itu dibandingkan, maka Metode CNN-ELM menunjukkan kinerja yang lebih baik daripada CNN pada semua skenario uji coba. Pada kondisi kinerja paling tinggi, selisih akurasi antara metode CNN-ELM dan CNN mencapai 8,81% dan selisih F1 Score mencapai 0,0729. Hasil penelitian ini memberikan informasi penting bahwa ukuran citra input memiliki pengaruh besar terhadap kinerja klasifikasi pneumonia, baik klasifikasi menggunakan metode CNN maupun CNN-ELM. Selain itu, pada semua ukuran citra input yang digunakan untuk proses klasifikasi, metode CNN-ELM menunjukkan kinerja yang lebih baik daripada metode CNN.</p><p class="Abstrak"> </p><p class="Abstrak"><em><strong>Abstract</strong></em></p><p class="Abstract"><em>This research developed a pneumonia detection machine based on the lungs' images from X-rays (x-rays). The method used is the Convolutional Neural Network (CNN) with a different architecture from some previous research. Also, the CNN model is modified, where the classification process uses the Extreme Learning Machine (ELM), which is then called the CNN-ELM method. The empirical experiments dataset used a collection of lung x-ray images on Kaggle consisting of 1,583 normal images and 4,237 pneumonia images. The original image's size on the Kaggle dataset varies, but almost all of the images are more than 1000x1000 pixels. For classification processing to be more effective, CNN machines usually use reduced-size images. In this research, experiments were carried out with various input image sizes to determine the effect on the classifier's performance. The experimental results show that the input images' size has a significant effect on the classification performance of pneumonia, both the CNN and CNN-ELM classification methods. At the 200x200 input image size, the CNN and CNN-ELM methods showed the highest performance. If the two methods' performance is compared, then the CNN-ELM Method shows better performance than CNN in all test scenarios. The difference in accuracy between the CNN-ELM and CNN methods reaches 8.81% at the highest performance conditions, and the difference in F1-Score reaches 0.0729. This research provides important information that the size of the input image has a major influence on the classification performance of pneumonia, both classification using the CNN and CNN-ELM methods. Also, on all input image sizes used for the classification process, the CNN-ELM method shows better performance than the CNN method.</em></p>

scholarly journals Progressive Convolutional Neural Network for Incremental Learning

Electronics ◽  
2021 ◽  
Vol 10 (16) ◽  
pp. 1879
Author(s):  
Zahid Ali Siddiqui ◽  
Unsang Park
Keyword(s):  
Neural Network ◽  
Incremental Learning ◽  
Network Architecture ◽  
Structural Changes ◽  
Training Time ◽  
Learning Technique ◽  
Incremental Training ◽  
Fully Connected ◽  
Fine Tune ◽  
Entire Network

In this paper, we present a novel incremental learning technique to solve the catastrophic forgetting problem observed in the CNN architectures. We used a progressive deep neural network to incrementally learn new classes while keeping the performance of the network unchanged on old classes. The incremental training requires us to train the network only for new classes and fine-tune the final fully connected layer, without needing to train the entire network again, which significantly reduces the training time. We evaluate the proposed architecture extensively on image classification task using Fashion MNIST, CIFAR-100 and ImageNet-1000 datasets. Experimental results show that the proposed network architecture not only alleviates catastrophic forgetting but can also leverages prior knowledge via lateral connections to previously learned classes and their features. In addition, the proposed scheme is easily scalable and does not require structural changes on the network trained on the old task, which are highly required properties in embedded systems.

scholarly journals Pre-training with non-expert human demonstration for deep reinforcement learning

2019 ◽  
Vol 34 ◽  
Author(s):  
Gabriel V. de la Cruz ◽  
Yunshu Du ◽  
Matthew E. Taylor
Keyword(s):  
Reinforcement Learning ◽  
Feature Learning ◽  
Feature Representation ◽  
Superior Performance ◽  
Learning Goals ◽  
Training Time ◽  
Learning Speed ◽  
Small Set ◽  
Data Efficiency ◽  
Human Demonstration

Abstract Deep reinforcement learning (deep RL) has achieved superior performance in complex sequential tasks by using deep neural networks as function approximators to learn directly from raw input images. However, learning directly from raw images is data inefficient. The agent must learn feature representation of complex states in addition to learning a policy. As a result, deep RL typically suffers from slow learning speeds and often requires a prohibitively large amount of training time and data to reach reasonable performance, making it inapplicable to real-world settings where data are expensive. In this work, we improve data efficiency in deep RL by addressing one of the two learning goals, feature learning. We leverage supervised learning to pre-train on a small set of non-expert human demonstrations and empirically evaluate our approach using the asynchronous advantage actor-critic algorithms in the Atari domain. Our results show significant improvements in learning speed, even when the provided demonstration is noisy and of low quality.

Sign in / Sign up

Export Citation Format

Share Document