scholarly journals Pattern Classification Based on RBF Networks with Self-Constructing Clustering and Hybrid Learning

2020 ◽  
Vol 10 (17) ◽  
pp. 5886
Author(s):  
Zan-Rong He ◽  
Yan-Ting Lin ◽  
Chen-Yu Wu ◽  
Ying-Jie You ◽  
Shie-Jue Lee
Keyword(s):  
Pattern Classification ◽  
Clustering Algorithm ◽  
Least Squares Method ◽  
Hybrid Learning ◽  
Basis Functions ◽  
Training Data ◽  
Label Pattern ◽  
Classification Problems ◽  
Rbf Networks ◽  
Hidden Layer

Radial basis function (RBF) networks are widely adopted to solve problems in the field of pattern classification. However, in the construction phase of such networks, there are several issues encountered, such as the determination of the number of nodes in the hidden layer, the form and initialization of the basis functions, and the learning of the parameters involved in the networks. In this paper, we present a novel approach for constructing RBF networks for pattern classification problems. An iterative self-constructing clustering algorithm is used to produce a desired number of clusters from the training data. Accordingly, the number of nodes in the hidden layer is determined. Basis functions are then formed, and their centers and deviations are initialized to be the centers and deviations of the corresponding clusters. Then, the parameters of the network are refined with a hybrid learning strategy, involving hyperbolic tangent sigmoid functions, steepest descent backpropagation, and least squares method. As a result, optimized RBF networks are obtained. With this approach, the number of nodes in the hidden layer is determined and basis functions are derived automatically, and higher classification rates can be achieved. Furthermore, the approach is applicable to construct RBF networks for solving both single-label and multi-label pattern classification problems.

Adaptive Genetic Algorithm for Multilayer RBF Network and its Application on Real Function Approximation

2013 ◽  
Vol 380-384 ◽  
pp. 1166-1169 ◽  
Author(s):  
Yan Hui Wang ◽  
Kun Zhang
Keyword(s):  
Genetic Algorithm ◽  
Computer Simulation ◽  
Real Function ◽  
Function Approximation ◽  
Clustering Algorithm ◽  
Least Squares Method ◽  
Adaptive Genetic Algorithm ◽  
Rbf Networks ◽  
Rbf Network ◽  
Hidden Layer

With the application of adaptive genetic algorithm to the training of multi-layer RBF networks and the optimization of the hidden layer centers and width values and using regularized least squares method, weight vectors is obtained. Computer simulation shows that the precision of real function approximation by this algorithm is much higher than the precision by clustering algorithm for multi-layer RBF networks.

scholarly journals A study of extreme learning machine on small sample-sized classification problems

2021 ◽  
Vol 2107 (1) ◽  
pp. 012013
Author(s):  
Boon Pin Ooi ◽  
Norasmadi Abdul Rahim ◽  
Maz Jamilah Masnan ◽  
Ammar Zakaria
Keyword(s):  
Extreme Learning Machine ◽  
Small Sample ◽  
Majority Voting ◽  
Training Data ◽  
Ensemble Model ◽  
Classification Problems ◽  
Total Data ◽  
Learning Machine ◽  
Hidden Layer ◽  
Mlp Model

Abstract Extreme learning machine (ELM) is a special type of single hidden layer feedforward neural network that emphasizes training speed and optimal generalization. The ELM model proposes that the weights of hidden neurons need not be tuned, and the weights of output neurons can be calculated by finding the Moore-Penrose generalized inverse method. Thus, the ELM classifier is suitable to use in a homogeneous ensemble model due to the untuned random hidden weights which promote diversity even with the same training data. This paper studies the effectiveness of the ELM ensemble models in solving small sample-sized classification problems. The research involves two variants of the ensemble model: the normal ELM ensemble with majority voting (ELE), and the random subspace method (RS-ELM). To simulate the small sample cases, only 30% of the total data will be used as the training data. Experiment results show that the RS-ELM model can outperform a multi-layer perceptron (MLP) model under the assumptions of a Friedman test. Furthermore, the ELE model has similar performance as an MLP model under the same assumptions.

scholarly journals MultiKOC: Multi-One-Class Classifier Based K-Means Clustering

Algorithms ◽  
2021 ◽  
Vol 14 (5) ◽  
pp. 134
Author(s):  
Loai Abdallah ◽  
Murad Badarna ◽  
Waleed Khalifa ◽  
Malik Yousef
Keyword(s):  
Clustering Algorithm ◽  
Main Idea ◽  
Molecular Classification ◽  
Positive Sample ◽  
Classification Problems ◽  
Multiple Cancer ◽  
Multiple Tumor ◽  
One Class Classifier ◽  
The Given

In the computational biology community there are many biological cases that are considered as multi-one-class classification problems. Examples include the classification of multiple tumor types, protein fold recognition and the molecular classification of multiple cancer types. In all of these cases the real world appropriately characterized negative cases or outliers are impractical to achieve and the positive cases might consist of different clusters, which in turn might lead to accuracy degradation. In this paper we present a novel algorithm named MultiKOC multi-one-class classifiers based K-means to deal with this problem. The main idea is to execute a clustering algorithm over the positive samples to capture the hidden subdata of the given positive data, and then building up a one-class classifier for every cluster member’s examples separately: in other word, train the OC classifier on each piece of subdata. For a given new sample, the generated classifiers are applied. If it is rejected by all of those classifiers, the given sample is considered as a negative sample, otherwise it is a positive sample. The results of MultiKOC are compared with the traditional one-class, multi-one-class, ensemble one-classes and two-class methods, yielding a significant improvement over the one-class and like the two-class performance.

scholarly journals Exploiting heterogeneity in operational neural networks by synaptic plasticity

2021 ◽  
Author(s):  
Serkan Kiranyaz ◽  
Junaid Malik ◽  
Habib Ben Abdallah ◽  
Turker Ince ◽  
Alexandros Iosifidis ◽  
...  
Keyword(s):  
Neural Networks ◽  
Synaptic Plasticity ◽  
Network Model ◽  
Neuron Model ◽  
Linear Operators ◽  
Training Data ◽  
Learning Performance ◽  
Minimal Network ◽  
Hidden Layer ◽  
Hidden Neurons

AbstractThe recently proposed network model, Operational Neural Networks (ONNs), can generalize the conventional Convolutional Neural Networks (CNNs) that are homogenous only with a linear neuron model. As a heterogenous network model, ONNs are based on a generalized neuron model that can encapsulate any set of non-linear operators to boost diversity and to learn highly complex and multi-modal functions or spaces with minimal network complexity and training data. However, the default search method to find optimal operators in ONNs, the so-called Greedy Iterative Search (GIS) method, usually takes several training sessions to find a single operator set per layer. This is not only computationally demanding, also the network heterogeneity is limited since the same set of operators will then be used for all neurons in each layer. To address this deficiency and exploit a superior level of heterogeneity, in this study the focus is drawn on searching the best-possible operator set(s) for the hidden neurons of the network based on the “Synaptic Plasticity” paradigm that poses the essential learning theory in biological neurons. During training, each operator set in the library can be evaluated by their synaptic plasticity level, ranked from the worst to the best, and an “elite” ONN can then be configured using the top-ranked operator sets found at each hidden layer. Experimental results over highly challenging problems demonstrate that the elite ONNs even with few neurons and layers can achieve a superior learning performance than GIS-based ONNs and as a result, the performance gap over the CNNs further widens.

Universal Approximation Using Radial-Basis-Function Networks

1991 ◽  
Vol 3 (2) ◽  
pp. 246-257 ◽  
Author(s):  
J. Park ◽  
I. W. Sandberg
Keyword(s):  
Finite Number ◽  
Radial Basis Function ◽  
Basis Function ◽  
Universal Approximation ◽  
Radial Basis Function Networks ◽  
Feedforward Networks ◽  
Rbf Networks ◽  
Smoothing Factor ◽  
Radial Basis ◽  
Hidden Layer

There have been several recent studies concerning feedforward networks and the problem of approximating arbitrary functionals of a finite number of real variables. Some of these studies deal with cases in which the hidden-layer nonlinearity is not a sigmoid. This was motivated by successful applications of feedforward networks with nonsigmoidal hidden-layer units. This paper reports on a related study of radial-basis-function (RBF) networks, and it is proved that RBF networks having one hidden layer are capable of universal approximation. Here the emphasis is on the case of typical RBF networks, and the results show that a certain class of RBF networks with the same smoothing factor in each kernel node is broad enough for universal approximation.

Polynomial Representation of the Gaussian Process

2018 ◽  
Author(s):  
Jesper Kristensen ◽  
Isaac Asher ◽  
Liping Wang
Keyword(s):  
Gaussian Process ◽  
Basis Functions ◽  
Training Data ◽  
Computational Time ◽  
High Dimensional ◽  
High Dimensional Model Representation ◽  
Analysis Tool ◽  
Sobol Indices ◽  
Robust Fitting ◽  
Uncertainty Estimates

Gaussian Process (GP) regression is a well-established probabilistic meta-modeling and data analysis tool. The posterior distribution of the GP parameters can be estimated using, e.g., Markov Chain Monte Carlo (MCMC). The ability to make predictions is a key aspect of using such surrogate models. To make a GP prediction, the MCMC chain as well as the training data are required. For some applications, GP predictions can require too much computational time and/or memory, especially for many training data points. This motivates the present work to represent the GP in an equivalent polynomial (or other global functional) form called a portable GP. The portable GP inherits many benefits of the GP including feature ranking via Sobol indices, robust fitting to non-linear and high-dimensional data, accurate uncertainty estimates, etc. The framework expands the GP in a high-dimensional model representation (HDMR). After fitting each HDMR basis function with a polynomial, they are all added together to form the portable GP. A ranking of which basis functions to use in the fitting process is automatically provided via Sobol indices. The uncertainty from the fitting process can be propagated to the final GP polynomial estimate. In applications where speed and accuracy are paramount, spline fits to the basis functions give very good results. Finally, portable BHM provides an alternative set of assumptions with regards to extrapolation behavior which may be more appropriate than the assumptions inherent in GPs.

An Effective Solution to Regression Problem by RBF Neuron Network

2015 ◽  
Vol 6 (4) ◽  
pp. 57-74 ◽  
Author(s):  
Dang Thi Thu Hien ◽  
Hoang Xuan Huan ◽  
Le Xuan Minh Hoang
Keyword(s):  
Regression Function ◽  
Training Data ◽  
Neuron Network ◽  
Open Problems ◽  
Regression Problem ◽  
Rbf Network ◽  
Network Training ◽  
Hidden Layer ◽  
Definition Of ◽  
Selection Of

Radial Basis Function (RBF) neuron network is being applied widely in multivariate function regression. However, selection of neuron number for hidden layer and definition of suitable centre in order to produce a good regression network are still open problems which have been researched by many people. This article proposes to apply grid equally space nodes as the centre of hidden layer. Then, the authors use k-nearest neighbour method to define the value of regression function at the center and an interpolation RBF network training algorithm with equally spaced nodes to train the network. The experiments show the outstanding efficiency of regression function when the training data has Gauss white noise.

Sign in / Sign up

Export Citation Format

Share Document