scholarly journals Optimize TSK Fuzzy Systems for Classification Problems: Mini-Batch Gradient Descent with Uniform Regularization and Batch Normalization

2020 ◽  
pp. 1-1 ◽  
Author(s):  
Yuqi Cui ◽  
Dongrui Wu ◽  
Jian Huang
Keyword(s):  
Fuzzy Systems ◽  
Gradient Descent ◽  
Classification Problems ◽  
Batch Normalization ◽  
Uniform Regularization

scholarly journals Weight Optimization in Recurrent Neural Networks with Hybrid Metaheuristic Cuckoo Search Techniques for Data Classification

2015 ◽  
Vol 2015 ◽  
pp. 1-12 ◽  
Author(s):  
Nazri Mohd Nawi ◽  
Abdullah Khan ◽  
M. Z. Rehman ◽  
Haruna Chiroma ◽  
Tutut Herawan
Keyword(s):  
Gradient Descent ◽  
Search Algorithm ◽  
Error Function ◽  
Back Propagation ◽  
Data Classification ◽  
Cuckoo Search ◽  
Recurrent Network ◽  
Training Algorithms ◽  
Local Minima ◽  
Classification Problems

Recurrent neural network (RNN) has been widely used as a tool in the data classification. This network can be educated with gradient descent back propagation. However, traditional training algorithms have some drawbacks such as slow speed of convergence being not definite to find the global minimum of the error function since gradient descent may get stuck in local minima. As a solution, nature inspired metaheuristic algorithms provide derivative-free solution to optimize complex problems. This paper proposes a new metaheuristic search algorithm called Cuckoo Search (CS) based on Cuckoo bird’s behavior to train Elman recurrent network (ERN) and back propagation Elman recurrent network (BPERN) in achieving fast convergence rate and to avoid local minima problem. The proposed CSERN and CSBPERN algorithms are compared with artificial bee colony using BP algorithm and other hybrid variants algorithms. Specifically, some selected benchmark classification problems are used. The simulation results show that the computational efficiency of ERN and BPERN training process is highly enhanced when coupled with the proposed hybrid method.

An approach for non-singleton generalized Type-2 fuzzy classifiers

2020 ◽  
Vol 39 (5) ◽  
pp. 7203-7215
Author(s):  
Emanuel Ontiveros-Robles ◽  
Oscar Castillo ◽  
Patricia Melin
Keyword(s):  
Fuzzy Systems ◽  
General Type ◽  
Fuzzy Inference ◽  
Classification Problems ◽  
Fuzzy Inference Systems ◽  
Classification Rate ◽  
Inference System ◽  
Fuzzy Classifiers ◽  
Inference Systems

In recent years, successful applications of singleton fuzzy inference systems have been made in a plethora of different kinds of problems, for example in the areas of control, digital image processing, time series prediction, fault detection and classification. However, there exists another relatively less explored approach, which is the use of non-singleton fuzzy inference systems. This approach offers an interesting way for handling uncertainty in complex problems by considering inputs with uncertainty, while the conventional Fuzzy Systems have their inputs with crisp values (singleton systems). Non-singleton systems have as inputs Type-1 membership functions, and this difference increases the complexity of the fuzzification, but provides the systems with additional non-linearities and robustness. The main limitations of using a non-singleton fuzzy inference system is that it requires an additional computational overhead and are usually more difficult to apply in some problems. Based on these limitations, we propose in this work an approach for efficiently processing non-singleton fuzzy systems. To verify the advantages of the proposed approach we consider the case of general type-2 fuzzy systems with non-singleton inputs and their application in the classification area. The main contribution of the paper is the implementation of non-singleton General Type-2 Fuzzy Inference Systems for the classification task, aiming at analyzing its potential advantage in classification problems. In the present paper we propose that the use of non-singleton inputs in Type-2 Fuzzy Classifiers can improve the classification rate and based on the realized experiments we can observe that General Type-2 Fuzzy Classifiers, but with non-singleton fuzzification, obtain better results in comparison with respect to their singleton counterparts.

scholarly journals Design of a hierarchical fuzzy model predictive controller

2015 ◽  
Vol 4 (2) ◽  
pp. 342 ◽  
Author(s):  
Zeinab Fallah ◽  
Mojtaba Ahmadieh Khanesar ◽  
Mohammad Teshnehlab
Keyword(s):  
Nonlinear Model ◽  
Fuzzy Systems ◽  
Gradient Descent ◽  
Fuzzy Controller ◽  
Fuzzy Model ◽  
Least Square ◽  
Continuous Stirred Tank Reactor ◽  
Descent Algorithm ◽  
Neuro Fuzzy ◽  
Gradient Descent Algorithm

In order to control a nonlinear system using Nonlinear Model Predictive Control (NMPC), a nonlinear model from system is required. In this paper, a hierarchical neuro-fuzzy model is used for nonlinear identification of the plant. The use of hierarchical neuro-fuzzy systems makes it possible to overcome the curse of dimensionality. In neuro-fuzzy systems, if the input number increases, then the number of rules increases exponentially. One solution to this problem is making use of Hierarchical Fuzzy System Mamdani (HFS) in which the number of the rules increases linearly. Gradient descent and recursive least square algorithm are used simultaneously to train the parameters of the HFS. Gradient Descent Algorithm is utilized to train the parameters, which appear nonlinearly in the output of HFS, and RLS is used to train the parameters of consequent the part, which appears linearly in the output of HFS. Finally, a model predictive fuzzy controller based on a predictive cost function is proposed. Using Gradient Descent Algorithm, the parameters of the controller are optimized. The proposed controller is simulated on the control of continuous stirred tank reactor. It is shown that the proposed method can control the system with high performance.

STUDYING THE EFFECT OF ADAPTIVE MOMENTUM IN IMPROVING THE ACCURACY OF GRADIENT DESCENT BACK PROPAGATION ALGORITHM ON CLASSIFICATION PROBLEMS

2012 ◽  
Vol 09 ◽  
pp. 432-439 ◽  
Author(s):  
MUHAMMAD ZUBAIR REHMAN ◽  
NAZRI MOHD. NAWI
Keyword(s):  
Gradient Descent ◽  
Back Propagation ◽  
Activation Function ◽  
Descent Method ◽  
Learning Rate ◽  
Gradient Descent Method ◽  
Back Propagation Algorithm ◽  
Classification Problems ◽  
Propagation Algorithm ◽  
Adaptive Momentum

Despite being widely used in the practical problems around the world, Gradient Descent Back-propagation algorithm comes with problems like slow convergence and convergence to local minima. Previous researchers have suggested certain modifications to improve the convergence in gradient Descent Back-propagation algorithm such as careful selection of input weights and biases, learning rate, momentum, network topology, activation function and value for 'gain' in the activation function. This research proposed an algorithm for improving the working performance of back-propagation algorithm which is 'Gradient Descent with Adaptive Momentum (GDAM)' by keeping the gain value fixed during all network trials. The performance of GDAM is compared with 'Gradient Descent with fixed Momentum (GDM)' and 'Gradient Descent Method with Adaptive Gain (GDM-AG)'. The learning rate is fixed to 0.4 and maximum epochs are set to 3000 while sigmoid activation function is used for the experimentation. The results show that GDAM is a better approach than previous methods with an accuracy ratio of 1.0 for classification problems like Wine Quality, Mushroom and Thyroid disease.

scholarly journals Generating Interpretable Fuzzy Systems for Classification Problems

TecnoLógicas ◽  
2009 ◽  
pp. 239
Author(s):  
Juan A. Contreras-Montes ◽  
Oscar S. Acuña-Camacho
Keyword(s):  
Breast Cancer ◽  
Fuzzy Systems ◽  
Data Classification ◽  
Classification Problem ◽  
Classification Problems ◽  
Iris Data

En este artículo se presenta un nuevo método para generar sistemas difusos interpretables, a partir de datos experimentalesde entrada y salida, para resolver problemas de clasificación. En la partición antecedente se emplean conjuntos triangulares con interpolación de 0.5 lo cual evita la presencia de solapamientos complejos que suceden en otros métodos. Los consecuentes, tipo Singleton, son generados por la proyección de los valores modales de cada función de membresía triangular en el espacio de salida y se emplea el método de mínimos cuadrados para el ajuste de los consecuentes. El método propuesto consigue una mayor precisión que la alcanzada con los métodos actuales existentes, empleando un número reducido de reglas y parámetros y sin sacrificar la interpretabilidad del modelo difuso. El enfoque propuesto es aplicado a dos problemas clásicos de clasificación: el Wisconsin Breast Cancer (WBC) y el Iris Data Classification Problem, para mostrar las ventajas del método y comparar los resultados con los alcanzados por otros investigadores.

scholarly journals Deep Neural Networks with Multistate Activation Functions

2015 ◽  
Vol 2015 ◽  
pp. 1-10 ◽  
Author(s):  
Chenghao Cai ◽  
Yanyan Xu ◽  
Dengfeng Ke ◽  
Kaile Su
Keyword(s):  
Neural Networks ◽  
Gradient Descent ◽  
Deep Neural Networks ◽  
Error Rates ◽  
Stochastic Gradient Descent ◽  
Activation Functions ◽  
Classification Problems ◽  
Training Set ◽  
Relative Improvement ◽  
Better Than

We propose multistate activation functions (MSAFs) for deep neural networks (DNNs). These MSAFs are new kinds of activation functions which are capable of representing more than two states, including theN-order MSAFs and the symmetrical MSAF. DNNs with these MSAFs can be trained via conventional Stochastic Gradient Descent (SGD) as well as mean-normalised SGD. We also discuss how these MSAFs perform when used to resolve classification problems. Experimental results on the TIMIT corpus reveal that, on speech recognition tasks, DNNs with MSAFs perform better than the conventional DNNs, getting a relative improvement of 5.60% on phoneme error rates. Further experiments also reveal that mean-normalised SGD facilitates the training processes of DNNs with MSAFs, especially when being with large training sets. The models can also be directly trained without pretraining when the training set is sufficiently large, which results in a considerable relative improvement of 5.82% on word error rates.

Sign in / Sign up

Export Citation Format

Share Document