Feed-Forward Neural Networks Using Hermite Polynomial Activation Functions

2006 ◽  
pp. 323-333 ◽  
Author(s):  
Gerasimos G. Rigatos ◽  
Spyros G. Tzafestas
Keyword(s):  
Neural Networks ◽  
Hermite Polynomial ◽  
Activation Functions ◽  
Feed Forward ◽  
Feed Forward Neural Networks ◽  
Polynomial Activation Functions

scholarly journals FASTER TRAINING USING FUSION OF ACTIVATION FUNCTIONS FOR FEED FORWARD NEURAL NETWORKS

2009 ◽  
Vol 19 (06) ◽  
pp. 437-448 ◽  
Author(s):  
MD. ASADUZZAMAN ◽  
MD. SHAHJAHAN ◽  
KAZUYUKI MURASE
Keyword(s):  
Neural Networks ◽  
Time Series ◽  
Error Function ◽  
Time Series Prediction ◽  
Back Propagation ◽  
Activation Functions ◽  
Classification Problems ◽  
Feed Forward ◽  
Feed Forward Neural Networks ◽  
Prediction Problems

Multilayer feed-forward neural networks are widely used based on minimization of an error function. Back propagation (BP) is a famous training method used in the multilayer networks but it often suffers from the drawback of slow convergence. To make the learning faster, we propose 'Fusion of Activation Functions' (FAF) in which different conventional activation functions (AFs) are combined to compute final activation. This has not been studied extensively yet. One of the sub goals of the paper is to check the role of linear AFs in combination. We investigate whether FAF can enable the learning to be faster. Validity of the proposed method is examined by performing simulations on challenging nine real benchmark classification and time series prediction problems. The FAF has been applied to 2-bit, 3-bit and 4-bit parity, the breast cancer, Diabetes, Heart disease, Iris, wine, Glass and Soybean classification problems. The algorithm is also tested with Mackey-Glass chaotic time series prediction problem. The algorithm is shown to work better than other AFs used independently in BP such as sigmoid (SIG), arctangent (ATAN), logarithmic (LOG).

An Evolutionary Approach to the Use of Neural Networks in the Segmentation of Handwritten Numerals

1997 ◽  
Vol 11 (05) ◽  
pp. 717-734 ◽  
Author(s):  
J. M. Westall ◽  
M. S. Narasimha
Keyword(s):  
Neural Networks ◽  
Production System ◽  
Background Noise ◽  
Evolutionary Approach ◽  
Feed Forward ◽  
Feed Forward Neural Networks ◽  
Decision Functions

Neural networks are now widely and successfully used in the recognition of handwritten numerals. Despite their wide use in recognition, neural networks have not seen widespread use in segmentation. Segmentation can be extremely difficult in the presence of connected numerals, fragmented numerals, and background noise, and its failure is a principal cause of rejected and incorrectly read documents. Therefore, strategies leading to the successful application of neural technologies to segmentation are likely to yield important performance benefits. In this paper we identify problems that have impeded the use of neural networks in segmentation and describe an evolutionary approach to applying neural networks in segmentation. Our approach, based upon the use of monotonic fuzzy valued decision functions computed by feed-forward neural networks, has been successfully employed in a production system.

Sign in / Sign up

Export Citation Format

Share Document