Multistability of Recurrent Neural Networks With Piecewise-Linear Radial Basis Functions and State-Dependent Switching Parameters

2020 ◽  
Vol 50 (11) ◽  
pp. 4458-4471 ◽  
Author(s):  
Zhenyuan Guo ◽  
Linlin Liu ◽  
Jun Wang
Keyword(s):  
Neural Networks ◽  
Radial Basis Functions ◽  
Recurrent Neural Networks ◽  
Piecewise Linear ◽  
Basis Functions ◽  
State Dependent ◽  
Radial Basis

Regularization Theory and Neural Networks Architectures

1995 ◽  
Vol 7 (2) ◽  
pp. 219-269 ◽  
Author(s):  
Federico Girosi ◽  
Michael Jones ◽  
Tomaso Poggio
Keyword(s):  
Neural Networks ◽  
Tensor Product ◽  
Radial Basis Functions ◽  
Basis Functions ◽  
Additive Models ◽  
Approximation Schemes ◽  
Tensor Product Splines ◽  
Radial Basis ◽  
Regularization Networks ◽  
Hidden Layer

We had previously shown that regularization principles lead to approximation schemes that are equivalent to networks with one layer of hidden units, called regularization networks. In particular, standard smoothness functionals lead to a subclass of regularization networks, the well known radial basis functions approximation schemes. This paper shows that regularization networks encompass a much broader range of approximation schemes, including many of the popular general additive models and some of the neural networks. In particular, we introduce new classes of smoothness functionals that lead to different classes of basis functions. Additive splines as well as some tensor product splines can be obtained from appropriate classes of smoothness functionals. Furthermore, the same generalization that extends radial basis functions (RBF) to hyper basis functions (HBF) also leads from additive models to ridge approximation models, containing as special cases Breiman's hinge functions, some forms of projection pursuit regression, and several types of neural networks. We propose to use the term generalized regularization networks for this broad class of approximation schemes that follow from an extension of regularization. In the probabilistic interpretation of regularization, the different classes of basis functions correspond to different classes of prior probabilities on the approximating function spaces, and therefore to different types of smoothness assumptions. In summary, different multilayer networks with one hidden layer, which we collectively call generalized regularization networks, correspond to different classes of priors and associated smoothness functionals in a classical regularization principle. Three broad classes are (1) radial basis functions that can be generalized to hyper basis functions, (2) some tensor product splines, and (3) additive splines that can be generalized to schemes of the type of ridge approximation, hinge functions, and several perceptron-like neural networks with one hidden layer.

scholarly journals Feedback Analysis of Radial Basis Functions Neural Networks via Small Gain Theorem

2008 ◽  
Vol 41 (2) ◽  
pp. 7463-7467
Author(s):  
Ali S. Saad Azhar ◽  
Muhammad Shafiq ◽  
Jamil M. Bakhashwain ◽  
Fouad M. AL-Sunni
Keyword(s):  
Neural Networks ◽  
Radial Basis Functions ◽  
Basis Functions ◽  
Feedback Analysis ◽  
Small Gain Theorem ◽  
Radial Basis ◽  
Small Gain

On the Problem in Model Selection of Neural Network Regression in Overrealizable Scenario

2002 ◽  
Vol 14 (8) ◽  
pp. 1979-2002 ◽  
Author(s):  
Katsuyuki Hagiwara
Keyword(s):  
Neural Networks ◽  
Lower Bound ◽  
Model Selection ◽  
Radial Basis Functions ◽  
Selection Criterion ◽  
Basis Functions ◽  
Generalization Error ◽  
Radial Basis ◽  
Training Error ◽  
Selection Of

In considering a statistical model selection of neural networks and radial basis functions under an overrealizable case, the problem of unidentifiability emerges. Because the model selection criterion is an unbiased estimator of the generalization error based on the training error, this article analyzes the expected training error and the expected generalization error of neural networks and radial basis functions in overrealizable cases and clarifies the difference from regular models, for which identifiability holds. As a special case of an overrealizable scenario, we assumed a gaussian noise sequence as training data. In the least-squares estimation under this assumption, we first formulated the problem, in which the calculation of the expected errors of unidentifiable networks is reduced to the calculation of the expectation of the supremum of thex2 process. Under this formulation, we gave an upper bound of the expected training error and a lower bound of the expected generalization error, where the generalization is measured at a set of training inputs. Furthermore, we gave stochastic bounds on the training error and the generalization error. The obtained upper bound of the expected training error is smaller than in regular models, and the lower bound of the expected generalization error is larger than in regular models. The result tells us that the degree of overfitting in neural networks and radial basis functions is higher than in regular models. Correspondingly, it also tells us that the generalization capability is worse than in the case of regular models. The article may be enough to show a difference between neural networks and regular models in the context of the least-squares estimation in a simple situation. This is a first step in constructing a model selection criterion in an overrealizable case. Further important problems in this direction are also included in this article.

scholarly journals Use of Multiquadric Functions for Multivariable Representation of the Aerodynamic Coefficients of Airfoils

2021 ◽  
Vol 2021 ◽  
pp. 1-21
Author(s):  
Filipe Ribeiro ◽  
Pedro Albuquerque ◽  
Pedro Gamboa ◽  
Kouamana Bousson
Keyword(s):  
Radial Basis Functions ◽  
Piecewise Linear ◽  
Research Work ◽  
Computational Cost ◽  
Basis Functions ◽  
Aerodynamic Coefficients ◽  
Independent Variables ◽  
Advantages And Disadvantages ◽  
Radial Basis ◽  
Multivariable Interpolation

Given an array (or matrix) of values for a function of one or more variables, it is often desired to find a value between two given points. Multivariable interpolation and approximation by radial basis functions are important subjects in approximation theory that have many applications in Science and Engineering fields. During the last decades, radial basis functions (RBFs) have found increasingly widespread use for functional approximation of scattered data. This research work aims at benchmarking two different approaches: an approximation by radial basis functions and a piecewise linear multivariable interpolation in terms of their effectiveness and efficiency in order to conclude about the advantages and disadvantages of each approach in approximating the aerodynamic coefficients of airfoils. The main focus of this article is to study the main factors that affect the accuracy of the multiquadric functions, including the location and quantity of centers and the choice of the form factor. It also benchmarks them against piecewise linear multivariable interpolation regarding their precision throughout the selected domain and the computational cost required to accomplish a given amount of solutions associated with the aerodynamic coefficients of lift, drag and pitching moment. The approximation functions are applied to two different multidimensional cases: two independent variables, where the aerodynamic coefficients depend on the Reynolds number (Re) and the angle-of-attack (α), and four independent variables, where the aerodynamic coefficients depend on Re, α, flap chord ratio (cflap), and flap deflection (δflap).

Sign in / Sign up

Export Citation Format

Share Document