Universal approximation capability of EBF neural networks with arbitrary activation functions

1996 ◽  
Vol 15 (5) ◽  
pp. 671-683 ◽  
Author(s):  
Tianping Chen ◽  
Hong Chen
Keyword(s):  
Neural Networks ◽  
Universal Approximation ◽  
Activation Functions ◽  
Approximation Capability

Theoretical Analyses of the Universal Approximation Capability of a Class of Higher Order Neural Networks Based on Approximate Identity

2016 ◽  
pp. 1456-1470 ◽  
Author(s):  
Saeed Panahian Fard ◽  
Zarita Zainuddin
Keyword(s):  
Neural Networks ◽  
Approximation Theory ◽  
Special Class ◽  
Higher Order ◽  
Approximate Identity ◽  
Universal Approximation ◽  
Multivariate Functions ◽  
Higher Order Neural Networks ◽  
Approximation Capability ◽  
Theoretical Analyses

One of the most important problems in the theory of approximation functions by means of neural networks is universal approximation capability of neural networks. In this study, we investigate the theoretical analyses of the universal approximation capability of a special class of three layer feedforward higher order neural networks based on the concept of approximate identity in the space of continuous multivariate functions. Moreover, we present theoretical analyses of the universal approximation capability of the networks in the spaces of Lebesgue integrable multivariate functions. The methods used in proving our results are based on the concepts of convolution and epsilon-net. The obtained results can be seen as an attempt towards the development of approximation theory by means of neural networks.

scholarly journals Interval universal approximation for neural networks

2022 ◽  
Vol 6 (POPL) ◽  
pp. 1-29
Author(s):  
Zi Wang ◽  
Aws Albarghouthi ◽  
Gautam Prakriya ◽  
Somesh Jha
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Approximation Problem ◽  
Activation Function ◽  
Constructive Proof ◽  
Universal Approximation ◽  
Activation Functions ◽  
Complete Problems ◽  
Robust Network ◽  
Interval Approximation

To verify safety and robustness of neural networks, researchers have successfully applied abstract interpretation , primarily using the interval abstract domain. In this paper, we study the theoretical power and limits of the interval domain for neural-network verification. First, we introduce the interval universal approximation (IUA) theorem. IUA shows that neural networks not only can approximate any continuous function f (universal approximation) as we have known for decades, but we can find a neural network, using any well-behaved activation function, whose interval bounds are an arbitrarily close approximation of the set semantics of f (the result of applying f to a set of inputs). We call this notion of approximation interval approximation . Our theorem generalizes the recent result of Baader et al. from ReLUs to a rich class of activation functions that we call squashable functions . Additionally, the IUA theorem implies that we can always construct provably robust neural networks under ℓ ∞ -norm using almost any practical activation function. Second, we study the computational complexity of constructing neural networks that are amenable to precise interval analysis. This is a crucial question, as our constructive proof of IUA is exponential in the size of the approximation domain. We boil this question down to the problem of approximating the range of a neural network with squashable activation functions. We show that the range approximation problem (RA) is a Δ 2 -intermediate problem, which is strictly harder than NP -complete problems, assuming coNP ⊄ NP . As a result, IUA is an inherently hard problem : No matter what abstract domain or computational tools we consider to achieve interval approximation, there is no efficient construction of such a universal approximator. This implies that it is hard to construct a provably robust network, even if we have a robust network to start with.

Theoretical Analyses of the Universal Approximation Capability of a class of Higher Order Neural Networks based on Approximate Identity

2016 ◽  
pp. 192-207
Author(s):  
Saeed Panahian ◽  
Zarita Zainuddin
Keyword(s):  
Neural Networks ◽  
Approximation Theory ◽  
Special Class ◽  
Higher Order ◽  
Approximate Identity ◽  
Universal Approximation ◽  
Multivariate Functions ◽  
Higher Order Neural Networks ◽  
Approximation Capability ◽  
Theoretical Analyses

One of the most important problems in the theory of approximation functions by means of neural networks is universal approximation capability of neural networks. In this study, we investigate the theoretical analyses of the universal approximation capability of a special class of three layer feedforward higher order neural networks based on the concept of approximate identity in the space of continuous multivariate functions. Moreover, we present theoretical analyses of the universal approximation capability of the networks in the spaces of Lebesgue integrable multivariate functions. The methods used in proving our results are based on the concepts of convolution and epsilon-net. The obtained results can be seen as an attempt towards the development of approximation theory by means of neural networks.

scholarly journals Universal Approximation Theorem for Tessarine-Valued Neural Networks

2021 ◽  
Author(s):  
Rafael A. F. Carniello ◽  
Wington L. Vital ◽  
Marcos Eduardo Valle
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Approximation Theorem ◽  
Superior Performance ◽  
Universal Approximation ◽  
Activation Functions ◽  
Hidden Layer ◽  
Arbitrary Precision ◽  
Vector Valued Function ◽  
Vector Valued

The universal approximation theorem ensures that any continuous real-valued function defined on a compact subset can be approximated with arbitrary precision by a single hidden layer neural network. In this paper, we show that the universal approximation theorem also holds for tessarine-valued neural networks. Precisely, any continuous tessarine-valued function can be approximated with arbitrary precision by a single hidden layer tessarine-valued neural network with split activation functions in the hidden layer. A simple numerical example, confirming the theoretical result and revealing the superior performance of a tessarine-valued neural network over a real-valued model for interpolating a vector-valued function, is presented in the paper.

scholarly journals Estimation of Approximating Rate for Neural Network inLwpSpaces

2012 ◽  
Vol 2012 ◽  
pp. 1-8
Author(s):  
Jian-Jun Wang ◽  
Chan-Yun Yang ◽  
Jia Jing
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Degree Of Approximation ◽  
Activation Functions ◽  
Feedforward Network ◽  
Topology Construction ◽  
And Topology ◽  
The Neural Network ◽  
Hidden Layer ◽  
Approximation Capability

A class of Soblove type multivariate function is approximated by feedforward network with one hidden layer of sigmoidal units and a linear output. By adopting a set of orthogonal polynomial basis and under certain assumptions for the governing activation functions of the neural network, the upper bound on the degree of approximation can be obtained for the class of Soblove functions. The results obtained are helpful in understanding the approximation capability and topology construction of the sigmoidal neural networks.

Sign in / Sign up

Export Citation Format

Share Document