A Universal Approximation Theorem for Gaussian-Gated Mixture of Experts Models

2017 ◽  
Author(s):  
Hien Nguyen
Keyword(s):  
Approximation Theorem ◽  
Universal Approximation ◽  
Mixture Of Experts

scholarly journals A Universal Approximation Theorem for Mixture-of-Experts Models

2016 ◽  
Vol 28 (12) ◽  
pp. 2585-2593 ◽  
Author(s):  
Hien D. Nguyen ◽  
Luke R. Lloyd-Jones ◽  
Geoffrey J. McLachlan
Keyword(s):  
Network Architecture ◽  
Approximation Theorem ◽  
Target Function ◽  
Continuous Functions ◽  
Universal Approximation ◽  
Mixture Of Experts ◽  
Neural Network Architecture ◽  
Alternative Result ◽  
Unit Hypercube ◽  
Uniformly Convergent

The mixture-of-experts (MoE) model is a popular neural network architecture for nonlinear regression and classification. The class of MoE mean functions is known to be uniformly convergent to any unknown target function, assuming that the target function is from a Sobolev space that is sufficiently differentiable and that the domain of estimation is a compact unit hypercube. We provide an alternative result, which shows that the class of MoE mean functions is dense in the class of all continuous functions over arbitrary compact domains of estimation. Our result can be viewed as a universal approximation theorem for MoE models. The theorem we present allows MoE users to be confident in applying such models for estimation when data arise from nonlinear and nondifferentiable generative processes.

scholarly journals Universal approximation theorem for Dirichlet series

2006 ◽  
Vol 2006 ◽  
pp. 1-11 ◽  
Author(s):  
O. Demanze ◽  
A. Mouze
Keyword(s):  
Dirichlet Series ◽  
Simultaneous Approximation ◽  
Shift Operator ◽  
Approximation Theorem ◽  
Extension Theorem ◽  
Universal Approximation ◽  
Derivation Operator ◽  
Backward Shift ◽  
The Right ◽  
Density Results

The paper deals with an extension theorem by Costakis and Vlachou on simultaneous approximation for holomorphic function to the setting of Dirichlet series, which are absolutely convergent in the right half of the complex plane. The derivation operator used in the analytic case is substituted by a weighted backward shift operator in the Dirichlet case. We show the similarities and extensions in comparing both results. Several density results are proved that finally lead to the main theorem on simultaneous approximation.

scholarly journals Learning nonlinear operators via DeepONet based on the universal approximation theorem of operators

2021 ◽  
Vol 3 (3) ◽  
pp. 218-229 ◽  
Author(s):  
Lu Lu ◽  
Pengzhan Jin ◽  
Guofei Pang ◽  
Zhongqiang Zhang ◽  
George Em Karniadakis
Keyword(s):  
Approximation Theorem ◽  
Universal Approximation ◽  
Nonlinear Operators

scholarly journals Universal Approximation Theorem for Tessarine-Valued Neural Networks

2021 ◽  
Author(s):  
Rafael A. F. Carniello ◽  
Wington L. Vital ◽  
Marcos Eduardo Valle
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Approximation Theorem ◽  
Superior Performance ◽  
Universal Approximation ◽  
Activation Functions ◽  
Hidden Layer ◽  
Arbitrary Precision ◽  
Vector Valued Function ◽  
Vector Valued

The universal approximation theorem ensures that any continuous real-valued function defined on a compact subset can be approximated with arbitrary precision by a single hidden layer neural network. In this paper, we show that the universal approximation theorem also holds for tessarine-valued neural networks. Precisely, any continuous tessarine-valued function can be approximated with arbitrary precision by a single hidden layer tessarine-valued neural network with split activation functions in the hidden layer. A simple numerical example, confirming the theoretical result and revealing the superior performance of a tessarine-valued neural network over a real-valued model for interpolating a vector-valued function, is presented in the paper.

scholarly journals Implicit Non-linear Similarity Scoring for Recognizing Unseen Classes

2018 ◽  
Author(s):  
Yuchen Guo ◽  
Guiguang Ding ◽  
Jungong Han ◽  
Sicheng Zhao ◽  
Bin Wang
Keyword(s):  
Real World ◽  
Approximation Theorem ◽  
Important Task ◽  
Inner Product ◽  
Similarity Function ◽  
Universal Approximation ◽  
Deep Networks ◽  
Real World Applications ◽  
Non Linear ◽  
Shallow And Deep Networks

Recognizing unseen classes is an important task for real-world applications, due to: 1) it is common that some classes in reality have no labeled image exemplar for training; and 2) novel classes emerge rapidly. Recently, to address this task many zero-shot learning (ZSL) approaches have been proposed where explicit linear scores, like inner product score, are employed to measure the similarity between a class and an image. We argue that explicit linear scoring (ELS) seems too weak to capture complicated image-class correspondence. We propose a simple yet effective framework, called Implicit Non-linear Similarity Scoring (ICINESS). In particular, we train a scoring network which uses image and class features as input, fuses them by hidden layers, and outputs the similarity. Based on the universal approximation theorem, it can approximate the true similarity function between images and classes if a proper structure is used in an implicit non-linear way, which is more flexible and powerful. With ICINESS framework, we implement ZSL algorithms by shallow and deep networks, which yield consistently superior results.

Sign in / Sign up

Export Citation Format

Share Document