scholarly journals Spherical Minimum Description Length

Entropy ◽  
2018 ◽  
Vol 20 (8) ◽  
pp. 575
Author(s):  
Trevor Herntier ◽  
Koffi Ihou ◽  
Anthony Smith ◽  
Anand Rangarajan ◽  
Adrian Peter
Keyword(s):  
Model Selection ◽  
Parameter Space ◽  
Selection Criteria ◽  
Goodness Of Fit ◽  
Minimum Description Length ◽  
Current Model ◽  
Selection Criterion ◽  
Laplace Approximation ◽  
Model Complexity ◽  
Model Selection Criteria

We consider the problem of model selection using the Minimum Description Length (MDL) criterion for distributions with parameters on the hypersphere. Model selection algorithms aim to find a compromise between goodness of fit and model complexity. Variables often considered for complexity penalties involve number of parameters, sample size and shape of the parameter space, with the penalty term often referred to as stochastic complexity. Current model selection criteria either ignore the shape of the parameter space or incorrectly penalize the complexity of the model, largely because typical Laplace approximation techniques yield inaccurate results for curved spaces. We demonstrate how the use of a constrained Laplace approximation on the hypersphere yields a novel complexity measure that more accurately reflects the geometry of these spherical parameters spaces. We refer to this modified model selection criterion as spherical MDL. As proof of concept, spherical MDL is used for bin selection in histogram density estimation, performing favorably against other model selection criteria.

ON DIFFERENT MODEL SELECTION CRITERIA IN A FORWARD AND BACKWARD REGRESSION HYBRID NETWORK

2004 ◽  
Vol 18 (05) ◽  
pp. 847-865 ◽  
Author(s):  
SHIMON COHEN ◽  
NATHAN INTRATOR
Keyword(s):  
Model Selection ◽  
Selection Criteria ◽  
Minimum Description Length ◽  
Selection Criterion ◽  
Network Size ◽  
Hybrid Network ◽  
Data Sets ◽  
Ratio Test ◽  
Selection Algorithm ◽  
Model Selection Criteria

An assessment of the performance of a hybrid network with different model selection criteria is considered. These criteria are used in an automatic model selection algorithm for constructing a hybrid network of radial and perceptron hidden units for regression. A forward step builds the full hybrid network; a model selection criterion is used for controlling the network-size and another criterion is used for choosing the appropriate hidden unit for different regions of input space. This is followed by a conservative pruning step using Likelihood Ratio Test, Bayesian or Minimum Description Length, which leads to robust estimators with low variance. The result is a small architecture that performs well on difficult, realistic, benchmark data-sets of high dimensionality and small training size. Best results are obtained by using the Bayesian approach for the model selection.

scholarly journals Robust Model Selection Criteria Based on Pseudodistances

Entropy ◽  
2020 ◽  
Vol 22 (3) ◽  
pp. 304 ◽  
Author(s):  
Aida Toma ◽  
Alex Karagrigoriou ◽  
Paschalini Trentou
Keyword(s):  
Model Selection ◽  
Regression Models ◽  
Selection Criteria ◽  
Selection Criterion ◽  
Superior Performance ◽  
Linear Regression Models ◽  
Limit Laws ◽  
Model Selection Criteria ◽  
Robust Model ◽  
Robust Model Selection

In this paper, we introduce a new class of robust model selection criteria. These criteria are defined by estimators of the expected overall discrepancy using pseudodistances and the minimum pseudodistance principle. Theoretical properties of these criteria are proved, namely asymptotic unbiasedness, robustness, consistency, as well as the limit laws. The case of the linear regression models is studied and a specific pseudodistance based criterion is proposed. Monte Carlo simulations and applications for real data are presented in order to exemplify the performance of the new methodology. These examples show that the new selection criterion for regression models is a good competitor of some well known criteria and may have superior performance, especially in the case of small and contaminated samples.

scholarly journals Attractor Landscapes as a Model Selection Criterion in Data Poor Environments

2021 ◽  
Author(s):  
Cole A Lyman ◽  
Spencer Richman ◽  
Matthew C. Morris ◽  
Hongbao Cao ◽  
Antony Scerri ◽  
...  
Keyword(s):  
Model Selection ◽  
Selection Criteria ◽  
Model Identification ◽  
Selection Criterion ◽  
Optimality Criteria ◽  
Model Complexity ◽  
P Value ◽  
Response Dynamics ◽  
The Impact ◽  
Novel Model

Modeling of systems for which data is limited often leads to underdetermined model identification problems, where multiple candidate models are equally adherent to data. In such situations additional optimality criteria are useful in model selection apart from the conventional minimization of error and model complexity. This work presents the attractor landscape as a domain for novel model selection criteria, where the number and location of attractors impact desirability. A set of candidate models describing immune response dynamics to SARS-CoV infection is used as an example for model selection based on features of the attractor landscape. Using this selection criteria, the initial set of 18 models is ranked and reduced to 7 models that have a composite objective value with a p-value < 0.05. Additionally, the impact of pharmacologically induced remolding of the attractor landscape is presented.

Sign in / Sign up

Export Citation Format

Share Document