Computational Improvements to Estimating Kriging Metamodel Parameters

2009 ◽  
Vol 131 (8) ◽  
Author(s):  
Jay D. Martin
Keyword(s):  
Response Surface ◽  
Spatial Correlation ◽  
Likelihood Estimation ◽  
Kriging Model ◽  
Model Parameters ◽  
Computationally Efficient ◽  
Spatial Correlation Function ◽  
Computational Burden ◽  
Kriging Metamodel ◽  
Gradient Based

The details of a method to reduce the computational burden experienced while estimating the optimal model parameters for a Kriging model are presented. A Kriging model is a type of surrogate model that can be used to create a response surface based a set of observations of a computationally expensive system design analysis. This Kriging model can then be used as a computationally efficient surrogate to the original model, providing the opportunity for the rapid exploration of the resulting tradespace. The Kriging model can provide a more complex response surface than the more traditional linear regression response surface through the introduction of a few terms to quantify the spatial correlation of the observations. Implementation details and enhancements to gradient-based methods to estimate the model parameters are presented. It concludes with a comparison of these enhancements to using maximum likelihood estimation to estimate Kriging model parameters and their potential reduction in computational burden. These enhancements include the development of the analytic gradient and Hessian for the log-likelihood equation of a Kriging model that uses a Gaussian spatial correlation function. The suggested algorithm is similar to the SCORING algorithm traditionally used in statistics.

Using Maximum Likelihood Estimation to Estimate Kriging Model Parameters

2007 ◽  
Author(s):  
Jay D. Martin
Keyword(s):  
Maximum Likelihood ◽  
Maximum Likelihood Estimation ◽  
Likelihood Function ◽  
Likelihood Estimation ◽  
Kriging Model ◽  
Model Parameters ◽  
Spatial Correlation Function ◽  
Log Likelihood ◽  
Gradient Based ◽  
Computationally Expensive

A kriging model can be used as a surrogate to a more computationally expensive model or simulation. It is capable of providing a continuous mathematical relationship that can interpolate a set of observations. One of the major issues with using kriging models is the potentially computationally expensive process of estimating the best model parameters. One of the most common methods used to estimate model parameters is Maximum Likelihood Estimation (MLE). MLE of kriging model parameters requires the use of numerical optimization of a continuous but possibly multi-modal log-likelihood function. This paper presents some enhancements to gradient-based methods to make them more computationally efficient and compares the potential reduction in computational burden. These enhancements include the development of the analytic gradient and Hessian for the log-likelihood equation of a kriging model that uses a Gaussian spatial correlation function. The suggested algorithm is very similar to the Scoring algorithm traditionally used in statistics, a Newton-Raphson gradient-based optimization method.

scholarly journals Joint Maximum Likelihood of Phylogeny and Ancestral States Is Not Consistent

2019 ◽  
Vol 36 (10) ◽  
pp. 2352-2357
Author(s):  
David A Shaw ◽  
Vu C Dinh ◽  
Frederick A Matsen
Keyword(s):  
Maximum Likelihood ◽  
Maximum Likelihood Estimation ◽  
Likelihood Estimation ◽  
Branch Length ◽  
Continuous Model ◽  
Model Parameters ◽  
Infinite Length ◽  
Computationally Efficient ◽  
Ancestral States ◽  
Joint Method

Abstract Maximum likelihood estimation in phylogenetics requires a means of handling unknown ancestral states. Classical maximum likelihood averages over these unknown intermediate states, leading to provably consistent estimation of the topology and continuous model parameters. Recently, a computationally efficient approach has been proposed to jointly maximize over these unknown states and phylogenetic parameters. Although this method of joint maximum likelihood estimation can obtain estimates more quickly, its properties as an estimator are not yet clear. In this article, we show that this method of jointly estimating phylogenetic parameters along with ancestral states is not consistent in general. We find a sizeable region of parameter space that generates data on a four-taxon tree for which this joint method estimates the internal branch length to be exactly zero, even in the limit of infinite-length sequences. More generally, we show that this joint method only estimates branch lengths correctly on a set of measure zero. We show empirically that branch length estimates are systematically biased downward, even for short branches.

Adjoint- and Hybrid-Based Hessians for Optimization Problems in System Identification

2018 ◽  
Vol 140 (10) ◽  
Author(s):  
Souransu Nandi ◽  
Tarunraj Singh
Keyword(s):  
System Identification ◽  
Optimization Problems ◽  
Initial Conditions ◽  
Synthetic Data ◽  
Model Parameters ◽  
Computationally Efficient ◽  
Step Size ◽  
Adjoint Sensitivity ◽  
Gradient Based ◽  
Discrete Measurement

An adjoint sensitivity-based approach to determine the gradient and Hessian of cost functions for system identification of dynamical systems is presented. The motivation is the development of a computationally efficient approach relative to the direct differentiation (DD) technique and which overcomes the challenges of the step-size selection in finite difference (FD) approaches. An optimization framework is used to determine the parameters of a dynamical system which minimizes a summation of a scalar cost function evaluated at the discrete measurement instants. The discrete time measurements result in discontinuities in the Lagrange multipliers. Two approaches labeled as the Adjoint and the Hybrid are developed for the calculation of the gradient and Hessian for gradient-based optimization algorithms. The proposed approach is illustrated on the Lorenz 63 model where part of the initial conditions and model parameters are estimated using synthetic data. Examples of identifying model parameters of light curves of type 1a supernovae and a two-tank dynamic model using publicly available data are also included.

On the Use of Kriging Models to Approximate Deterministic Computer Models

2004 ◽  
Author(s):  
Jay D. Martin ◽  
Timothy W. Simpson
Keyword(s):  
Likelihood Estimation ◽  
Information Criterion ◽  
Computer Models ◽  
Kriging Model ◽  
Estimation Methods ◽  
Test Problems ◽  
Computationally Efficient ◽  
Design And Optimization ◽  
Parameter Estimation Methods ◽  
Computationally Efficient Algorithms

The use of kriging models for approximation and metamodel-based design and optimization has been steadily on the rise in the past decade. The widespread usage of kriging models appears to be hampered by (1) the lack of guidance in selecting the appropriate form of the kriging model, (2) computationally efficient algorithms for estimating the model’s parameters, and (3) an effective method to assess the resulting model’s quality. In this paper, we compare (1) Maximum Likelihood Estimation (MLE) and Cross-Validation (CV) parameter estimation methods for selecting a kriging model’s parameters given its form and (2) and an R2 of prediction and the corrected Akaike Information Criterion for assessing the quality of the created kriging model, permitting the comparison of different forms of a kriging model. These methods are demonstrated with six test problems. Finally, different forms of kriging models are examined to determine if more complex forms are more accurate and easier to fit than simple forms of kriging models for approximating computer models.

Nearly Exact Bayesian Estimation of Non-linear No-Arbitrage Term-Structure Models*

2021 ◽  
Author(s):  
Marcello Pericoli ◽  
Marco Taboga
Keyword(s):  
Bayesian Estimation ◽  
Term Structure ◽  
Broad Class ◽  
Model Parameters ◽  
State Variables ◽  
Computationally Efficient ◽  
Macroeconomic Factors ◽  
No Arbitrage ◽  
Term Structure Models ◽  
Non Linear

Abstract We propose a general method for the Bayesian estimation of a very broad class of non-linear no-arbitrage term-structure models. The main innovation we introduce is a computationally efficient method, based on deep learning techniques, for approximating no-arbitrage model-implied bond yields to any desired degree of accuracy. Once the pricing function is approximated, the posterior distribution of model parameters and unobservable state variables can be estimated by standard Markov Chain Monte Carlo methods. As an illustrative example, we apply the proposed techniques to the estimation of a shadow-rate model with a time-varying lower bound and unspanned macroeconomic factors.

scholarly journals Determination of the Optimal Neural Network Transfer Function for Response Surface Methodology and Robust Design

2021 ◽  
Vol 11 (15) ◽  
pp. 6768
Author(s):  
Tuan-Ho Le ◽  
Hyeonae Jang ◽  
Sangmun Shin
Keyword(s):  
Response Surface Methodology ◽  
Transfer Function ◽  
Response Surface ◽  
Robust Design ◽  
Transfer Functions ◽  
Desirability Function ◽  
Likelihood Estimation ◽  
Statistical Simulation ◽  
Screening Procedure ◽  
Function Estimation

Response surface methodology (RSM) has been widely recognized as an essential estimation tool in many robust design studies investigating the second-order polynomial functional relationship between the responses of interest and their associated input variables. However, there is scope for improvement in the flexibility of estimation models and the accuracy of their results. Although many NN-based estimations and optimization approaches have been reported in the literature, a closed functional form is not readily available. To address this limitation, a maximum-likelihood estimation approach for an NN-based response function estimation (NRFE) is used to obtain the functional forms of the process mean and standard deviation. While the estimation results of most existing NN-based approaches depend primarily on their transfer functions, this approach often requires a screening procedure for various transfer functions. In this study, the proposed NRFE identifies a new screening procedure to obtain the best transfer function in an NN structure using a desirability function family while determining its associated weight parameters. A statistical simulation was performed to evaluate the efficiency of the proposed NRFE method. In this particular simulation, the proposed NRFE method provided significantly better results than conventional RSM. Finally, a numerical example is used for validating the proposed method.

scholarly journals Evaluating the Observed Log-Likelihood Function in Two-Level Structural Equation Modeling with Missing Data: From Formulas to R Code

Psych ◽  
2021 ◽  
Vol 3 (2) ◽  
pp. 197-232
Author(s):  
Yves Rosseel
Keyword(s):  
Structural Equation ◽  
Structural Equation Models ◽  
Likelihood Function ◽  
Likelihood Estimation ◽  
Missing At Random ◽  
Equation Modeling ◽  
Computationally Efficient ◽  
Log Likelihood ◽  
Efficient Expression ◽  
Special Case

This paper discusses maximum likelihood estimation for two-level structural equation models when data are missing at random at both levels. Building on existing literature, a computationally efficient expression is derived to evaluate the observed log-likelihood. Unlike previous work, the expression is valid for the special case where the model implied variance–covariance matrix at the between level is singular. Next, the log-likelihood function is translated to R code. A sequence of R scripts is presented, starting from a naive implementation and ending at the final implementation as found in the lavaan package. Along the way, various computational tips and tricks are given.

scholarly journals Bayesian Inference of Species Trees using Diffusion Models

2020 ◽  
Vol 70 (1) ◽  
pp. 145-161 ◽  
Author(s):  
Marnus Stoltz ◽  
Boris Baeumer ◽  
Remco Bouckaert ◽  
Colin Fox ◽  
Gordon Hiscott ◽  
...  
Keyword(s):  
Bayesian Inference ◽  
Numerical Algorithms ◽  
Diffusion Models ◽  
Model Parameters ◽  
Data Sets ◽  
Species Trees ◽  
Computationally Efficient ◽  
Data Set ◽  
Snp Data ◽  
Binary Markers

Abstract We describe a new and computationally efficient Bayesian methodology for inferring species trees and demographics from unlinked binary markers. Likelihood calculations are carried out using diffusion models of allele frequency dynamics combined with novel numerical algorithms. The diffusion approach allows for analysis of data sets containing hundreds or thousands of individuals. The method, which we call Snapper, has been implemented as part of the BEAST2 package. We conducted simulation experiments to assess numerical error, computational requirements, and accuracy recovering known model parameters. A reanalysis of soybean SNP data demonstrates that the models implemented in Snapp and Snapper can be difficult to distinguish in practice, a characteristic which we tested with further simulations. We demonstrate the scale of analysis possible using a SNP data set sampled from 399 fresh water turtles in 41 populations. [Bayesian inference; diffusion models; multi-species coalescent; SNP data; species trees; spectral methods.]

scholarly journals Maximum Likelihood Estimation of the VAR(1) Model Parameters with Missing Observations

2013 ◽  
Vol 2013 ◽  
pp. 1-13 ◽  
Author(s):  
Helena Mouriño ◽  
Maria Isabel Barão
Keyword(s):  
Stochastic Process ◽  
Missing Data ◽  
Maximum Likelihood ◽  
Moving Average ◽  
Practical Importance ◽  
Likelihood Estimation ◽  
Model Parameters ◽  
Missing Observations ◽  
Data Set ◽  
The Impact

Missing-data problems are extremely common in practice. To achieve reliable inferential results, we need to take into account this feature of the data. Suppose that the univariate data set under analysis has missing observations. This paper examines the impact of selecting an auxiliary complete data set—whose underlying stochastic process is to some extent interdependent with the former—to improve the efficiency of the estimators for the relevant parameters of the model. The Vector AutoRegressive (VAR) Model has revealed to be an extremely useful tool in capturing the dynamics of bivariate time series. We propose maximum likelihood estimators for the parameters of the VAR(1) Model based on monotone missing data pattern. Estimators’ precision is also derived. Afterwards, we compare the bivariate modelling scheme with its univariate counterpart. More precisely, the univariate data set with missing observations will be modelled by an AutoRegressive Moving Average (ARMA(2,1)) Model. We will also analyse the behaviour of the AutoRegressive Model of order one, AR(1), due to its practical importance. We focus on the mean value of the main stochastic process. By simulation studies, we conclude that the estimator based on the VAR(1) Model is preferable to those derived from the univariate context.

Sign in / Sign up

Export Citation Format

Share Document