scholarly journals Uncertainty Quantification in Machine Learning and Nonlinear Least Squares Regression Models

AIChE Journal ◽  
2021 ◽  
Author(s):  
Ni Zhan ◽  
John R. Kitchin
Keyword(s):  
Machine Learning ◽  
Least Squares ◽  
Uncertainty Quantification ◽  
Regression Models ◽  
Nonlinear Least Squares ◽  
Least Squares Regression

scholarly journals Uncertainty Quantification in Machine Learning and Nonlinear Least Squares Regression Models

2021 ◽  
Author(s):  
Ni Zhan ◽  
John Kitchin
Keyword(s):  
Machine Learning ◽  
Least Squares ◽  
Uncertainty Quantification ◽  
Automatic Differentiation ◽  
Nonlinear Least Squares ◽  
Delta Method ◽  
Training Data ◽  
Model Parameters ◽  
Uncertainty Measure ◽  
Least Squares Regression

Machine learning (ML) models are valuable research tools for making accurate predictions. However, ML models often unreliably extrapolate outside their training data. We propose an uncertainty quantification method for ML models (and generally for other nonlinear models) with parameters trained by least squares regression. The uncertainty measure is based on the multiparameter delta method from statistics, which gives the standard error of the prediction. The uncertainty measure requires the gradient of the model prediction and the Hessian of the loss function, both with respect to model parameters. Both the gradient and Hessian can be readily obtained from most ML software frameworks by automatic differentiation. We show that the uncertainty measure is larger for input space regions that are not part of the training data. Therefore this method can be used to identify extrapolation and to aid in selecting training data or assessing model reliability.

Predictive Performance of Bayesian and Nonlinear Least-Squares Regression Programs for Lidocaine

1992 ◽  
Vol 14 (4) ◽  
pp. 286-291 ◽  
Author(s):  
Christopher J. Destache ◽  
Daniel E. Hilleman ◽  
Syed J. Mohiuddin ◽  
Patricia T. Lang
Keyword(s):  
Least Squares ◽  
Nonlinear Least Squares ◽  
Predictive Performance ◽  
Least Squares Regression
Sign in / Sign up

Export Citation Format

Share Document