Small sample properties of the logit maximum likelihood estimator:a simulation study

1993 ◽  
Vol 20 (2) ◽  
pp. 237-250
Author(s):  
Yu-Min Chen ◽  
Evan E. Anderson
Keyword(s):  
Maximum Likelihood ◽  
Simulation Study ◽  
Small Sample ◽  
Small Sample Properties

scholarly journals Estimating logit models with small samples

2021 ◽  
pp. 1-16
Author(s):  
Carlisle Rainey ◽  
Kelly McCaskey
Keyword(s):  
Maximum Likelihood ◽  
American Political Science Review ◽  
Substantial Improvement ◽  
Small Sample ◽  
Small Samples ◽  
Noticeable Improvement ◽  
Small Sample Properties ◽  
Substantial Bias ◽  
Ml Estimator ◽  
American Political Science

Abstract In small samples, maximum likelihood (ML) estimates of logit model coefficients have substantial bias away from zero. As a solution, we remind political scientists of Firth's (1993, Biometrika, 80, 27–38) penalized maximum likelihood (PML) estimator. Prior research has described and used PML, especially in the context of separation, but its small sample properties remain under-appreciated. The PML estimator eliminates most of the bias and, perhaps more importantly, greatly reduces the variance of the usual ML estimator. Thus, researchers do not face a bias-variance tradeoff when choosing between the ML and PML estimators—the PML estimator has a smaller bias and a smaller variance. We use Monte Carlo simulations and a re-analysis of George and Epstein (1992, American Political Science Review, 86, 323–337) to show that the PML estimator offers a substantial improvement in small samples (e.g., 50 observations) and noticeable improvement even in larger samples (e.g., 1000 observations).

scholarly journals Regression shrinkage methods for clinical prediction models do not guarantee improved performance: Simulation study

2020 ◽  
Vol 29 (11) ◽  
pp. 3166-3178 ◽  
Author(s):  
Ben Van Calster ◽  
Maarten van Smeden ◽  
Bavo De Cock ◽  
Ewout W Steyerberg
Keyword(s):  
Maximum Likelihood ◽  
Sample Size ◽  
Simulation Study ◽  
Prediction Models ◽  
Small Sample Size ◽  
Predictive Performance ◽  
Small Sample ◽  
Adaptive Lasso ◽  
Shrinkage Methods ◽  
Improved Performance

When developing risk prediction models on datasets with limited sample size, shrinkage methods are recommended. Earlier studies showed that shrinkage results in better predictive performance on average. This simulation study aimed to investigate the variability of regression shrinkage on predictive performance for a binary outcome. We compared standard maximum likelihood with the following shrinkage methods: uniform shrinkage (likelihood-based and bootstrap-based), penalized maximum likelihood (ridge) methods, LASSO logistic regression, adaptive LASSO, and Firth’s correction. In the simulation study, we varied the number of predictors and their strength, the correlation between predictors, the event rate of the outcome, and the events per variable. In terms of results, we focused on the calibration slope. The slope indicates whether risk predictions are too extreme (slope < 1) or not extreme enough (slope > 1). The results can be summarized into three main findings. First, shrinkage improved calibration slopes on average. Second, the between-sample variability of calibration slopes was often increased relative to maximum likelihood. In contrast to other shrinkage approaches, Firth’s correction had a small shrinkage effect but showed low variability. Third, the correlation between the estimated shrinkage and the optimal shrinkage to remove overfitting was typically negative, with Firth’s correction as the exception. We conclude that, despite improved performance on average, shrinkage often worked poorly in individual datasets, in particular when it was most needed. The results imply that shrinkage methods do not solve problems associated with small sample size or low number of events per variable.

Sign in / Sign up

Export Citation Format

Share Document