A quantitative bias analysis to assess the impact of unmeasured confounding on associations between diabetes and periodontitis

2020 ◽  
Vol 48 (1) ◽  
pp. 51-60
Author(s):  
Talal S. Alshihayb ◽  
Elizabeth A. Kaye ◽  
Yihong Zhao ◽  
Cataldo W. Leone ◽  
Brenda Heaton
Keyword(s):  
Bias Analysis ◽  
Unmeasured Confounding ◽  
Quantitative Bias Analysis ◽  
The Impact

OP208 Did Health Technology Assessments Make the Wrong Call? Quantitative Bias Analysis: Alectinib versus Ceritinib in Non-Small Cell Lung Cancer

2021 ◽  
Vol 37 (S1) ◽  
pp. 6-6
Author(s):  
Samantha Wilkinson ◽  
Alind Gupta ◽  
Eric Mackay ◽  
Paul Arora ◽  
Kristian Thorlund ◽  
...  
Keyword(s):  
Real World ◽  
Performance Status ◽  
Missing At Random ◽  
Health Technology ◽  
Bias Analysis ◽  
Ecog Performance Status ◽  
Unmeasured Confounding ◽  
Unmeasured Confounder ◽  
Innovative Therapies ◽  
Quantitative Bias Analysis

IntroductionThe German health technology assessment (HTA) rejected additional benefit of alectinib for second line (2L) ALK+ NSCLC, citing possible biases from missing ECOG performance status data and unmeasured confounding in real-world evidence (RWE) for 2L ceritinib that was submitted as a comparator to the single arm alectinib trial. Alectinib was approved in the US and therefore US post-launch RWE can be used to evaluate this HTA decision.MethodsWe compared the real-world effectiveness of alectinib with ceritinib in 2L post-crizotinib ALK+ NSCLC using the nationwide Flatiron Health electronic health record (EHR)-derived de-identified database. Using quantitative bias analysis (QBA), we estimated the strength of (i) unmeasured confounding and (ii) deviation from missing-at-random (MAR) assumptions needed to nullify any overall survival (OS) benefit.ResultsAlectinib had significantly longer median OS than ceritinib in complete case analysis. The estimated effect size (Hazard Ratio: 0.55) was robust to risk ratios of unmeasured confounder-outcome and confounder-exposure associations of <2.4.Based on tipping point analysis, missing baseline ECOG performance status for ceritinib-treated patients (49% missing) would need to be more than 3.4-times worse than expected under MAR to nullify the OS benefit observed for alectinib.ConclusionsOnly implausible levels of bias reversed our conclusions. These methods could provide a framework to explore uncertainty and aid decision-making for HTAs to enable patient access to innovative therapies.

Approaches to Improve Causal Inference in Physical Activity Epidemiology

2020 ◽  
Vol 17 (1) ◽  
pp. 80-84
Author(s):  
Brigid M. Lynch ◽  
Suzanne C. Dixon-Suen ◽  
Andrea Ramirez Varela ◽  
Yi Yang ◽  
Dallas R. English ◽  
...  
Keyword(s):  
Physical Activity ◽  
Causal Inference ◽  
Propensity Scores ◽  
Directed Acyclic Graphs ◽  
Epidemiological Methods ◽  
Bias Analysis ◽  
Randomized Controlled ◽  
Acyclic Graphs ◽  
Quantitative Bias Analysis ◽  
The Impact

Background: It is not always clear whether physical activity is causally related to health outcomes, or whether the associations are induced through confounding or other biases. Randomized controlled trials of physical activity are not feasible when outcomes of interest are rare or develop over many years. Thus, we need methods to improve causal inference in observational physical activity studies. Methods: We outline a range of approaches that can improve causal inference in observational physical activity research, and also discuss the impact of measurement error on results and methods to minimize this. Results: Key concepts and methods described include directed acyclic graphs, quantitative bias analysis, Mendelian randomization, and potential outcomes approaches which include propensity scores, g methods, and causal mediation. Conclusions: We provide a brief overview of some contemporary epidemiological methods that are beginning to be used in physical activity research. Adoption of these methods will help build a stronger body of evidence for the health benefits of physical activity.

scholarly journals Addressing Measurement Error in Random Forests using Quantitative Bias Analysis

2021 ◽  
Author(s):  
Tammy Jiang ◽  
Jaimie L Gradus ◽  
Timothy L Lash ◽  
Matthew P Fox
Keyword(s):  
Machine Learning ◽  
Measurement Error ◽  
Random Forest ◽  
Random Forests ◽  
Model Performance ◽  
Variable Importance ◽  
Bias Analysis ◽  
Variable Importance Measures ◽  
Quantitative Bias Analysis ◽  
The Impact

Abstract Although variables are often measured with error, the impact of measurement error on machine learning predictions is seldom quantified. The purpose of this study was to assess the impact of measurement error on random forest model performance and variable importance. First, we assessed the impact of misclassification (i.e., measurement error of categorical variables) of predictors on random forest model performance (e.g., accuracy, sensitivity) and variable importance (mean decrease in accuracy) using data from the United States National Comorbidity Survey Replication (2001 - 2003). Second, we simulated datasets in which we know the true model performance and variable importance measures and could verify that quantitative bias analysis was recovering the truth in misclassified versions of the datasets. Our findings show that measurement error in the data used to construct random forests can distort model performance and variable importance measures, and that bias analysis can recover the correct results. This study highlights the utility of applying quantitative bias analysis in machine learning to quantify the impact of measurement error on study results.

Invited Commentary: Quantitative Bias Analysis can see the Forest for the Trees

2021 ◽  
Author(s):  
Paul Gustafson
Keyword(s):  
Machine Learning ◽  
Statistical Models ◽  
Learning Algorithms ◽  
Machine Learning Algorithms ◽  
Bias Analysis ◽  
Quantitative Bias Analysis

Abstract The article by Jiang et al (Am J. Epidemiol.) extends quantitative bias analysis from the realm of statistical models to the realm of machine learning algorithms. Given the rooting of statistical models in the spirit of explanation and the rooting of machine learning algorithms in the spirt of prediction, this extension is thought provoking indeed. Some such thoughts are expounded here.

You Can’t Drive a Car With Only Three Wheels

2019 ◽  
Vol 188 (9) ◽  
pp. 1682-1685 ◽  
Author(s):  
Hailey R Banack
Keyword(s):  
Causal Inference ◽  
Observational Data ◽  
Causal Effect ◽  
Causal Effects ◽  
Measurement Bias ◽  
Exposure Misclassification ◽  
Unmeasured Confounding ◽  
Obstetrical Care ◽  
The Impact ◽  
Fundamental Requirement

Abstract Authors aiming to estimate causal effects from observational data frequently discuss 3 fundamental identifiability assumptions for causal inference: exchangeability, consistency, and positivity. However, too often, studies fail to acknowledge the importance of measurement bias in causal inference. In the presence of measurement bias, the aforementioned identifiability conditions are not sufficient to estimate a causal effect. The most fundamental requirement for estimating a causal effect is knowing who is truly exposed and unexposed. In this issue of the Journal, Caniglia et al. (Am J Epidemiol. 2019;000(00):000–000) present a thorough discussion of methodological challenges when estimating causal effects in the context of research on distance to obstetrical care. Their article highlights empirical strategies for examining nonexchangeability due to unmeasured confounding and selection bias and potential violations of the consistency assumption. In addition to the important considerations outlined by Caniglia et al., authors interested in estimating causal effects from observational data should also consider implementing quantitative strategies to examine the impact of misclassification. The objective of this commentary is to emphasize that you can’t drive a car with only three wheels, and you also cannot estimate a causal effect in the presence of exposure misclassification bias.

scholarly journals The association of active smoking with multiple cancers: national census-cancer registry cohorts with quantitative bias analysis

2013 ◽  
Vol 24 (6) ◽  
pp. 1243-1255 ◽  
Author(s):  
Tony Blakely ◽  
Jan J. Barendregt ◽  
Rachel H. Foster ◽  
Sarah Hill ◽  
June Atkinson ◽  
...  
Keyword(s):  
Cancer Registry ◽  
Active Smoking ◽  
Bias Analysis ◽  
Multiple Cancers ◽  
Quantitative Bias Analysis

scholarly journals A primer on quantitative bias analysis with positive predictive values in research using electronic health data

2019 ◽  
Vol 26 (12) ◽  
pp. 1664-1674 ◽  
Author(s):  
Sophia R Newcomer ◽  
Stan Xu ◽  
Martin Kulldorff ◽  
Matthew F Daley ◽  
Bruce Fireman ◽  
...  
Keyword(s):  
Data Quality ◽  
Predictive Value ◽  
Health Informatics ◽  
Health Data ◽  
Data Sources ◽  
Bias Analysis ◽  
Case Definitions ◽  
Predictive Values ◽  
Electronic Health ◽  
Quantitative Bias Analysis

Abstract Objective In health informatics, there have been concerns with reuse of electronic health data for research, including potential bias from incorrect or incomplete outcome ascertainment. In this tutorial, we provide a concise review of predictive value–based quantitative bias analysis (QBA), which comprises epidemiologic methods that use estimates of data quality accuracy to quantify the bias caused by outcome misclassification. Target Audience Health informaticians and investigators reusing large, electronic health data sources for research. Scope When electronic health data are reused for research, validation of outcome case definitions is recommended, and positive predictive values (PPVs) are the most commonly reported measure. Typically, case definitions with high PPVs are considered to be appropriate for use in research. However, in some studies, even small amounts of misclassification can cause bias. In this tutorial, we introduce methods for quantifying this bias that use predictive values as inputs. Using epidemiologic principles and examples, we first describe how multiple factors influence misclassification bias, including outcome misclassification levels, outcome prevalence, and whether outcome misclassification levels are the same or different by exposure. We then review 2 predictive value–based QBA methods and why outcome PPVs should be stratified by exposure for bias assessment. Using simulations, we apply and evaluate the methods in hypothetical electronic health record–based immunization schedule safety studies. By providing an overview of predictive value–based QBA, we hope to bridge the disciplines of health informatics and epidemiology to inform how the impact of data quality issues can be quantified in research using electronic health data sources.

Sign in / Sign up

Export Citation Format

Share Document