scholarly journals A four-step strategy for handling missing outcome data in randomised trials affected by a pandemic

2020 ◽  
Author(s):  
Suzie Cro ◽  
Tim P Morris ◽  
Brennan Kahan ◽  
Victoria R Cornelius ◽  
James Carpenter
Keyword(s):  
Sensitivity Analysis ◽  
Missing Data ◽  
Treatment Effect ◽  
Missing At Random ◽  
Outcome Data ◽  
Free World ◽  
Randomised Trials ◽  
Primary Analysis ◽  
Missing Not At Random ◽  
Clear Thinking

Background: The coronavirus pandemic (Covid-19) presents a variety of challenges for ongoing clinical trials, including an inevitably higher rate of missing outcome data, with new and non-standard reasons for missingness. International drug trial guidelines recommend trialists review plans for handling missing data in the conduct and statistical analysis, but clear recommendations are lacking.Methods: We present a four-step strategy for handling missing outcome data in the analysis of randomised trials that are ongoing during a pandemic. We consider handling missing data arising due to (i) participant infection, (ii) treatment disruptions and (iii) loss to follow-up. We consider both settings where treatment effects for a ‘pandemic-free world’ and ‘world including a pandemic’ are of interest. Results: In any trial, investigators should; (1) Clarify the treatment estimand of interest with respect to the occurrence of the pandemic; (2) Establish what data are missing for the chosen estimand; (3) Perform primary analysis under the most plausible missing data assumptions followed by; (4) Sensitivity analysis under alternative plausible assumptions. To obtain an estimate of the treatment effect in a ‘pandemic-free world’, participant data that are clinically affected by the pandemic (directly due to infection or indirectly via treatment disruptions) are not relevant and can be set to missing. For primary analysis, a missing-at-random assumption that conditions on all observed data that are expected to be associated with both the outcome and missingness may be most plausible. For the treatment effect in the ‘world including a pandemic’, all participant data is relevant and should be included in the analysis. For primary analysis, a missing-at-random assumption – potentially incorporating a pandemic time-period indicator and participant infection status – or a missing-not-at-random assumption with a poorer response may be most relevant, depending on the setting. In all scenarios, sensitivity analysis under credible missing-not-at-random assumptions should be used to evaluate the robustness of results. We highlight controlled multiple imputation as an accessible tool for conducting sensitivity analyses.Conclusions: Missing data problems will be exacerbated for trials active during the Covid-19 pandemic. This four-step strategy will facilitate clear thinking about the appropriate analysis for relevant questions of interest.

scholarly journals A four-step strategy for handling missing outcome data in randomised trials affected by a pandemic

2020 ◽  
Author(s):  
Suzie Cro ◽  
Tim P Morris ◽  
Brennan C Kahan ◽  
Victoria R Cornelius ◽  
James R Carpenter
Keyword(s):  
Sensitivity Analysis ◽  
Missing Data ◽  
Treatment Effect ◽  
Missing At Random ◽  
Outcome Data ◽  
Sensitivity Analyses ◽  
Free World ◽  
Randomised Trials ◽  
Primary Analysis ◽  
Missing Not At Random

Abstract Background: The coronavirus pandemic (Covid-19) presents a variety of challenges for ongoing clinical trials, including an inevitably higher rate of missing outcome data, with new and non-standard reasons for missingness. International drug trial guidelines recommend trialists review plans for handling missing data in the conduct and statistical analysis, but clear recommendations are lacking.Methods: We present a four-step strategy for handling missing outcome data in the analysis of randomised trials that are ongoing during a pandemic. We consider handling missing data arising due to (i) participant infection, (ii) treatment disruptions and (iii) loss to follow-up. We consider both settings where treatment effects for a ‘pandemic-free world’ and ‘world including a pandemic’ are of interest. Results: In any trial, investigators should; (1) Clarify the treatment estimand of interest with respect to the occurrence of the pandemic; (2) Establish what data are missing for the chosen estimand; (3) Perform primary analysis under the most plausible missing data assumptions followed by; (4) Sensitivity analysis under alternative plausible assumptions. To obtain an estimate of the treatment effect in a ‘pandemic-free world’, participant data that are clinically affected by the pandemic (directly due to infection or indirectly via treatment disruptions) are not relevant and can be set to missing. For primary analysis, a missing-at-random assumption that conditions on all observed data that are expected to be associated with both the outcome and missingness may be most plausible. For the treatment effect in the ‘world including a pandemic’, all participant data is relevant and should be included in the analysis. For primary analysis, a missing-at-random assumption – potentially incorporating a pandemic time-period indicator and participant infection status – or a missing-not-at-random assumption with a poorer response may be most relevant, depending on the setting. In all scenarios, sensitivity analysis under credible missing-not-at-random assumptions should be used to evaluate the robustness of results. We highlight controlled multiple imputation as an accessible tool for conducting sensitivity analyses.Conclusions: Missing data problems will be exacerbated for trials active during the Covid-19 pandemic. This four-step strategy will facilitate clear thinking about the appropriate analysis for relevant questions of interest.

scholarly journals A four-step strategy for handling missing outcome data in randomised trials affected by a pandemic

2020 ◽  
Author(s):  
Suzie Cro ◽  
Tim P Morris ◽  
Brennan C Kahan ◽  
Victoria R Cornelius ◽  
James R Carpenter
Keyword(s):  
Sensitivity Analysis ◽  
Missing Data ◽  
Treatment Effect ◽  
Missing At Random ◽  
Outcome Data ◽  
Sensitivity Analyses ◽  
Free World ◽  
Randomised Trials ◽  
Primary Analysis ◽  
Missing Not At Random

Abstract Background The coronavirus pandemic (Covid-19) presents a variety of challenges for ongoing clinical trials, including an inevitably higher rate of missing outcome data, with new and non-standard reasons for missingness. International drug trial guidelines recommend trialists review plans for handling missing data in the conduct and statistical analysis, but clear recommendations are lacking. Methods We present a four-step strategy for handling missing outcome data in the analysis of randomised trials that are ongoing during a pandemic. We consider handling missing data arising due to (i) participant infection, (ii) treatment disruptions and (iii) loss to follow-up. We consider both settings where treatment effects for a ‘pandemic-free world’ and ‘world including a pandemic’ are of interest. Results In any trial, investigators should; (1) Clarify the treatment estimand of interest; (2) Establish what data are missing for the estimand at hand; (3) Perform primary analysis under the most plausible missing data assumptions followed by; (4) Sensitivity analysis under alternative plausible assumptions. To obtain an estimate of the treatment effect in a ‘pandemic-free world’, data from participants clinically affected by the pandemic (directly via infection or indirectly via treatment disruptions) are not relevant and can be set to missing. For primary analysis, a missing-at-random assumption that conditions on all observed data that are expected to be associated with both the outcome and missingness may be most plausible. For the treatment effect in the ‘world including a pandemic’, all participant data is relevant and should be included in the analysis. For primary analysis, a missing-at-random assumption – potentially incorporating a pandemic time-period indicator and participant infection status – or a missing-not-at-random assumption with a poorer response may be most relevant, depending on the setting. In all scenarios, sensitivity analysis under credible missing-not-at-random assumptions should be used to evaluate the robustness of results. We highlight controlled multiple imputation as an accessible tool for conducting sensitivity analyses. Conclusions Missing data problems will be exacerbated for trials active during the Covid-19 pandemic. This four-step strategy will facilitate clear thinking about the appropriate analysis for relevant questions of interest.

scholarly journals A review of the use of controlled multiple imputation in randomised controlled trials with missing outcome data

2021 ◽  
Vol 21 (1) ◽  
Author(s):  
Ping-Tee Tan ◽  
Suzie Cro ◽  
Eleanor Van Vogt ◽  
Matyas Szigeti ◽  
Victoria R. Cornelius
Keyword(s):  
Sensitivity Analysis ◽  
Missing Data ◽  
Multiple Imputation ◽  
Randomised Controlled Trials ◽  
Missing At Random ◽  
Sensitivity Analyses ◽  
Controlled Trials ◽  
Primary Analysis ◽  
Randomised Controlled ◽  
The Impact

Abstract Background Missing data are common in randomised controlled trials (RCTs) and can bias results if not handled appropriately. A statistically valid analysis under the primary missing-data assumptions should be conducted, followed by sensitivity analysis under alternative justified assumptions to assess the robustness of results. Controlled Multiple Imputation (MI) procedures, including delta-based and reference-based approaches, have been developed for analysis under missing-not-at-random assumptions. However, it is unclear how often these methods are used, how they are reported, and what their impact is on trial results. This review evaluates the current use and reporting of MI and controlled MI in RCTs. Methods A targeted review of phase II-IV RCTs (non-cluster randomised) published in two leading general medical journals (The Lancet and New England Journal of Medicine) between January 2014 and December 2019 using MI. Data was extracted on imputation methods, analysis status, and reporting of results. Results of primary and sensitivity analyses for trials using controlled MI analyses were compared. Results A total of 118 RCTs (9% of published RCTs) used some form of MI. MI under missing-at-random was used in 110 trials; this was for primary analysis in 43/118 (36%), and in sensitivity analysis for 70/118 (59%) (3 used in both). Sixteen studies performed controlled MI (1.3% of published RCTs), either with a delta-based (n = 9) or reference-based approach (n = 7). Controlled MI was mostly used in sensitivity analysis (n = 14/16). Two trials used controlled MI for primary analysis, including one reporting no sensitivity analysis whilst the other reported similar results without imputation. Of the 14 trials using controlled MI in sensitivity analysis, 12 yielded comparable results to the primary analysis whereas 2 demonstrated contradicting results. Only 5/110 (5%) trials using missing-at-random MI and 5/16 (31%) trials using controlled MI reported complete details on MI methods. Conclusions Controlled MI enabled the impact of accessible contextually relevant missing data assumptions to be examined on trial results. The use of controlled MI is increasing but is still infrequent and poorly reported where used. There is a need for improved reporting on the implementation of MI analyses and choice of controlled MI parameters.

scholarly journals Should multiple imputation be the method of choice for handling missing data in randomized trials?

2016 ◽  
Vol 27 (9) ◽  
pp. 2610-2626 ◽  
Author(s):  
Thomas R Sullivan ◽  
Ian R White ◽  
Amy B Salter ◽  
Philip Ryan ◽  
Katherine J Lee
Keyword(s):  
Missing Data ◽  
Multiple Imputation ◽  
Treatment Effect ◽  
Randomized Trials ◽  
Missing At Random ◽  
Outcome Data ◽  
Average Treatment Effect ◽  
Analysis Model ◽  
Data Simulation ◽  
Using Data

The use of multiple imputation has increased markedly in recent years, and journal reviewers may expect to see multiple imputation used to handle missing data. However in randomized trials, where treatment group is always observed and independent of baseline covariates, other approaches may be preferable. Using data simulation we evaluated multiple imputation, performed both overall and separately by randomized group, across a range of commonly encountered scenarios. We considered both missing outcome and missing baseline data, with missing outcome data induced under missing at random mechanisms. Provided the analysis model was correctly specified, multiple imputation produced unbiased treatment effect estimates, but alternative unbiased approaches were often more efficient. When the analysis model overlooked an interaction effect involving randomized group, multiple imputation produced biased estimates of the average treatment effect when applied to missing outcome data, unless imputation was performed separately by randomized group. Based on these results, we conclude that multiple imputation should not be seen as the only acceptable way to handle missing data in randomized trials. In settings where multiple imputation is adopted, we recommend that imputation is carried out separately by randomized group.

scholarly journals Handling missing data in modelling quality of clinician-prescribed routine care: Sensitivity analysis of departure from missing at random assumption

2020 ◽  
Vol 29 (10) ◽  
pp. 3076-3092 ◽  
Author(s):  
Susan Gachau ◽  
Matteo Quartagno ◽  
Edmund Njeru Njagi ◽  
Nelson Owuor ◽  
Mike English ◽  
...  
Keyword(s):  
Sensitivity Analysis ◽  
Missing Data ◽  
Multiple Imputation ◽  
Missing At Random ◽  
Parameter Estimates ◽  
Analysis Model ◽  
Major Drawback ◽  
Missing Not At Random ◽  
Prior Distributions ◽  
Random Mechanism

Missing information is a major drawback in analyzing data collected in many routine health care settings. Multiple imputation assuming a missing at random mechanism is a popular method to handle missing data. The missing at random assumption cannot be confirmed from the observed data alone, hence the need for sensitivity analysis to assess robustness of inference. However, sensitivity analysis is rarely conducted and reported in practice. We analyzed routine paediatric data collected during a cluster randomized trial conducted in Kenyan hospitals. We imputed missing patient and clinician-level variables assuming the missing at random mechanism. We also imputed missing clinician-level variables assuming a missing not at random mechanism. We incorporated opinions from 15 clinical experts in the form of prior distributions and shift parameters in the delta adjustment method. An interaction between trial intervention arm and follow-up time, hospital, clinician and patient-level factors were included in a proportional odds random-effects analysis model. We performed these analyses using R functions derived from the jomo package. Parameter estimates from multiple imputation under the missing at random mechanism were similar to multiple imputation estimates assuming the missing not at random mechanism. Our inferences were insensitive to departures from the missing at random assumption using either the prior distributions or shift parameters sensitivity analysis approach.

scholarly journals Weighted multiple imputation of ethnicity data that are missing not at random in primary care databases

2017 ◽  
Vol 1 (1) ◽  
Author(s):  
Tra My Pham ◽  
Irene Petersen ◽  
James Carpenter ◽  
Tim Morris
Keyword(s):  
Primary Care ◽  
Missing Data ◽  
Multiple Imputation ◽  
Simulation Study ◽  
Case Analysis ◽  
Missing At Random ◽  
Complete Case ◽  
Missing Not At Random ◽  
Health Records ◽  
Ethnicity Data

ABSTRACT BackgroundEthnicity is an important factor to be considered in health research because of its association with inequality in disease prevalence and the utilisation of healthcare. Ethnicity recording has been incorporated in primary care electronic health records, and hence is available in large UK primary care databases such as The Health Improvement Network (THIN). However, since primary care data are routinely collected for clinical purposes, a large amount of data that are relevant for research including ethnicity is often missing. A popular approach for missing data is multiple imputation (MI). However, the conventional MI method assuming data are missing at random does not give plausible estimates of the ethnicity distribution in THIN compared to the general UK population. This might be due to the fact that ethnicity data in primary care are likely to be missing not at random. ObjectivesI propose a new MI method, termed ‘weighted multiple imputation’, to deal with data that are missing not at random in categorical variables.MethodsWeighted MI combines MI and probability weights which are calculated using external data sources. Census summary statistics for ethnicity can be used to form weights in weighted MI such that the correct marginal ethnic breakdown is recovered in THIN. I conducted a simulation study to examine weighted MI when ethnicity data are missing not at random. In this simulation study which resembled a THIN dataset, ethnicity was an independent variable in a survival model alongside other covariates. Weighted MI was compared to the conventional MI and other traditional missing data methods including complete case analysis and single imputation.ResultsWhile a small bias was still present in ethnicity coefficient estimates under weighted MI, it was less severe compared to MI assuming missing at random. Complete case analysis and single imputation were inadequate to handle data that are missing not at random in ethnicity.ConclusionsAlthough not a total cure, weighted MI represents a pragmatic approach that has potential applications not only in ethnicity but also in other incomplete categorical health indicators in electronic health records.

scholarly journals P427 A hybrid approach of handling missing data in inflammatory bowel disease (IBD) trials: results from VISIBLE 1 and VARSITY

2020 ◽  
Vol 14 (Supplement_1) ◽  
pp. S388-S389
Author(s):  
J Chen ◽  
S Hunter ◽  
K Kisfalvi ◽  
R A Lirio
Keyword(s):  
Sensitivity Analysis ◽  
Missing Data ◽  
Statistical Power ◽  
Hybrid Approach ◽  
Missing At Random ◽  
P Value ◽  
Two Phase ◽  
Treatment Difference ◽  
Mayo Score ◽  
The Impact

Abstract Background Missing data is common in IBD trials. Depending on the volume and nature of missing data, it can reduce statistical power for detecting treatment difference, introduce potential bias and invalidate conclusions. Non-responder imputation (NRI), where patients (patients) with missing data are considered treatment failures, is widely used to handle missing data for dichotomous efficacy endpoints in IBD trials. However, it does not consider the mechanisms leading to missing data and can potentially underestimate the treatment effect. We proposed a hybrid (HI) approach combining NRI and multiple imputation (MI) as an alternative to NRI in the analyses of two phase 3 trials of vedolizumab (VDZ) in patients with moderate-to-severe UC – VISIBLE 11 and VARSITY2. Methods VISIBLE 1 and VARSITY assessed efficacy using dichotomous endpoints based on complete Mayo score. Full methodologies reported previously.1,2 Our proposed HI approach is aimed at imputing missing Mayo scores, instead of imputing the missing dichotomous efficacy endpoint. To assess the impact of dropouts for different missing data mechanisms (categorised as ‘missing not at random [MNAR]’ and ‘missing at random [MAR]’, HI was implemented as a potential sensitivity analysis, where dropouts owing to safety or lack of efficacy were imputed using NRI (assuming MNAR) and other missing data were imputed using MI (assuming MAR). For MI, each component of the Mayo score was imputed via a multivariate stepwise approach using a fully conditional specification ordinal logistic method. Missing baseline scores were imputed using baseline characteristics data. Missing scores from each subsequent visit were imputed using all previous visits in a stepwise fashion. Fifty imputation datasets were computed for each component of Mayo score. The complete Mayo score and relevant efficacy endpoints were derived subsequently. The analysis was performed within each imputed dataset to determine treatment difference, 95% CI and p-value, which were then combined via Rubin’s rules3. Results Tables 1 and 2 show a comparison of efficacy in the two studies using the primary NRI analysis vs. the alternative HI approach for handling missing data. Conclusion HI and NRI approaches can provide consistent efficacy analyses in IBD trials. The HI approach can serve as a useful sensitivity analysis to assess the impact of dropouts under different missing data mechanisms and evaluate the robustness of efficacy conclusions. Reference

A post-hoc health-related quality of life (HRQoL) analysis of patients with metastatic colorectal cancer (mCRC) in the phase III CONCUR trial.

2015 ◽  
Vol 33 (3_suppl) ◽  
pp. 667-667
Author(s):  
Jane Chang ◽  
Dawn Odom ◽  
Christina Radder ◽  
Christian Kappeler ◽  
Rui-hua Xu ◽  
...  
Keyword(s):  
Missing Data ◽  
Treatment Effect ◽  
Minimal Important Difference ◽  
Progression Free Survival ◽  
Missing At Random ◽  
Phase Iii ◽  
Similar Proportion ◽  
Informative Missing Data ◽  
Post Hoc ◽  
Over Time

667 Background: CONCUR (NCT01584830) showed that regorafenib (REG) significantly improves overall survival (OS) and progression-free survival (PFS) vs. placebo (PBO) in Asian patients with mCRC who progressed after standard therapy (J Li, et al. WCGI 2014). Post hoc exploratory analyses were conducted to assess the effect of treatment on HRQoL. Methods: Patients were randomly assigned 2:1 to treatment with either REG (n=136) or PBO (n=68). The HRQoL analyses included all 204 patients and were selected a priori based on clinical relevance; the global health status/QoL (QL) and the physical functioning (PF) scales of the EORTC QLQ-C30 questionnaire were used. A linear mixed-effects model (LMM) was used to examine the treatment effect on HRQoL and trends over time, assuming that data were missing at random. A pattern-mixture model (PMM) was applied to assess the treatment effect while accounting for potentially informative missing data. Time-to-deterioration (TTD) of HRQoL and responder analyses were conducted to determine the treatment effect based on timing and proportion of patients reaching a minimal important difference (MID) change in QL/PF (≥10 points). Results: The QL and PF changes over time were numerically similar between REG and PBO based on the LMM. The PMM grouped patients based on timing of last HRQoL assessment (<3 or ≥3 cycles) and had results similar to the LMM, demonstrating little impact of informative missing data. For the TTD analysis, when an event was defined as the earliest MID decrease in QL/PF, disease progression, or death, REG showed significantly different TTD curves from PBO (QL: median 8.0 vs. 7.0 weeks, hazard ratio (HR)=0.54; PF: median 7.9 vs. 7.0 weeks, HR=0.59, respectively; all p<0.01). Median TTD was comparable between treatments after removing progression/death from the definition. The responder analyses showed that a similar proportion of patients achieved an improvement in MID in REG vs. PBO (QL: 27.2% vs. 29.4%; PF: 14.0% vs.16.2%, respectively). Conclusions: The findings of this exploratory analysis demonstrate that HRQoL is similar for the REG and PBO groups, indicating that REG prolongs OS and PFS vs. PBO while maintaining a comparable HRQoL. Clinical trial information: NCT01584830.

scholarly journals Missing data in randomised controlled trials of rheumatoid arthritis drug therapy are substantial and handled inappropriately

RMD Open ◽  
2021 ◽  
Vol 7 (2) ◽  
pp. e001708
Author(s):  
Nasim A Khan ◽  
Karina D Torralba ◽  
Fawad Aslam
Keyword(s):  
Rheumatoid Arthritis ◽  
Sensitivity Analysis ◽  
Drug Therapy ◽  
Missing Data ◽  
Primary Outcome ◽  
Outcome Data ◽  
Data Handling ◽  
Significant Discrepancy ◽  
Imputation Methods ◽  
Simple Imputation

ObjectivesTo analyse the amount, reporting and handling of missing data, approach to intention-to-treat (ITT) principle application and sensitivity analysis utilisation in randomised clinical trials (RCTs) of rheumatoid arthritis (RA). To assess the trend in such reporting 10 years apart (2006 and 2016).MethodsParallel group drug therapy RA RCTs with a clinical primary endpoint.Results176 studies enrolling a median of 160 (IQR 62–339) patients were eligible. In terms of actual analysis: 81 (46%) RCTs conducted ITT, 42 (23.9%) conducted modified ITT while 53 (30.1%) conducted non-ITT analysis. Only 58 of 97 (59.8%) RCTs reporting an ITT analysis actually performed it. The median (IQR) numbers of participants completing the trial and included in analysis for primary outcome were 86% (74%–91%) and 100% (97.1%–100%), respectively. 53 (32.7%) and 65 (40.1%) RCTs had >20% and 10%–20% missing primary outcome data, respectively. Missing data handling was unreported by 58 of 171 (33.9%) RCTs. When reported, vast majority used simple imputation methods. No significant trend towards improved reporting was seen between 2006 and 2016. Sensitivity analysis numerically improved from 2006 to 2016 (14.7% vs 21.4%).ConclusionsThere is significant discrepancy in the reported and the actual performed analysis in RA drug therapy RCTs. Nearly one-third of RCTs had >20% missing data. The reporting and methods of missing data handling remain inadequate with high usage of non-preferred simple imputation methods. Sensitivity analysis utilisation was low. No trend towards better missing data reporting and handling was seen.

Sign in / Sign up

Export Citation Format

Share Document