An External Validation of the QCovid Risk Prediction Algorithm for Risk of Mortality from COVID-19 in Adults: National Validation Cohort Study in Scotland

2021 ◽  
Author(s):  
Colin Simpson ◽  
Chris Robertson ◽  
Steven Kerr ◽  
Ting Shi ◽  
Eleftheria Vasileiou ◽  
...  
Keyword(s):  
Cohort Study ◽  
Risk Prediction ◽  
Validation Cohort ◽  
External Validation ◽  
Prediction Algorithm ◽  
Risk Of Mortality

scholarly journals Living risk prediction algorithm (QCOVID) for risk of hospital admission and mortality from coronavirus 19 in adults: national derivation and validation cohort study

BMJ ◽  
2020 ◽  
pp. m3731 ◽  
Author(s):  
Ash K Clift ◽  
Carol A C Coupland ◽  
Ruth H Keogh ◽  
Karla Diaz-Ordaz ◽  
Elizabeth Williamson ◽  
...  
Keyword(s):  
Cohort Study ◽  
Confidence Interval ◽  
Hospital Admission ◽  
Risk Prediction ◽  
Validation Cohort ◽  
Population Based ◽  
Prediction Algorithm ◽  
Derivation Cohort ◽  
Time To Death ◽  
Time Periods

Abstract Objective To derive and validate a risk prediction algorithm to estimate hospital admission and mortality outcomes from coronavirus disease 2019 (covid-19) in adults. Design Population based cohort study. Setting and participants QResearch database, comprising 1205 general practices in England with linkage to covid-19 test results, Hospital Episode Statistics, and death registry data. 6.08 million adults aged 19-100 years were included in the derivation dataset and 2.17 million in the validation dataset. The derivation and first validation cohort period was 24 January 2020 to 30 April 2020. The second temporal validation cohort covered the period 1 May 2020 to 30 June 2020. Main outcome measures The primary outcome was time to death from covid-19, defined as death due to confirmed or suspected covid-19 as per the death certification or death occurring in a person with confirmed severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection in the period 24 January to 30 April 2020. The secondary outcome was time to hospital admission with confirmed SARS-CoV-2 infection. Models were fitted in the derivation cohort to derive risk equations using a range of predictor variables. Performance, including measures of discrimination and calibration, was evaluated in each validation time period. Results 4384 deaths from covid-19 occurred in the derivation cohort during follow-up and 1722 in the first validation cohort period and 621 in the second validation cohort period. The final risk algorithms included age, ethnicity, deprivation, body mass index, and a range of comorbidities. The algorithm had good calibration in the first validation cohort. For deaths from covid-19 in men, it explained 73.1% (95% confidence interval 71.9% to 74.3%) of the variation in time to death (R 2 ); the D statistic was 3.37 (95% confidence interval 3.27 to 3.47), and Harrell’s C was 0.928 (0.919 to 0.938). Similar results were obtained for women, for both outcomes, and in both time periods. In the top 5% of patients with the highest predicted risks of death, the sensitivity for identifying deaths within 97 days was 75.7%. People in the top 20% of predicted risk of death accounted for 94% of all deaths from covid-19. Conclusion The QCOVID population based risk algorithm performed well, showing very high levels of discrimination for deaths and hospital admissions due to covid-19. The absolute risks presented, however, will change over time in line with the prevailing SARS-C0V-2 infection rate and the extent of social distancing measures in place, so they should be interpreted with caution. The model can be recalibrated for different time periods, however, and has the potential to be dynamically updated as the pandemic evolves.

scholarly journals Development and external validation of a COVID-19 mortality risk prediction algorithm: a multicentre retrospective cohort study

BMJ Open ◽  
2020 ◽  
Vol 10 (12) ◽  
pp. e044028
Author(s):  
Jin Mei ◽  
Weihua Hu ◽  
Qijian Chen ◽  
Chang Li ◽  
Zaishu Chen ◽  
...  
Keyword(s):  
Cohort Study ◽  
Respiratory Failure ◽  
Simple Model ◽  
Mortality Risk ◽  
Retrospective Cohort Study ◽  
Retrospective Cohort ◽  
Validation Cohort ◽  
External Validation ◽  
Prediction Algorithm ◽  
Full Model

ObjectiveThis study aimed to develop and externally validate a COVID-19 mortality risk prediction algorithm.DesignRetrospective cohort study.SettingFive designated tertiary hospitals for COVID-19 in Hubei province, China.ParticipantsWe routinely collected medical data of 1364 confirmed adult patients with COVID-19 between 8 January and 19 March 2020. Among them, 1088 patients from two designated hospitals in Wuhan were used to develop the prognostic model, and 276 patients from three hospitals outside Wuhan were used for external validation. All patients were followed up for a maximal of 60 days after the diagnosis of COVID-19.MethodsThe model discrimination was assessed by the area under the receiver operating characteristic curve (AUC) and Somers’ D test, and calibration was examined by the calibration plot. Decision curve analysis was conducted.Main outcome measuresThe primary outcome was all-cause mortality within 60 days after the diagnosis of COVID-19.ResultsThe full model included seven predictors of age, respiratory failure, white cell count, lymphocytes, platelets, D-dimer and lactate dehydrogenase. The simple model contained five indicators of age, respiratory failure, coronary heart disease, renal failure and heart failure. After cross-validation, the AUC statistics based on derivation cohort were 0.96 (95% CI, 0.96 to 0.97) for the full model and 0.92 (95% CI, 0.89 to 0.95) for the simple model. The AUC statistics based on the external validation cohort were 0.97 (95% CI, 0.96 to 0.98) for the full model and 0.88 (95% CI, 0.80 to 0.96) for the simple model. Good calibration accuracy of these two models was found in the derivation and validation cohort.ConclusionThe prediction models showed good model performance in identifying patients with COVID-19 with a high risk of death in 60 days. It may be useful for acute risk classification.Web calculatorWe provided a freely accessible web calculator (https://www.whuyijia.com/).

scholarly journals An external validation of the QCovid risk prediction algorithm for risk of mortality from COVID-19 in adults: national validation cohort study in England

2021 ◽  
Author(s):  
Vahe Nafilyan ◽  
Ben Humberstone ◽  
Nisha Mehta ◽  
Ian Diamond ◽  
Carol Coupland ◽  
...  
Keyword(s):  
Risk Prediction ◽  
Prediction Models ◽  
Validation Cohort ◽  
Population Based ◽  
Prediction Algorithm ◽  
Risk Prediction Models ◽  
Men And Women ◽  
Time Period ◽  
Time Periods ◽  
First Time

SUMMARYBackgroundTo externally validate a risk prediction algorithm (QCovid) to estimate mortality outcomes from COVID-19 in adults in England.MethodsPopulation-based cohort study using the ONS Public Health Linked Data Asset, a cohort based on the 2011 Census linked to Hospital Episode Statistics, the General Practice Extraction Service Data for pandemic planning and research, radiotherapy and systemic chemotherapy records. The primary outcome was time to COVID-19 death, defined as confirmed or suspected COVID-19 death as per death certification. Two time periods were used: (a) 24th January to 30th April 2020; and (b) 1st May to 28th July 2020. We evaluated the performance of the QCovid algorithms using measures of discrimination and calibration for each validation time period.FindingsThe study comprises 34,897,648 adults aged 19-100 years resident in England. There were 26,985 COVID-19 deaths during the first time-period and 13,177 during the second. The algorithms had good calibration in the validation cohort in both time periods with close correspondence of observed and predicted risks. They explained 77.1% (95% CI: 76.9% to 77.4%) of the variation in time to death in men in the first time-period (R2); the D statistic was 3.76 (95% CI: 3.73 to 3.79); Harrell’s C was 0.935 (0.933 to 0.937). Similar results were obtained for women, and in the second time-period. In the top 5% of patients with the highest predicted risks of death, the sensitivity for identifying deaths in the first time period was 65.9% for men and 71.7% for women. People in the top 20% of predicted risks of death accounted for 90.8% of all COVID-19 deaths for men and 93.0% for women.InterpretationThe QCovid population-based risk algorithm performed well, showing very high levels of discrimination for COVID-19 deaths in men and women for both time periods. It has the potential to be dynamically updated as the pandemic evolves and therefore, has potential use in guiding national policy.FundingNational Institute of Health ResearchRESEARCH IN CONTEXTEvidence before this studyPublic policy measures and clinical risk assessment relevant to COVID-19 need to be aided by rigorously developed and validated risk prediction models. A recent living systematic review of published risk prediction models for COVID-19 found most models are subject to a high risk of bias with optimistic reported performance, raising concern that these models may be unreliable when applied in practice. A population-based risk prediction model, QCovid risk prediction algorithm, has recently been developed to identify adults at high risk of serious COVID-19 outcomes, which overcome many of the limitations of previous tools.Added value of this studyCommissioned by the Chief Medical Officer for England, we validated the novel clinical risk prediction model (QCovid) to identify risks of short-term severe outcomes due to COVID-19. We used national linked datasets from general practice, death registry and hospital episode data for a population-representative sample of over 34 million adults. The risk models have excellent discrimination in men and women (Harrell’s C statistic>0.9) and are well calibrated. QCovid represents a new, evidence-based opportunity for population risk-stratification.Implications of all the available evidenceQCovid has the potential to support public health policy, from enabling shared decision making between clinicians and patients in relation to health and work risks, to targeted recruitment for clinical trials, and prioritisation of vaccination, for example.

Sign in / Sign up

Export Citation Format

Share Document