scholarly journals Development and validation of a cardiovascular disease risk-prediction model using population health surveys: the Cardiovascular Disease Population Risk Tool (CVDPoRT)

2018 ◽  
Vol 190 (29) ◽  
pp. E871-E882 ◽  
Author(s):  
Douglas G. Manuel ◽  
Meltem Tuna ◽  
Carol Bennett ◽  
Deirdre Hennessy ◽  
Laura Rosella ◽  
...  
Keyword(s):  
Cardiovascular Disease ◽  
Prediction Model ◽  
Population Health ◽  
Risk Prediction ◽  
Health Surveys ◽  
Disease Risk ◽  
Cardiovascular Disease Risk ◽  
Risk Prediction Model ◽  
Development And Validation

Development and Validation of a Risk Prediction Model for Patients with COVID-19 Using Linked Population Health Data (Preprint)

2021 ◽  
Author(s):  
Nikolaos Mastellos ◽  
Richard Betteridge ◽  
Prasanth Peddaayyavarla ◽  
Andrew Moran ◽  
Jurgita Kaubryte ◽  
...  
Keyword(s):  
Health Care ◽  
Prediction Model ◽  
Population Health ◽  
Risk Prediction ◽  
Validation Cohort ◽  
Risk Prediction Model ◽  
Adult Patients ◽  
Internal Validation ◽  
Temporal Validation ◽  
Development And Validation

BACKGROUND The impact of the COVID-19 pandemic on health care utilisation and associated costs has been significant, with one in ten patients becoming severely ill and being admitted to hospital with serious complications during the first wave of the pandemic. Risk prediction models can help health care providers identify high-risk patients in their populations and intervene to improve health outcomes and reduce associated costs. OBJECTIVE To develop and validate a hospitalisation risk prediction model for adult patients with laboratory confirmed Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2). METHODS The model was developed using pre-linked and standardised data of adult patients with laboratory confirmed SARS-CoV-2 from Cerner’s population health management platform (HealtheIntent®) in the London Borough of Lewisham. A total of 14,203 patients who tested positive for SARS-CoV-2 between 1st March 2020 and 28th February 2021 were included in the development and internal validation cohorts. A second temporal validation cohort covered the period between 1st March 2021 to 30th April 2021. The outcome variable was hospital admission in adult patients with laboratory confirmed SARS-CoV-2. A generalised linear model was used to train the model. The predictive performance of the model was assessed using the area under the receiver operator characteristic curve (ROC-AUC). RESULTS Overall, 14,203 patients were included. Of those, 9,755 (68.7%) were assigned to the development cohort, 2,438 (17.2%) to the internal validation cohort, and 2,010 (14.1%) to the temporal validation cohort. A total of 917 (9.4%) patients were admitted to hospital in the development cohort, 210 (8.6%) in the internal validation cohort, and a further 204 (10.1%) in the temporal validation cohort. The model had a ROC-AUC of 0.85 in both the development and validation cohorts. The most predictive factors were older age, male sex, Asian or Other ethnic minority background, obesity, chronic kidney disease, hypertension and diabetes. CONCLUSIONS The COVID-19 hospitalisation risk prediction model demonstrated very good performance and can be used to stratify risk in the Lewisham population to help providers reduce unnecessary hospital admissions and associated costs, improve patient outcomes, and target those at greatest risk to ensure full vaccination against SARS-CoV-2. Further research may examine the external validity of the model in other populations.

scholarly journals Impact of sample size on the stability of risk scores from clinical prediction models: a case study in cardiovascular disease

2020 ◽  
Vol 4 (1) ◽  
Author(s):  
Alexander Pate ◽  
Richard Emsley ◽  
Matthew Sperrin ◽  
Glen P. Martin ◽  
Tjeerd van Staa
Keyword(s):  
Cardiovascular Disease ◽  
Prediction Model ◽  
Sample Size ◽  
Risk Prediction ◽  
Prediction Models ◽  
Disease Risk ◽  
Cardiovascular Disease Risk ◽  
Risk Scores ◽  
Model Derivation ◽  
The Stability

Abstract Background Stability of risk estimates from prediction models may be highly dependent on the sample size of the dataset available for model derivation. In this paper, we evaluate the stability of cardiovascular disease risk scores for individual patients when using different sample sizes for model derivation; such sample sizes include those similar to models recommended in the national guidelines, and those based on recently published sample size formula for prediction models. Methods We mimicked the process of sampling N patients from a population to develop a risk prediction model by sampling patients from the Clinical Practice Research Datalink. A cardiovascular disease risk prediction model was developed on this sample and used to generate risk scores for an independent cohort of patients. This process was repeated 1000 times, giving a distribution of risks for each patient. N = 100,000, 50,000, 10,000, Nmin (derived from sample size formula) and Nepv10 (meets 10 events per predictor rule) were considered. The 5–95th percentile range of risks across these models was used to evaluate instability. Patients were grouped by a risk derived from a model developed on the entire population (population-derived risk) to summarise results. Results For a sample size of 100,000, the median 5–95th percentile range of risks for patients across the 1000 models was 0.77%, 1.60%, 2.42% and 3.22% for patients with population-derived risks of 4–5%, 9–10%, 14–15% and 19–20% respectively; for N = 10,000, it was 2.49%, 5.23%, 7.92% and 10.59%, and for N using the formula-derived sample size, it was 6.79%, 14.41%, 21.89% and 29.21%. Restricting this analysis to models with high discrimination, good calibration or small mean absolute prediction error reduced the percentile range, but high levels of instability remained. Conclusions Widely used cardiovascular disease risk prediction models suffer from high levels of instability induced by sampling variation. Many models will also suffer from overfitting (a closely linked concept), but at acceptable levels of overfitting, there may still be high levels of instability in individual risk. Stability of risk estimates should be a criterion when determining the minimum sample size to develop models.

Sign in / Sign up

Export Citation Format

Share Document