Using Artificial Intelligence to Create Risk Scores: From Evidence to Practice in the Department of Veterans Affairs

2020 ◽  
Author(s):  
Christos Makridis ◽  
Tim Strebel ◽  
Vincent C. Marconi ◽  
Gil Alterovitz
2021 ◽  
Author(s):  
Christos Makridis ◽  
Seth Hurley ◽  
Mary Klote ◽  
Gil Alterovitz

UNSTRUCTURED There is widespread agreement that, while artificial intelligence offers significant potential benefits for individuals and society at large, there are also serious challenges to overcome with respect to its governance. Recent policymaking has focused on establishing principles for the trustworthy use of AI. Adhering to these principles is especially important to adhere to protect vulnerable groups and ensure their confidence in the technology and its uses. Using the Department of Veterans Affairs as a case study, we focus on three principles of particular interest: (i) designing, developing, acquiring, and using AI where the benefits of use significantly outweigh the risks and the risks are assessed and managed, (ii) ensuring that the application of AI occurs in well-defined domains and are accurate, effective, and fit for intended purposes, and (iii) ensure the operations and outcomes of AI applications are sufficiently interpretable and understandable by all subject matter experts, users, and others. We argue that these principles and applications apply to vulnerable groups more generally and that adherence to them can allow the VA and other organizations to continue modernizing its technology governance, leveraging the gains of AI and managing its risks.


2021 ◽  
Vol 28 (1) ◽  
pp. e100312
Author(s):  
Christos A Makridis ◽  
Tim Strebel ◽  
Vincent Marconi ◽  
Gil Alterovitz

Using administrative data on all Veterans who enter Department of Veterans Affairs (VA) medical centres throughout the USA, this paper uses artificial intelligence (AI) to predict mortality rates for patients with COVID-19 between March and August 2020. First, using comprehensive data on over 10 000 Veterans’ medical history, demographics and lab results, we estimate five AI models. Our XGBoost model performs the best, producing an area under the receive operator characteristics curve (AUROC) and area under the precision-recall curve of 0.87 and 0.41, respectively. We show how focusing on the performance of the AUROC alone can lead to unreliable models. Second, through a unique collaboration with the Washington D.C. VA medical centre, we develop a dashboard that incorporates these risk factors and the contributing sources of risk, which we deploy across local VA medical centres throughout the country. Our results provide a concrete example of how AI recommendations can be made explainable and practical for clinicians and their interactions with patients.


Sign in / Sign up

Export Citation Format

Share Document