Real-time clinical decision supports the prediction and management of in-hospital acute kidney injury with recurrent neural networks (Preprint)

BACKGROUND Acute kidney injury(AKI) is commonly encountered in clinical practice and associated with poor patient outcomes and increased healthcare costs. AKI poses significant challenges for clinicians but effective measures for the prediction and prevention of AKI are lacking. Previously published AKI prediction models mostly have simple design without external validation. Furthermore, little is known about how to link the model output and clinical decision supports due to the blackbox nature of the neural network models. OBJECTIVE We aimed to present an externally validated recurrent neural network (RNN)-based prediction model for in-hospital AKI, and to show the explainability of the model in relation to clinical decision support. METHODS Study populations were all patients aged ≥ 18 years and hospitalized more than a week from 2013 to 2017 in two tertiary hospitals in Korea (Seoul National University Bundang Hospital and Seoul National University Hospital). All demographics, laboratory values, vital signs, and clinical conditions were obtained from the EHR of each hospital. A total of 102 variables included in the model. Each variable falls into two categories: static and dynamic variables. We developed two-stage hierarchical prediction models (model 1 and model 2) using RNN algorithms. The outcome variable for Model 1 was the occurrence of AKI within 7 days from the present. Model 2 predicts the future trajectory of Cr values up to 72 hours. Internal validation was performed by 5-fold cross validation using the training set, and then external validation was done using independent test set. RESULTS Of a total of 118,893 patients initially screened, after excluding cases with missing data and estimated glomerular filtration rate <15 ml/min/1.73m2 or end-stage kidney disease, 40,552 patients in training cohort and 4,084 in external validation cohort (test cohort) were used for model development. Model 1 with the observation window of 3 days predicts AKI development with the area under the curve of 0.80 (sensitivity 0.72, specificity 0.89) in external validation. The model 2 predicted the future creatinine values within 3 days with the mean square errors of 0.04-0.06 for patients with higher risks of AKI and 0.05-0.12 for those with lower risks. On the basis of the developed models, we showed the probability of AKI according to the feature values in total patients and each individual with partial dependence plots and individual conditional expectation plots. In addition, we estimated the effects of feature modifications such as nephrotoxic drug discontinuation on the future creatinine levels. CONCLUSIONS We developed and externally validated a real-time AKI prediction model using RNN algorithms. Our model could provide real-time assessment of future AKI occurrences and individualized risk factors for AKI in general inpatient cohorts. These suggest approaches to support clinical decisions based on the prediction models for in-hospital AKI.

Assessing workflow of emergency physicians in the use of clinical decision support

The emergency department (ED) is a complex environment where diagnoses must often be made quickly, based on incomplete information. Pulmonary embolism (PE) is an especially challenging diagnosis that is frequently delayed or missed due to its non-specific symptoms, and can be life-threatening when not treated. Clinical decision supports (CDS) have the potential to improve these difficult decisions; however, previous efforts to implement CDS in the ED have faced challenges due to poor usability and lack of workflow integration. The objective of this study is to identify potential barriers to workflow integration from the technology’s implementation and inform the CDS design; this is achieved by analyzing ED physicians’ workflow during a usability evaluation of two different CDS, a web-based risk scoring CDS and a CDS designed using an human-centered design (HCD) process and human factors (HF) design principles. The number of cases matching the guideline-based workflow and the percent of correct diagnostic decisions increased from the use of the HF-based CDS, but varied depending on the patient scenario. We identified three workflow variations, which had both positive and negative implications for the CDS design and implementation. The workflow analysis can be used to inform the CDS design and improve the technology’s usability and integration in physician workflow prior to implementation.

An evaluation of artificial intelligence-based clinical decision supports use in Brazil.

e18081 Background: The Instituto do Câncer do Ceará (ICC), a 160-bed oncology hospital located in Brazil, serves approximately 23,000 patients monthly. In December of 2017, ICC implemented Watson for Oncology (WFO), an artificial intelligence (AI)-based clinical decision-support (CDS) tool to help enhance personalized cancer care. As of December 2018, 903 cases involving mainly breast, prostate and gastric cancers were entered in WFO. The purpose of this study was to investigate how implementation of WFO and use by oncologists affects clinical decision-making and workflow. Methods: 7 oncologists who employed WfO during and after the patients’ first visit were recruited to complete a survey regarding usability, decision-making and workflow. The group consisted of 1 urologist, 3 gastric surgeons, 1 gynecologist, 1 breast surgeon, 1 head-neck surgeon. Survey questions integrated the CDS Five Rights framework. Results: Most oncologists agreed that WFO is easy to understand and provides complete, relevant and actionable information at an appropriate time (Table). Opinions on the impact on treatment decisions varied. 71.4% expressed positive statements (agree or strongly agree) pertaining to the use of WFO. Conclusions: In this study, oncologists felt WFO met 5 Rights expectations for CDS; 57% felt that WFO exceed expectations. Further research is needed to understand how variation in experience affects decision impact. [Table: see text]

Residents’ use of mobile technologies: three challenges for graduate medical education

IntroductionThe practice of medicine involves, among other things, managing ambiguity, interpreting context and making decisions in the face of uncertainty. These uncertainties, amplified for learners, can be negotiated in a variety of ways; however, the promise, efficiency and availability of mobile technologies and clinical decision supports make these tools an appealing way to manage ambiguity.Mobile technologies are becoming increasingly prevalent in medical education and in the practice of medicine. Because of this, we explored how the use of mobile technologies is influencing residents’ experiences of graduate medical education.MethodsWe conducted an 18-month qualitative investigation to explore this issue. Our research was conceptually and theoretically framed in sociomaterial studies of professional learning. Specifically, our methods included logging of technology use and related reflexive writing by residents (n=10), interviews with residents (n=12) and interviews with faculty (n=6).ResultsWe identified three challenges for graduate medical education related to mobile technology use: (1) efficiency versus critical thinking; (2) patient context versus evidence-based medicine and (3) home/work-life balance.DiscussionIn this digital age, decontextualised knowledge is readily available. Our data indicate that rather than access to accurate knowledge, the more pressing challenge for medical educators is managing how, when and why learners choose to access that information.