scholarly journals Development, Implementation, and Evaluation of a Personalized Machine Learning Algorithm for Clinical Decision Support: Case Study With Shingles Vaccination (Preprint)

2019 ◽  
Author(s):  
Ji Chen ◽  
Sara Chokshi ◽  
Roshini Hegde ◽  
Javier Gonzalez ◽  
Eduardo Iturrate ◽  
...  
Keyword(s):  
Machine Learning ◽  
Decision Support ◽  
Clinical Decision Support ◽  
Learning Algorithm ◽  
Signal To Noise Ratio ◽  
Clinical Decision ◽  
Control Group ◽  
Signal To Noise ◽  
Noise Optimization ◽  
Foundational Work

BACKGROUND Although clinical decision support (CDS) alerts are effective reminders of best practices, their effectiveness is blunted by clinicians who fail to respond to an overabundance of inappropriate alerts. An electronic health record (EHR)–integrated machine learning (ML) algorithm is a potentially powerful tool to increase the signal-to-noise ratio of CDS alerts and positively impact the clinician’s interaction with these alerts in general. OBJECTIVE This study aimed to describe the development and implementation of an ML-based signal-to-noise optimization system (SmartCDS) to increase the <i>signal</i> of alerts by decreasing the volume of low-value herpes zoster (shingles) vaccination alerts. METHODS We built and deployed SmartCDS, which builds personalized user activity profiles to suppress shingles vaccination alerts unlikely to yield a clinician’s interaction. We extracted all records of shingles alerts from January 2017 to March 2019 from our EHR system, including 327,737 encounters, 780 providers, and 144,438 patients. RESULTS During the 6 weeks of pilot deployment, the SmartCDS system suppressed an average of 43.67% (15,425/35,315) potential shingles alerts (appointments) and maintained stable counts of weekly shingles vaccination orders (326.3 with system active vs 331.3 in the control group; <i>P</i>=.38) and weekly user-alert interactions (1118.3 with system active vs 1166.3 in the control group; <i>P</i>=.20). CONCLUSIONS All key statistics remained stable while the system was turned on. Although the results are promising, the characteristics of the system can be subject to future data shifts, which require automated logging and monitoring. We demonstrated that an automated, ML-based method and data architecture to suppress alerts are feasible without detriment to overall order rates. This work is the first alert suppression ML-based model deployed in practice and serves as foundational work in encounter-level customization of alert display to maximize effectiveness.

scholarly journals Development, Implementation, and Evaluation of a Personalized Machine Learning Algorithm for Clinical Decision Support: Case Study With Shingles Vaccination

10.2196/16848 ◽  
2020 ◽  
Vol 22 (4) ◽  
pp. e16848
Author(s):  
Ji Chen ◽  
Sara Chokshi ◽  
Roshini Hegde ◽  
Javier Gonzalez ◽  
Eduardo Iturrate ◽  
...  
Keyword(s):  
Machine Learning ◽  
Decision Support ◽  
Clinical Decision Support ◽  
Learning Algorithm ◽  
Signal To Noise Ratio ◽  
Clinical Decision ◽  
Control Group ◽  
Signal To Noise ◽  
Noise Optimization ◽  
Foundational Work

Background Although clinical decision support (CDS) alerts are effective reminders of best practices, their effectiveness is blunted by clinicians who fail to respond to an overabundance of inappropriate alerts. An electronic health record (EHR)–integrated machine learning (ML) algorithm is a potentially powerful tool to increase the signal-to-noise ratio of CDS alerts and positively impact the clinician’s interaction with these alerts in general. Objective This study aimed to describe the development and implementation of an ML-based signal-to-noise optimization system (SmartCDS) to increase the signal of alerts by decreasing the volume of low-value herpes zoster (shingles) vaccination alerts. Methods We built and deployed SmartCDS, which builds personalized user activity profiles to suppress shingles vaccination alerts unlikely to yield a clinician’s interaction. We extracted all records of shingles alerts from January 2017 to March 2019 from our EHR system, including 327,737 encounters, 780 providers, and 144,438 patients. Results During the 6 weeks of pilot deployment, the SmartCDS system suppressed an average of 43.67% (15,425/35,315) potential shingles alerts (appointments) and maintained stable counts of weekly shingles vaccination orders (326.3 with system active vs 331.3 in the control group; P=.38) and weekly user-alert interactions (1118.3 with system active vs 1166.3 in the control group; P=.20). Conclusions All key statistics remained stable while the system was turned on. Although the results are promising, the characteristics of the system can be subject to future data shifts, which require automated logging and monitoring. We demonstrated that an automated, ML-based method and data architecture to suppress alerts are feasible without detriment to overall order rates. This work is the first alert suppression ML-based model deployed in practice and serves as foundational work in encounter-level customization of alert display to maximize effectiveness.

scholarly journals Application of Clinical Decision Support System to Assist Breast Cancer Patients with Lifestyle Modifications during the COVID-19 Pandemic: A Randomised Controlled Trial

Nutrients ◽  
2021 ◽  
Vol 13 (6) ◽  
pp. 2115
Author(s):  
Panos Papandreou ◽  
Aristea Gioxari ◽  
Frantzeska Nimee ◽  
Maria Skouroliakou
Keyword(s):  
Breast Cancer ◽  
Quality Of Life ◽  
Decision Support ◽  
Clinical Decision Support ◽  
Clinical Decision ◽  
Control Group ◽  
Healthy Weight ◽  
Lifestyle Modifications ◽  
Randomised Controlled

Clinical decision support systems (CDSS) are data aggregation tools based on computer technology that assist clinicians to promote healthy weight management and prevention of cardiovascular diseases. We carried out a randomised controlled 3-month trial to implement lifestyle modifications in breast cancer (BC) patients by means of CDSS during the COVID-19 pandemic. In total, 55 BC women at stages I-IIIA were enrolled. They were randomly assigned either to Control group, receiving general lifestyle advice (n = 28) or the CDSS group (n = 27), to whom the CDSS provided personalised dietary plans based on the Mediterranean diet (MD) together with physical activity guidelines. Food data, anthropometry, blood markers and quality of life were evaluated. At 3 months, higher adherence to MD was recorded in the CDSS group, accompanied by lower body weight (kg) and body fat mass percentage compared to control (p < 0.001). In the CDSS arm, global health/quality of life was significantly improved at the trial endpoint (p < 0.05). Fasting blood glucose and lipid levels (i.e., cholesterol, LDL, triacylglycerols) of the CDSS arm remained unchanged (p > 0.05) but were elevated in the control arm at 3 months (p < 0.05). In conclusion, CDSS could be a promising tool to assist BC patients with lifestyle modifications during the COVID-19 pandemic.

scholarly journals NCOG-31. DEVELOPMENT AND EVALUATION OF PROGNOSTIC MODELS TO PREDICT SURVIVAL OF PATIENTS WITH GLIOBLASTOMA MULTIFORME

2020 ◽  
Vol 22 (Supplement_2) ◽  
pp. ii135-ii136
Author(s):  
John Lin ◽  
Michelle Mai ◽  
Saba Paracha
Keyword(s):  
Machine Learning ◽  
Decision Support ◽  
Glioblastoma Multiforme ◽  
Random Forest ◽  
Regression Model ◽  
Clinical Decision Support ◽  
Multivariate Regression ◽  
Visual Analysis ◽  
Clinical Decision ◽  
Prognostic Models

Abstract Glioblastoma multiforme (GBM), the most common form of glioma, is a malignant tumor with a high risk of mortality. By providing accurate survival estimates, prognostic models have been identified as promising tools in clinical decision support. In this study, we produced and validated two machine learning-based models to predict survival time for GBM patients. Publicly available clinical and genomic data from The Cancer Genome Atlas (TCGA) and Broad Institute GDAC Firehouse were obtained through cBioPortal. Random forest and multivariate regression models were created to predict survival. Predictive accuracy was assessed and compared through mean absolute error (MAE) and root mean square error (RMSE) calculations. 619 GBM patients were included in the dataset. There were 381 (62.9%) cases of recurrence/progression and 53 (8.7%) cases of disease-free survival. The MAE and RMSE values were 0.553 and 0.887 years respectively for the random forest regression model, and they were 1.756 and 2.451 years respectively for the multivariate regression model. Both models accurately predicted overall survival. Comparison of models through MAE, RMSE, and visual analysis produced higher accuracy values for random forest than multivariate linear regression. Further investigation on feature selection and model optimization may improve predictive power. These findings suggest that using machine learning in GBM prognostic modeling will improve clinical decision support. *Co-first authors.

GUESS: projecting machine learning scores to well-calibrated probability estimates for clinical decision-making

2018 ◽  
Vol 35 (14) ◽  
pp. 2458-2465 ◽  
Author(s):  
Johanna Schwarz ◽  
Dominik Heider
Keyword(s):  
Machine Learning ◽  
Decision Support ◽  
Clinical Decision Support ◽  
State Of The Art ◽  
Clinical Decision Support Systems ◽  
Clinical Decision ◽  
Calibration Method ◽  
Probability Estimates ◽  
Calibration Methods ◽  
Novel Method

Abstract Motivation Clinical decision support systems have been applied in numerous fields, ranging from cancer survival toward drug resistance prediction. Nevertheless, clinical decision support systems typically have a caveat: many of them are perceived as black-boxes by non-experts and, unfortunately, the obtained scores cannot usually be interpreted as class probability estimates. In probability-focused medical applications, it is not sufficient to perform well with regards to discrimination and, consequently, various calibration methods have been developed to enable probabilistic interpretation. The aims of this study were (i) to develop a tool for fast and comparative analysis of different calibration methods, (ii) to demonstrate their limitations for the use on clinical data and (iii) to introduce our novel method GUESS. Results We compared the performances of two different state-of-the-art calibration methods, namely histogram binning and Bayesian Binning in Quantiles, as well as our novel method GUESS on both, simulated and real-world datasets. GUESS demonstrated calibration performance comparable to the state-of-the-art methods and always retained accurate class discrimination. GUESS showed superior calibration performance in small datasets and therefore may be an optimal calibration method for typical clinical datasets. Moreover, we provide a framework (CalibratR) for R, which can be used to identify the most suitable calibration method for novel datasets in a timely and efficient manner. Using calibrated probability estimates instead of original classifier scores will contribute to the acceptance and dissemination of machine learning based classification models in cost-sensitive applications, such as clinical research. Availability and implementation GUESS as part of CalibratR can be downloaded at CRAN.

“Send & hold” clinical decision support rules improvement to reduce unnecessary testing of vitamins A, E, K, B1, B2, B3, B6 and C

2018 ◽  
Vol 56 (7) ◽  
pp. 1063-1070 ◽  
Author(s):  
Enrique Rodriguez-Borja ◽  
Africa Corchon-Peyrallo ◽  
Esther Barba-Serrano ◽  
Celia Villalba Martínez ◽  
Arturo Carratala Calvo
Keyword(s):  
Decision Support ◽  
Clinical Decision Support ◽  
Primary Care Physicians ◽  
Vitamin B1 ◽  
Clinical Decision ◽  
Health Department ◽  
Leadership Role ◽  
Control Group ◽  
Vitamin B3 ◽  
The Impact

Abstract Background: We assessed the impact of several “send & hold” clinical decision support rules (CDSRs) within the electronical request system for vitamins A, E, K, B1, B2, B3, B6 and C for all outpatients at a large health department. Methods: When ordered through electronical request, providers (except for all our primary care physicians who worked as a non-intervention control group) were always asked to answer several compulsory questions regarding main indication, symptomatology, suspected diagnosis, vitamin active treatments, etc., for each vitamin test using a drop-down list format. After samples arrival, tests were later put on hold internally by our laboratory information system (LIS) until review for their appropriateness was made by two staff pathologists according to the provided answers and LIS records (i.e. “send & hold”). The number of tests for each analyte was compared between the 10-month period before and after CDSRs implementation in both groups. Results: After implementation, vitamins test volumes decreased by 40% for vitamin A, 29% for vitamin E, 42% for vitamin K, 37% for vitamin B1, 85% for vitamin B2, 68% for vitamin B3, 65% for vitamin B6 and 59% for vitamin C (all p values 0.03 or lower except for vitamin B3), whereas in control group, the majority increased or remained stable. In patients with rejected vitamins, no new requests and/or adverse clinical outcome comments due to this fact were identified. Conclusions: “Send & hold” CDSRs are a promising informatics tool that can support in utilization management and enhance the pathologist’s leadership role as tests specialist.

An Assessment of Federated Machine Learning for Translational Research

2020 ◽  
pp. 123-142
Author(s):  
Manoj A. Thomas ◽  
Diya Suzanne Abraham ◽  
Dapeng Liu
Keyword(s):  
Machine Learning ◽  
Decision Making ◽  
Decision Support ◽  
Translational Research ◽  
Clinical Decision Support ◽  
Clinical Decision ◽  
High Quality ◽  
Intermediate Data ◽  
Translational Informatics ◽  
Data Analytic

Translational research (TR) is the harnessing of knowledge from basic science and clinical research to advance healthcare. As a sister discipline, translational informatics (TI) concerns the application of informatics theories, methods, and frameworks to TR. This chapter builds upon TR concepts and aims to advance the use of machine learning (ML) and data analytics for improving clinical decision support. A federated machine learning (FML) architecture is described to aggregate multiple ML endpoints, and intermediate data analytic processes and products to output high quality knowledge discovery and decision making. The proposed architecture is evaluated for its operational performance based on three propositions, and a case for clinical decision support in the prediction of adult Sepsis is presented. The chapter illustrates contributions to the advancement of FML and TI.

Sign in / Sign up

Export Citation Format

Share Document