scholarly journals Assessing the International Transferability of a Machine Learning Model for Detecting Medication Error in the General Internal Medicine Clinic: Multicenter Preliminary Validation Study (Preprint)

2020 ◽  
Author(s):  
Yen Po Harvey Chin ◽  
Wenyu Song ◽  
Chia En Lien ◽  
Chang Ho Yoon ◽  
Wei-Chen Wang ◽  
...  
Keyword(s):  
Machine Learning ◽  
Medication Error ◽  
Model Performance ◽  
Learning Model ◽  
Study Cohort ◽  
Learning Approach ◽  
Hospital Data ◽  
General Internal ◽  
Machine Learning Model ◽  
Outpatient Prescriptions

BACKGROUND Although most current medication error prevention systems are rule-based, these systems may result in alert fatigue because of poor accuracy. Previously, we had developed a machine learning (ML) model based on Taiwan’s local databases (TLD) to address this issue. However, the international transferability of this model is unclear. OBJECTIVE This study examines the international transferability of a machine learning model for detecting medication errors and whether the federated learning approach could further improve the accuracy of the model. METHODS The study cohort included 667,572 outpatient prescriptions from 2 large US academic medical centers. Our ML model was applied to build the original model (O model), the local model (L model), and the hybrid model (H model). The O model was built using the data of 1.34 billion outpatient prescriptions from TLD. A validation set with 8.98% (60,000/667,572) of the prescriptions was first randomly sampled, and the remaining 91.02% (607,572/667,572) of the prescriptions served as the local training set for the L model. With a federated learning approach, the H model used the association values with a higher frequency of co-occurrence among the O and L models. A testing set with 600 prescriptions was classified as <i>substantiated</i> and <i>unsubstantiated</i> by 2 independent physician reviewers and was then used to assess model performance. RESULTS The interrater agreement was significant in terms of classifying prescriptions as <i>substantiated</i> and <i>unsubstantiated</i> (κ=0.91; 95% CI 0.88 to 0.95). With thresholds ranging from 0.5 to 1.5, the alert accuracy ranged from 75%-78% for the O model, 76%-78% for the L model, and 79%-85% for the H model. CONCLUSIONS Our ML model has good international transferability among US hospital data. Using the federated learning approach with local hospital data could further improve the accuracy of the model.

scholarly journals Assessing the International Transferability of a Machine Learning Model for Detecting Medication Error in the General Internal Medicine Clinic: Multicenter Preliminary Validation Study

10.2196/23454 ◽  
2021 ◽  
Vol 9 (1) ◽  
pp. e23454
Author(s):  
Yen Po Harvey Chin ◽  
Wenyu Song ◽  
Chia En Lien ◽  
Chang Ho Yoon ◽  
Wei-Chen Wang ◽  
...  
Keyword(s):  
Machine Learning ◽  
Medication Error ◽  
Model Performance ◽  
Learning Model ◽  
Study Cohort ◽  
Learning Approach ◽  
Hospital Data ◽  
General Internal ◽  
Machine Learning Model ◽  
Outpatient Prescriptions

Background Although most current medication error prevention systems are rule-based, these systems may result in alert fatigue because of poor accuracy. Previously, we had developed a machine learning (ML) model based on Taiwan’s local databases (TLD) to address this issue. However, the international transferability of this model is unclear. Objective This study examines the international transferability of a machine learning model for detecting medication errors and whether the federated learning approach could further improve the accuracy of the model. Methods The study cohort included 667,572 outpatient prescriptions from 2 large US academic medical centers. Our ML model was applied to build the original model (O model), the local model (L model), and the hybrid model (H model). The O model was built using the data of 1.34 billion outpatient prescriptions from TLD. A validation set with 8.98% (60,000/667,572) of the prescriptions was first randomly sampled, and the remaining 91.02% (607,572/667,572) of the prescriptions served as the local training set for the L model. With a federated learning approach, the H model used the association values with a higher frequency of co-occurrence among the O and L models. A testing set with 600 prescriptions was classified as substantiated and unsubstantiated by 2 independent physician reviewers and was then used to assess model performance. Results The interrater agreement was significant in terms of classifying prescriptions as substantiated and unsubstantiated (κ=0.91; 95% CI 0.88 to 0.95). With thresholds ranging from 0.5 to 1.5, the alert accuracy ranged from 75%-78% for the O model, 76%-78% for the L model, and 79%-85% for the H model. Conclusions Our ML model has good international transferability among US hospital data. Using the federated learning approach with local hospital data could further improve the accuracy of the model.

scholarly journals The importance of round-robin validation when assessing machine-learning-based vertical extrapolation of wind speeds

2020 ◽  
Author(s):  
Nicola Bodini ◽  
Mike Optis
Keyword(s):  
Machine Learning ◽  
Random Forest ◽  
Power Law ◽  
Wind Farm ◽  
Sonic Anemometer ◽  
Model Performance ◽  
Learning Model ◽  
Round Robin ◽  
Wind Speeds ◽  
Machine Learning Model

Abstract. The extrapolation of wind speeds measured at a meteorological mast to wind turbine hub heights is a key component in a bankable wind farm energy assessment and a significant source of uncertainty. Industry-standard methods for extrapolation include the power law and logarithmic profile. The emergence of machine-learning applications in wind energy has led to several studies demonstrating substantial improvements in vertical extrapolation accuracy in machine-learning methods over these conventional power law and logarithmic profile methods. In all cases, these studies assess relative model performance at a measurement site where, critically, the machine-learning algorithm requires knowledge of the hub-height wind speeds in order to train the model. This prior knowledge provides fundamental advantages to the site-specific machine-learning model over the power law and log profile, which, by contrast, are not highly tuned to hub-height measurements but rather can generalize to any site. Furthermore, there is no practical benefit in applying a machine-learning model at a site where hub-height winds are known; rather, its performance at nearby locations (i.e., across a wind farm site) without hub-height measurements is of most practical interest. To more fairly and practically compare machine-learning-based extrapolation to standard approaches, we implemented a round-robin extrapolation model comparison, in which a random forest machine-learning model is trained and evaluated at different sites and then compared against the power law and logarithmic profile. We consider 20 months of lidar and sonic anemometer data collected at four sites between 50–100 kilometers apart in the central United States. We find that the random forest outperforms the standard extrapolation approaches, especially when incorporating surface measurements as inputs to include the influence of atmospheric stability. When compared at a single site (the traditional comparison approach), the machine-learning improvement in mean absolute error was 28 % and 23 % over the power law and logarithmic profile, respectively. Using the round-robin approach proposed here, this improvement drops to 19 % and 14 %, respectively. These latter values better represent practical model performance, and we conclude that round-robin validation should be the standard for machine-learning-based, wind-speed extrapolation methods.

scholarly journals Can machine learning model with static features be fooled: an adversarial machine learning approach

2020 ◽  
Vol 23 (4) ◽  
pp. 3233-3253 ◽  
Author(s):  
Rahim Taheri ◽  
Reza Javidan ◽  
Mohammad Shojafar ◽  
P. Vinod ◽  
Mauro Conti
Keyword(s):  
Machine Learning ◽  
Learning Model ◽  
Learning Approach ◽  
Machine Learning Model ◽  
Machine Learning Approach

scholarly journals A Machine Learning Approach for Predicting Student Performance

2019 ◽  
pp. 462-465
Author(s):  
C. Selvi ◽  
R. Shalini ◽  
V. Navaneethan ◽  
L. Santhiya
Keyword(s):  
Machine Learning ◽  
Academic Performance ◽  
Student Performance ◽  
Learning Model ◽  
Learning Approach ◽  
Economic Prosperity ◽  
Machine Learning Model ◽  
Machine Learning Approach ◽  
Novel Method ◽  
Predicting Student Performance

An University’s reputation and its standard are weighted by its students performance and their part in the future economic prosperity of the nation, hence a novel method of predicting the student’s upcoming academic performance is really essential to provide a pre-requisite information upon their performances. A machine learning model can be developed to predict the student’s upcoming scores or their entire performance depending upon their previous academic performances.

scholarly journals New methods to improve the vertical extrapolation of near-surface offshore wind speeds

2021 ◽  
Author(s):  
Mike Optis ◽  
Nicola Bodini ◽  
Mithu Debnath ◽  
Paula Doubrawa
Keyword(s):  
Machine Learning ◽  
Wind Speed ◽  
Model Performance ◽  
Learning Model ◽  
Offshore Wind ◽  
Offshore Wind Turbine ◽  
Near Surface ◽  
Wind Speeds ◽  
Machine Learning Model ◽  
Surface Measurements

Abstract. Accurate characterization of the offshore wind resource has been hindered by a sparsity of wind speed observations that span offshore wind turbine rotor-swept heights. Although public availability of floating lidar data is increasing, most offshore wind speed observations continue to come from buoy-based and satellite-based near-surface measurements. The aim of this study is to develop and validate novel vertical extrapolation methods that can accurately estimate wind speed time series across rotor-swept heights using these near-surface measurements. We contrast the conventional logarithmic profile against three novel approaches: a logarithmic profile with a long-term stability correction, a single-column model, and a machine-learning model. These models are developed and validated using 1 year of observations from two floating lidars deployed in U.S. Atlantic offshore wind energy areas. We find that the machine-learning model significantly outperforms all other models across all stability regimes, seasons, and times of day. Machine-learning model performance is considerably improved by including the air-sea temperature difference, which provides some accounting for offshore atmospheric stability. Finally, we find no degradation in machine-learning model performance when tested 83 km from its training location, suggesting promising future applications in extrapolating 10-m wind speeds from spatially resolved satellite-based wind atlases.

Improving Machine Learning Model Performance in Predicting the Indicated Mean Effective Pressure of a Natural Gas Engine

2020 ◽  
Author(s):  
Jinlong Liu ◽  
Christopher Ulishney ◽  
Cosmin E. Dumitrescu
Keyword(s):  
Machine Learning ◽  
Natural Gas ◽  
Decision Trees ◽  
Engine Speed ◽  
Model Performance ◽  
Learning Model ◽  
Operating Conditions ◽  
Effective Pressure ◽  
Indicated Mean Effective Pressure ◽  
Machine Learning Model

Abstract Converting existing compression ignition engines to spark ignition approach is a promising approach to increase the application of natural gas in the heavy-duty transportation sector. However, the diesel-like environment dramatically affects the engine performance and emissions. As a result, experimental tests are needed to investigate the characteristics of such converted engines. A machine learning model based on bagged decision trees algorithm was established in this study to reduce the experimental cost and identify the operating conditions of special interest for analysis. Preliminary engine tests that changed spark timing, mixture equivalence ratio, and engine speed (three key engine operation variables) but maintained intake and boundary conditions were applied as model input to train such a correlative model. The model output was the indicated mean effective pressure, which is an engine parameter generally used to assist in locating high engine efficiency regions at constant engine speed and fuel/air ratio. After training, the correlative model can provide acceptable prediction performance except few outliers. Subsequently, boosting ensemble learning approach was applied in this study to help improve the model performance. Furthermore, the results showed that the boosted decision trees algorithm better described the combustion process inside the cylinder, as least for the operating conditions investigated in this study.

scholarly journals New methods to improve the vertical extrapolation of near-surface offshore wind speeds

2021 ◽  
Vol 6 (3) ◽  
pp. 935-948
Author(s):  
Mike Optis ◽  
Nicola Bodini ◽  
Mithu Debnath ◽  
Paula Doubrawa
Keyword(s):  
Machine Learning ◽  
Wind Speed ◽  
Model Performance ◽  
Learning Model ◽  
Offshore Wind ◽  
Offshore Wind Turbine ◽  
Near Surface ◽  
Wind Speeds ◽  
Machine Learning Model ◽  
Surface Measurements

Abstract. Accurate characterization of the offshore wind resource has been hindered by a sparsity of wind speed observations that span offshore wind turbine rotor-swept heights. Although public availability of floating lidar data is increasing, most offshore wind speed observations continue to come from buoy-based and satellite-based near-surface measurements. The aim of this study is to develop and validate novel vertical extrapolation methods that can accurately estimate wind speed time series across rotor-swept heights using these near-surface measurements. We contrast the conventional logarithmic profile against three novel approaches: a logarithmic profile with a long-term stability correction, a single-column model, and a machine-learning model. These models are developed and validated using 1 year of observations from two floating lidars deployed in US Atlantic offshore wind energy areas. We find that the machine-learning model significantly outperforms all other models across all stability regimes, seasons, and times of day. Machine-learning model performance is considerably improved by including the air–sea temperature difference, which provides some accounting for offshore atmospheric stability. Finally, we find no degradation in machine-learning model performance when tested 83 km from its training location, suggesting promising future applications in extrapolating 10 m wind speeds from spatially resolved satellite-based wind atlases.

Early Detection of Readmission Risk for Decision Support Based on Clinical Notes

2021 ◽  
Vol 11 (2) ◽  
pp. 529-534
Author(s):  
Kareen Teo ◽  
Ching Wai Yong ◽  
Joon Huang Chuah ◽  
Belinda Pingguan Murphy ◽  
Khin Wee Lai
Keyword(s):  
Machine Learning ◽  
Characteristic Curve ◽  
Model Performance ◽  
Discharge Summary ◽  
Learning Model ◽  
Readmission Rates ◽  
Clinical Notes ◽  
Medical Care Costs ◽  
Machine Learning Model ◽  
Readmission Risk

Hospital readmission shortly after discharge is contributing to rising medical care costs. Attempts have been exerted to reduce readmission rates by predicting patients at high risk of this episode on the basis of unstructured clinical notes. Discharge summary as part of the clinical prose is effective at modeling readmission risk. However, the predictive value of notes written upon discharge offers few opportunities to reduce the chance of readmission because the target patient might have already been discharged. This paper presents the use of early clinical notes in building a machine learning model to predict readmission at 48 h immediately after a patient's admission. Extensive feature engineering, testing multiple algorithms, and algorithm tuning were performed to enhance model performance. A risk scoring framework that combines the data- and knowledge-driven feature scores in risk computation was developed. The proposed predictive model showed better prognostic capability than the machine learning model alone in terms of the ability to detect readmission. In specific, the proposed algorithm showed improvements of 11%–28% in sensitivity and 1%–3% in the area-under-the-receiver operating characteristic curve.

scholarly journals Early Prediction of Ventilator-associated Pneumonia in Critical Care Patients: a Machine Learning Model

2020 ◽  
Author(s):  
Yingjian Liang ◽  
Chengrui Zhu ◽  
Cong Tian ◽  
Qizhong Lin ◽  
Zhiliang Li ◽  
...  
Keyword(s):  
Machine Learning ◽  
Prediction Model ◽  
Sensitivity And Specificity ◽  
External Validation ◽  
Model Performance ◽  
Learning Model ◽  
Ventilator Associated Pneumonia ◽  
Data Set ◽  
Machine Learning Model ◽  
Mimic Iii

Abstract Background: This study was performed to develop and validate machine learning models for the early detection of ventilator-associated pneumonia (VAP) in patients 24 h before the diagnosis that enables VAP patients to receive early intervention and reduces the occurrence of complications.Patients and Methods: This study was based on the MIMIC-III dataset, which was a retrospective cohort. The random forest algorithm was applied to construct a base classifier, and the area under the receiver operating characteristic (ROC) curve (AUC), sensitivity and specificity of the prediction model were evaluated. Meanwhile, a Clinical Pulmonary Infection Score (CPIS)-based model (threshold value≥3) using the same training and test data set was used as the control model.Results: A total of 38,515 ventilation durations occurred in 61,532 ICU admissions. VAP occurred in 212 of these durations. We incorporated 42 VAP risk factors on admission and routinely measured vital characteristics and laboratory results. Five-fold cross-validation was performed to evaluate the model performance, and the model achieved an AUC of 84.4%±1.7% on validation, 74.3%±2.5% sensitivity and 70.7.6%±1.2% specificity 24 h before the gold standard time (at least 48 h after ventilation). Our VAP machine learning model improved the AUC of the CPIS-based model by almost 25%, and the sensitivity and specificity were also improved by almost 14% and 15%, respectively.Conclusions: We developed and internally validated an automated model of VAP prediction in the MIMIC-III cohort. The VAP prediction model achieved high performance for AUC, sensitivity and specificity. and its performance was superior to that of the CPIS model. External validation and prospective interventional or outcome studies using this prediction model are envisioned as future work.

Sign in / Sign up

Export Citation Format

Share Document