Mobile Clinical Decision Tools Among Emergency Department Clinicians: Web-Based Survey and Analytic Data for Evaluation of The Ottawa Rules App

Background The Canadian CT Head Rule (CCHR), the Canadian Transient Ischemic Attack (TIA) Score, and the Subarachnoid Hemorrhage (SAH) Rule have all previously demonstrated the potential to significantly standardize care and improve the management of patients in emergency departments (EDs). On the basis of user feedback, we believe that the addition of these rules to the Ottawa Rules App has the potential to increase the app’s usability and user acceptability. Objective This study aimed to evaluate the perceived usefulness, acceptability, and uptake of the enhanced Ottawa Rules App (which now includes CCHR, TIA, and SAH Rules) among ED clinicians (medical students, residents, nurses, and physicians). Methods The enhanced Ottawa Rules App was publicly released for free on iOS and Android operating systems in November 2018. This study was conducted across 2 tertiary EDs in Ottawa, Canada. Posters, direct enrollment, snowball sampling, and emails were used for study recruitment. A 24-question Web-based survey was administered to participants via email, and this was used to determine user acceptability of the app and Technology Readiness Index (TRI) scores. In-app user analytics were collected to track user behavior, such as the number of app sessions, length of app sessions, frequency of rule use, and the date app was first opened. Results A total of 77 ED clinicians completed the study, including 34 nurses, 12 residents, 14 physicians, and 17 medical students completing ED rotations. The median TRI score for this group was 3.38, indicating a higher than average propensity to embrace and adopt new technologies to accomplish goals in their work or daily lives. The majority of respondents agreed or strongly agreed that the app helped participants accurately carry out the clinical rules (56/77, 73%) and that they would recommend this app to their colleagues (64/77, 83%). Feedback from study participants suggested further expansion of the app—more clinical decision rules (CDRs) and different versions of the app tailored to the clinician role. Analysis and comparison of Google Analytics data and in-app data revealed similar usage behavior among study-enrolled users and all app users globally. Conclusions This study provides evidence that using the Ottawa Rules App (version 3.0.2) to improve and guide patient care would be feasible and widely accepted. The ability to verify self-reported user data (via a Web-based survey) against server analytics data is a notable strength of this study. Participants’ continued app use and request for the addition of more CDRs warrant the further development of this app and call for additional studies to evaluate its feasibility and usability in different settings as well as assessment of clinical impact.

Mobile Clinical Decision Tools Among Emergency Department Clinicians: Web-Based Survey and Analytic Data for Evaluation of The Ottawa Rules App (Preprint)

BACKGROUND The Canadian CT Head Rule (CCHR), the Canadian Transient Ischemic Attack (TIA) Score, and the Subarachnoid Hemorrhage (SAH) Rule have all previously demonstrated the potential to significantly standardize care and improve the management of patients in emergency departments (EDs). On the basis of user feedback, we believe that the addition of these rules to the <i>Ottawa Rules App</i> has the potential to increase the app’s usability and user acceptability. OBJECTIVE This study aimed to evaluate the perceived usefulness, acceptability, and uptake of the enhanced <i>Ottawa Rules App</i> (which now includes CCHR, TIA, and SAH Rules) among ED clinicians (medical students, residents, nurses, and physicians). METHODS The enhanced <i>Ottawa Rules App</i> was publicly released for free on iOS and Android operating systems in November 2018. This study was conducted across 2 tertiary EDs in Ottawa, Canada. Posters, direct enrollment, snowball sampling, and emails were used for study recruitment. A 24-question Web-based survey was administered to participants via email, and this was used to determine user acceptability of the app and Technology Readiness Index (TRI) scores. In-app user analytics were collected to track user behavior, such as the number of app sessions, length of app sessions, frequency of rule use, and the date app was first opened. RESULTS A total of 77 ED clinicians completed the study, including 34 nurses, 12 residents, 14 physicians, and 17 medical students completing ED rotations. The median TRI score for this group was 3.38, indicating a higher than average propensity to embrace and adopt new technologies to accomplish goals in their work or daily lives. The majority of respondents agreed or strongly agreed that the app helped participants accurately carry out the clinical rules (56/77, 73%) and that they would recommend this app to their colleagues (64/77, 83%). Feedback from study participants suggested further expansion of the app—more clinical decision rules (CDRs) and different versions of the app tailored to the clinician role. Analysis and comparison of Google Analytics data and in-app data revealed similar usage behavior among study-enrolled users and all app users globally. CONCLUSIONS This study provides evidence that using the <i>Ottawa Rules App</i> (version 3.0.2) to improve and guide patient care would be feasible and widely accepted. The ability to verify self-reported user data (via a Web-based survey) against server analytics data is a notable strength of this study. Participants’ continued app use and request for the addition of more CDRs warrant the further development of this app and call for additional studies to evaluate its feasibility and usability in different settings as well as assessment of clinical impact.

Acceptability of a Mobile Clinical Decision Tool Among Emergency Department Clinicians: Development and Evaluation of The Ottawa Rules App (Preprint)

BACKGROUND The Ottawa Ankle Rules, Ottawa Knee Rule, and Canadian C-Spine Rule—together known as The Ottawa Rules—are a set of internationally validated clinical decision rules developed to decrease unnecessary diagnostic imaging in the emergency department. In this study, we sought to develop and evaluate the use of a mobile app version of The Ottawa Rules. OBJECTIVE The primary objective of this study was to determine acceptability of The Ottawa Rules app among emergency department clinicians. The secondary objective was to evaluate the effect of publicity efforts on uptake of The Ottawa Rules app. METHODS The Ottawa Rules app was developed and publicly released for free on iOS and Android operating systems in April 2016. Local and national news and academic media coverage coincided with app release. This study was conducted at a large tertiary trauma care center in Ottawa, Canada. The study was advertised through posters and electronically by email. Emergency department clinicians were approached in person to enroll via in-app consent for a 1-month study during which time they were encouraged to use the app when evaluating patients with suspected knee, foot, or neck injuries. A 23-question survey was administered at the end of the study period via email to determine self-reported frequency, perceived ease of use of the app, and participant Technology Readiness Index scores. RESULTS A total of 108 emergency department clinicians completed the study including 42 nurses, 33 residents, 20 attending physicians, and 13 medical students completing emergency department rotations. The median Technology Readiness Index for this group was 3.56, indicating a moderate degree of openness for technological adoption. The majority of survey respondents indicated favorable receptivity to the app including finding it helpful to applying the rules (73/108, 67.6%), that they would recommend the app to colleagues (81/108, 75.0%), and that they would continue using the app (73/108, 67.6%). Feedback from study participants highlighted a desire for access to more clinical decision rules and a higher degree of interactivity of the app. Between April 21, 2016, and June 1, 2017, The Ottawa Rules app was downloaded approximately 4000 times across 89 countries. CONCLUSIONS We have found The Ottawa Rules app to be an effective means to disseminate the Ottawa Ankle Rules, Ottawa Knee Rule, and Canadian C-Spine Rule among all levels of emergency department clinicians. We have been successful in monitoring uptake and access of the rules in the app as a result of our publicity efforts. Mobile technology can be leveraged to improve the accessibility of clinical decision tools to health professionals.

Can the Ottawa and Pittsburgh rules reduce requests for radiography in patients referred to acute knee clinics?

Our aim was to study the role of the Ottawa and Pittsburgh rules to reduce the unnecessary use of radiographs following knee injury. We prospectively reviewed 106 patients who were referred to our clinic over a 3-month period. The Ottawa and Pittsburgh rules were applied to individual patients to evaluate the need for radiography. One hundred and one patients (95%) had radiography of their knee. Five patients (5%) had a fracture of their knee and in all cases, the Ottawa and Pittsburgh knee rules were fulfilled. Using the Ottawa rules, 27 radiographs (25%) could have been avoided without missing a fracture. Using the Pittsburgh rules, 32 radiographs (30%) could have been avoided. The Ottawa and Pittsburgh rules have a high sensitivity for the detection of knee fractures. Their use can aid efficient clinical evaluation without adverse clinical outcome and may reduce healthcare costs.