Enriching the value of patient experience feedback: interactive dashboard development using co-design and heuristic evaluation (Preprint)

2021 ◽  
Author(s):  
Mustafa Khanbhai ◽  
Joshua Symons ◽  
Kelsey Flott ◽  
Stephanie Harrison-White ◽  
Jamie Spofforth ◽  
...  
Keyword(s):  
Patient Experience ◽  
User Satisfaction ◽  
Care Delivery ◽  
Healthcare Providers ◽  
National Policy ◽  
Usability Testing ◽  
Visual Display ◽  
Free Text ◽  
Heuristic Evaluation ◽  
Usability Problem

BACKGROUND There is an abundance of patient experience data held within healthcare organisations but stakeholders and staff are often unable to use the output in a meaningful and timely way to improve care delivery. Dashboards, which use visualised data to summarise key patient experience feedback, have the potential to address these issues. OBJECTIVE The aim of this study was to develop a patient experience dashboard with an emphasis on FFT reporting as per the national policy drive. An iterative process involving co-design involving key stakeholders was used to develop the dashboard, followed by heuristic usability testing. METHODS A two staged approach was employed; participatory co-design involving 20 co-designers to develop a dashboard prototype followed by iterative dashboard testing. Language analysis was performed on free-text patient experience data from the Friends and Family Test (FFT) and the themes and sentiment generated was used to populate the dashboard with associated FFT metrics. Heuristic evaluation and usability testing were conducted to refine the dashboard and assess user satisfaction using the system usability score (SUS). RESULTS Qualitative analysis from the co-design process informed development of the dashboard prototype with key dashboard requirements and a significant preference for bubble chart display. Heuristic evaluation revelated the majority of cumulative scores had no usability problem (n=18), cosmetic problem only (n=7), or minor usability problem (n= 5). Mean SUS was 89.7 (SD 7.9) suggesting an excellent rating. CONCLUSIONS The growing capacity to collect and process patient experience data suggests that data visualisation will be increasingly important in turning the feedback into improvements to care. Through heuristic usability we demonstrated that very large FFT data can be presented into a thematically driven, simple visual display without loss of the nuances and still allow for exploration of the original free-text comments. This study establishes guidance for optimising the design of patient experience dashboards that healthcare providers find meaningful which in-turn drives patient-centred care.

scholarly journals Associations Between Nonclinical Services and Patient-Experience Outcomes in US Acute Care Hospitals

2019 ◽  
Vol 7 (6) ◽  
pp. 1086-1093
Author(s):  
Melody K Schiaffino ◽  
Yukari Suzuki ◽  
Tarryn Ho ◽  
Tracy L Finlayson ◽  
Jeffrey S Harman
Keyword(s):  
Patient Education ◽  
Patient Experience ◽  
Care Delivery ◽  
Healthcare Providers ◽  
Organizational Factors ◽  
Relevant Information ◽  
Full Time ◽  
Cross Sectional ◽  
Logistic Regression Models ◽  
Experience Of Care

Background: Patient experience is an important measure of hospital quality and performance. Since the passage of the Affordable Care Act, patient experiences with their care encounters are embedded into the framework of payment incentives. However, drivers of patient experience in the context of the supportive, nonclinical, services that relate to patient care have not been as well understood. Aims: To assess the role of organizational factors on patient experience. Methods: This cross-sectional analysis integrates hospital patient-experience scores from Hospital Consumer Assessment of Healthcare Providers and Systems, and Centers for Medicaid and Medicare Service data from 2013 to 2015 (N = 3392). Based on hospitals with “top-box” responses, the aggregate proportion of hospital patients responding “always” on a Likert scale represented a top-box hospital. Domains were split at the mean for analysis (above average = 1). Multivariable logistic regression models for each domain were analyzed against hospital factors and services, including offering a patient education center, patient-enabling services, and language services. Results: Most hospitals reported a full-time hospitalist (64.4%) and a patient education center (60.4%), while fewer provided enabling/support services (33.7%). In multivariable models, small and medium hospitals performed better compared to the largest hospitals (300+ beds; P < .0001). Structurally, medium and small hospitals reported significantly greater odds of top-box patient-experience versus large hospitals. Across all domains, only hospitals with patient education centers returned better performance (adjusted odds ratio: 1.27-1.64; P = .0002-.0166). Discussion/Conclusion: Patient education centers provide relevant information at the point of service and may improve overall patient experience of care. Given the growing reliance on accountable care delivery models, opportunities to partner with community health education partners may be profitable.

Demographic, socioeconomic, and clinical factors associated with oncology patient experience in the Ontario cancer system.

2020 ◽  
Vol 38 (29_suppl) ◽  
pp. 160-160
Author(s):  
Mary Mahler ◽  
Brett Nicholls ◽  
Catherine Chan ◽  
Narges Nazeri-Rad ◽  
Kelvin K. Chan ◽  
...  
Keyword(s):  
Patient Experience ◽  
Cancer Care ◽  
Care Delivery ◽  
Healthcare Providers ◽  
Oncology Patient ◽  
Negative Experience ◽  
Care Experience ◽  
Cancer Services ◽  
Patient Reported ◽  
Disease Site

160 Background: Your Voice Matters (YVM) is an electronic real time Patient Reported Experience Measure (PREM) collected on all adult patients accessing cancer services in Ontario by Cancer Care Ontario/Ontario Health. To our knowledge, this is the largest oncology PREM dataset worldwide. A total of 25 Items are assessed including care coordination, wait times, access to care and satisfaction with healthcare providers. We attempted to identify demographic, socioeconomic and disease factors that predict for a positive patient experience. Methods: Responses were collated from 18 individual cancer centers between Jan 2017 and Dec 2019. Each item was dichotomized into positive/negative experiences. Multivariable logistic regression was constructed for each of the 25 items. Results: Demographics are described in table. Males (OR=1.11, p=0.0032), genitourinary patients (OR=1.30, p=0.0031) and those receiving radiation (OR=1.39, p=<0.0001) were more likely to have a positive experience. Patients aged 18-39 (OR=0.74, p=<0.0001), receiving chemotherapy (OR=0.76, p=0.0002), and with central nervous system or lung cancer (OR= 0.56, p=<0.001; OR=0.79 p=0.0059, respectively) were more likely to have a negative experience. Lowest income patients were more likely to have a negative experience with healthcare providers (OR=0.83, p=0.0119) and patients from the highest immigration areas had a worse quality experience (OR=0.82, p=0.0029). Conclusions: Age, sex, disease site, visit type, income and immigration status significantly influence patients cancer care experience. This information helps promote equity and the use of PREM data to improve cancer care delivery. [Table: see text]

Formative Usability Evaluation of a Novel Tool for Medication Reconciliation

2017 ◽  
Vol 61 (1) ◽  
pp. 602-602 ◽  
Author(s):  
Alissa L. Russ ◽  
Michelle A. Jahn ◽  
Himalaya Patel ◽  
Brian W. Porter ◽  
Khoa A. Nguyen ◽  
...  
Keyword(s):  
Medication Errors ◽  
Phase Ii ◽  
User Satisfaction ◽  
Medication Reconciliation ◽  
Healthcare Professionals ◽  
Usability Testing ◽  
Usability Evaluation ◽  
Heuristic Evaluation ◽  
Efficiency Gains ◽  
Design Changes

To decrease medication errors, a common cause of injury to patients, we developed a novel electronic tool to facilitate asynchronous communication between healthcare professionals (HCPs) and patients for medication reconciliation. However, it was unknown whether the tool adequately supported HCPs’ usability needs. Our objective was to conduct an iterative usability evaluation of the tool with physicians, nurses, and pharmacists, in preparation for a randomized controlled trial. We hypothesized that we would identify design weaknesses that could be addressed via interface modifications prior to the trial. We completed a mixed-method, formative usability evaluation with 20 HCPs in the Veterans Affairs (VA) Health Services Research and Development, Human-Computer Interaction and Simulation Laboratory located within a major medical center. The tool in this study is formally known as the Secure Messaging for Medication Reconciliation Tool (SMMRT). The evaluation consisted of four sequential steps: 1) phase I usability testing to assess the baseline tool along with small, iterative design changes throughout testing; 2) heuristic evaluation; 3) implement major design changes that incorporate findings from previous steps; and 4) phase II usability testing to assess the implemented design changes and further refine the tool. This presentation focuses on steps 1 and 4 related to usability testing. During testing, HCPs worked through a real case consisting of a patient discharged from the hospital within the past 30 days who had at least 5 outpatient medications. We collected data on efficiency, usability errors, and participants’ satisfaction, along with participants’ ability to detect and address three distinct types of medication errors via the tool. For the latter, we inserted three safety probes into the simulation: 1) a missing medication (i.e., omission); 2) an extraneous medication (i.e., commission); and 3) an inaccurate dose (i.e., dose discrepancy). Data were analyzed descriptively, rather than via statistical comparisons, due to the formative and iterative nature of this research. There was no indication of efficiency gains during iterative prototyping and testing. Highlights of usability errors included confusion about medication entry fields; incorrect assumptions regarding medication list accuracy; inadequate medication information sorting and organization; and premature closure. Additionally, HCPs described usability errors that might occur in clinical practice. For example, medication images on the tool may not match what is dispensed to patients. HCPs also expressed concern that medication updates made via the tool may not be consistently updated in the electronic health record. In terms of satisfaction, HCPs’ ratings tended to increase as design modifications were implemented. After phase II usability testing, their overall satisfaction was favorable. Finally, for each of the three safety probes, 50% or fewer of HCPs identified the associated medication error. This research illustrates the importance of usability evaluations as a precursor to randomized trials of health information technology. Our multi-step approach to usability testing, with heuristic evaluation at the midpoint, may inform the design of other usability evaluations. While efficiency gains were not realized, user satisfaction improved. The inclusion of safety probes was especially valuable, since probes allowed us to assess error detection rates. There may be opportunities for human factors professionals to expand the sophistication and types of probes used in future healthcare research. Future studies are needed to develop more advanced design approaches that facilitate healthcare professionals’ detection of medication errors.

scholarly journals Patient Experience in Primary Care: A Systematic Review of CG-CAHPS Surveys

2018 ◽  
Vol 6 (2) ◽  
pp. 93-102 ◽  
Author(s):  
Jeana M Holt
Keyword(s):  
Systematic Review ◽  
Primary Care ◽  
Health Care ◽  
Health Care System ◽  
Patient Experience ◽  
Care Delivery ◽  
Healthcare Providers ◽  
Empirical Literature ◽  
Patient Reports ◽  
Care System

The National Academy of Medicine’s (NAM) vision for 21st-century health care underscored the need for increased patient engagement and charged health-care researchers to develop tools to evaluate patient experience. The most widely studied patient experience tools are the Consumer Assessments of Healthcare Providers and Systems (CAHPS) surveys. The Clinician and Group (CG)-CAHPS survey is the preferred patient experience survey for primary care, and thus a systematic review of patient reports from the CG-CAHPS empirical literature is ideal to appreciate the voice of health-care consumers. This systematic review revealed patient subjective reports regarding the acceptability of health-care delivery models, the effectiveness of interventions, the timeliness of care in different practice climates, and their responses to quality improvement initiatives. The synthesized results inform clinicians, organizations, and the health-care system where to prioritize and how to adapt services to efficiently provide equitable care, achieving the NAM’s vision for a patient-centered US health-care system.

Web-Based Patient Experience Surveys to Enhance Response Rates

2017 ◽  
Vol 107 (6) ◽  
pp. 516-521
Author(s):  
Jonathan Labovitz ◽  
Neil Patel ◽  
Israel Santander
Keyword(s):  
Patient Experience ◽  
Response Rate ◽  
Care Delivery ◽  
Effective Means ◽  
Healthcare Providers ◽  
Cost Effective ◽  
Response Rates ◽  
Web Based ◽  
Survey Administration ◽  
E Mail

Background: Patient experience is a critical component of continuous quality improvement and value-based health-care delivery. This study aimed to identify a simple, cost-effective means of administering a validated patient experience survey in ambulatory-care settings. Methods: Patients were randomly assigned to groups to complete the validated Consumer Assessment of Healthcare Providers and Systems (CAHPS) clinician and group patient satisfaction survey using a Web-based platform. The response rate was assessed for in-clinic and e-mail distribution and was compared with the historical response rates. Patients were able to change groups after randomization to assess effect on response rate and potential bias. The cost of survey administration was compared. Results: Of 132 participants, 87 completed surveys (65.9%), with no significant differences among distribution methods. Twenty-three participants self-selected the in-clinic survey after being randomized to the e-mail cohort. Survey responses were statistically significantly different in only three of 34 questions. Web-based survey administration costs two to four times less than standard mail, phone, and mixed-modal survey administration. Conclusions: We recommend that ambulatory clinics use Web-based technology to administer CAHPS clinician and group surveys, using both e-mail and in-clinic distribution to enhance the response rate.

scholarly journals Delivering direct patient care in the haemodialysis unit: a focused ethnographic study of care delivery

2019 ◽  
Vol 24 (8) ◽  
pp. 712-725
Author(s):  
Alison F Wood ◽  
Jennifer Tocher ◽  
Sheila Rodgers
Keyword(s):  
Patient Care ◽  
Patient Experience ◽  
Care Delivery ◽  
Healthcare Providers ◽  
Ethnographic Study ◽  
Direct Patient Care ◽  
Structured Interviews ◽  
Staff Perceptions ◽  
Clinical Support ◽  
Patient Care Delivery

Background Direct patient care is a term used within nursing and healthcare to help quantify and qualify care delivery. Direct patient care time is considered as a valuable measure by healthcare providers to indicate efficiency and to quantify nursing work, however little is understood of the patient experience and care delivery in haemodialysis settings. Aim The aim of this study was to gain an understanding of patients’ and nursing staff perceptions and experiences of ‘direct patient care’ within one haemodialysis unit. Methods A focused ethnographic approach utilised participant observations, informal questioning, photographs and 27 semi-structured interviews of registered nurses, clinical support workers and patients. Observation notes and interviews were transcribed and thematically analysed. Results The key finding was the construction and reconceptualisation of care delivery in this setting. Care was identified to be delivered in two distinct ways, both of which allowed patients to feel cared for. ‘Active care’ where patients feel cared for when they are being dealt with directly by staff and ‘Passive care’ where patients feel cared for through staff availability and visibility. Conclusion Developing this understanding of patient care delivery in this specialism has highlighted some important aspects to the way care can be delivered which challenges current traditional understandings of direct patient care. Time spent with a patient is not the only important consideration to patient experience in haemodialysis. This understanding of passive care could improve care experiences in this setting.

scholarly journals Benefits and Challenges of Using Virtual Primary Care During the COVID-19 Pandemic: From Key Lessons to a Framework for Implementation

2021 ◽  
Author(s):  
Edmond C Li ◽  
Rosy Tsopra ◽  
Geronimo Larrain Gimenez ◽  
Alice Serafini ◽  
Gustavo Gusso ◽  
...  
Keyword(s):  
Primary Care ◽  
Care Delivery ◽  
Healthcare Providers ◽  
Lessons Learned ◽  
Free Text ◽  
Online Questionnaire ◽  
Face To Face ◽  
Need To Evaluate ◽  
Barriers And Challenges ◽  
Remote Care

Background: With the onset of COVID-19, general practitioners (GPs) and patients worldwide swiftly transitioned from face-to-face to digital remote consultations. There is a need to evaluate how this global shift has impacted patient care, healthcare providers, patient and carer experience, and health systems. Objective: We explored GPs' perspectives on the main benefits and challenges of using digital remote care. Methods: GPs across 20 countries completed an online questionnaire between June - September 2020. GPs' perceptions on main barriers and challenges were explored using free-text questions. Thematic analysis was used to analyse the data. Results: 1,605 respondents participated in our survey. The benefits identified included reducing COVID-19 transmission risks, guaranteeing access and continuity of care, improved efficiency, faster access to care, improved convenience and communication with patients, greater work flexibility for providers, and hastening the digital transformation of primary care and the accompanying legal frameworks. Main challenges included patient's preference for face-to-face consultations, digital exclusion, lack of physical examinations, clinical uncertainty, delays in diagnosis and treatment, overuse and misuse of digital remote care, and unsuitability for certain types of consultations. Other challenges include the lack of formal guidance, higher workloads, remuneration issues, organisational culture, technical difficulties, implementation and financial issues, and regulatory weaknesses. Conclusion: At the frontline of care delivery, GPs can provide important insights on what worked well, why, and how. Lessons learned during the emergency phase can be used to inform the stable adoption of virtual care solutions, and co-design processes and platforms that are technologically robust, secure, and supported by a strategic long-term plan.

HEURISTIC EVALUATION FOR MOBILE APPLICATION (STUDI KASUS: APLIKASI DEPO AUTO 2000 TANJUNG API API PALEMBANG)

2017 ◽  
Vol 8 (2) ◽  
pp. 563 ◽  
Author(s):  
Usman Ependi
Keyword(s):  
Real World ◽  
Mobile Application ◽  
Usability Testing ◽  
Heuristic Evaluation ◽  
End User ◽  
Error Prevention ◽  
The Real ◽  
User Control ◽  
Cosmetic Problem

Heuristic evaluation merupakan salah satu bentuk usability testing perangkat lunak yang dinilai oleh pengguna (evaluator). Dalam melakukan heuristic evaluation instrumen penilaian terdiri dari sepuluh (10) pernyataan dengan lima pilihan jawaban dalam skala severity ratings. Dalam penelitian ini heuristic evaluation terhadap aplikasi Depo Auto 2000 Tanjung Api-Api Palembang yang dilakukan oleh 4 evaluator.  Hasil dari heuristic evaluation dikelompokkan kedalam  masing-masing instrumen yaitu visibility of system status dengan nilai 0,75, match between system and the real world dengan nilai 0,25, user control and freedom dengan nilai 0,25, consistency and standards dengan nilai 0,75, error prevention dengan nilai 1, recognition rather than recall dengan nilai 1,25, flexibility and efficiency of use dengan nilai 0,25, Aesthetic and minimalist design dengan nilai 0,25, help users recognize, diagnose, and recover from errors dengan nilai 1 dan Help and documentation dengan nilai 0. Dari hasil heuristic evaluation yang dilakukan menunjukkan bahwa evaluator memberikan nilai 0 dan 1 aplikasi Depo Atuo 2000 Tanjung Api-Api Palembang. Hasil penilaian tersebut menunjukkan bahwa aplikasi yang buat tidak ada masalah usability dan hanya memiliki cosmetic problem sehingga aplikasi Depo Auto 2000 Tanjung Api Api Palembang  dapat dinyatakan layak untuk didistribusikan kepada pengguna akhir (end user). 

scholarly journals Applying natural language processing and machine learning techniques to patient experience feedback: a systematic review

2021 ◽  
Vol 28 (1) ◽  
pp. e100262
Author(s):  
Mustafa Khanbhai ◽  
Patrick Anyadi ◽  
Joshua Symons ◽  
Kelsey Flott ◽  
Ara Darzi ◽  
...  
Keyword(s):  
Machine Learning ◽  
Systematic Review ◽  
Social Media ◽  
Natural Language Processing ◽  
Natural Language ◽  
Patient Experience ◽  
Language Processing ◽  
Performance Metrics ◽  
Free Text ◽  
Patient Feedback

ObjectivesUnstructured free-text patient feedback contains rich information, and analysing these data manually would require a lot of personnel resources which are not available in most healthcare organisations.To undertake a systematic review of the literature on the use of natural language processing (NLP) and machine learning (ML) to process and analyse free-text patient experience data.MethodsDatabases were systematically searched to identify articles published between January 2000 and December 2019 examining NLP to analyse free-text patient feedback. Due to the heterogeneous nature of the studies, a narrative synthesis was deemed most appropriate. Data related to the study purpose, corpus, methodology, performance metrics and indicators of quality were recorded.ResultsNineteen articles were included. The majority (80%) of studies applied language analysis techniques on patient feedback from social media sites (unsolicited) followed by structured surveys (solicited). Supervised learning was frequently used (n=9), followed by unsupervised (n=6) and semisupervised (n=3). Comments extracted from social media were analysed using an unsupervised approach, and free-text comments held within structured surveys were analysed using a supervised approach. Reported performance metrics included the precision, recall and F-measure, with support vector machine and Naïve Bayes being the best performing ML classifiers.ConclusionNLP and ML have emerged as an important tool for processing unstructured free text. Both supervised and unsupervised approaches have their role depending on the data source. With the advancement of data analysis tools, these techniques may be useful to healthcare organisations to generate insight from the volumes of unstructured free-text data.

Sign in / Sign up

Export Citation Format

Share Document