scholarly journals Benchmarking Triage Capability of Symptom Checkers Against That of Medical Laypersons: Survey Study (Preprint)

2020 ◽  
Author(s):  
Malte L Schmieding ◽  
Rudolf Mörgeli ◽  
Maike A L Schmieding ◽  
Markus A Feufel ◽  
Felix Balzer

BACKGROUND Symptom checkers (SCs) are tools developed to provide clinical decision support to laypersons. Apart from suggesting probable diagnoses, they commonly advise when users should seek care (<i>triage advice</i>). SCs have become increasingly popular despite prior studies rating their performance as mediocre. To date, it is unclear whether SCs can triage better than those who might choose to use them. OBJECTIVE This study aims to compare triage accuracy between SCs and their potential users (ie, laypersons). METHODS On Amazon Mechanical Turk, we recruited 91 adults from the United States who had no professional medical background. In a web-based survey, the participants evaluated 45 fictitious clinical case vignettes. Data for 15 SCs that had processed the same vignettes were obtained from a previous study. As main outcome measures, we assessed the accuracy of the triage assessments made by participants and SCs for each of the three triage levels (ie, <i>emergency care</i>, <i>nonemergency care</i>, <i>self-care)</i> and overall, the proportion of participants outperforming each SC in terms of accuracy, and the risk aversion of participants and SCs by comparing the proportion of cases that were overtriaged. RESULTS The mean overall triage accuracy was similar for participants (60.9%, SD 6.8%; 95% CI 59.5%-62.3%) and SCs (58%, SD 12.8%). Most participants outperformed all but 5 SCs. On average, SCs more reliably detected emergencies (80.6%, SD 17.9%) than laypersons did (67.5%, SD 16.4%; 95% CI 64.1%-70.8%). Although both SCs and participants struggled with cases requiring self-care (the least urgent triage category), SCs more often wrongly classified these cases as emergencies (43/174, 24.7%) compared with laypersons (56/1365, 4.10%). CONCLUSIONS Most SCs had no greater triage capability than an average layperson, although the triage accuracy of the five best SCs was superior to the accuracy of most participants. SCs might improve early detection of emergencies but might also needlessly increase resource utilization in health care. Laypersons sometimes require support in deciding when to rely on self-care but it is in that very situation where SCs perform the worst. Further research is needed to determine how to best combine the strengths of humans and SCs.

2011 ◽  
Vol 17 (7) ◽  
pp. 580-583 ◽  
Author(s):  
David D. Luxton ◽  
Christina M. Armstrong ◽  
Emily E. Fantelli ◽  
Elissa K. Thomas

2011 ◽  
Vol 29 (7_suppl) ◽  
pp. 207-207
Author(s):  
A. Gupta ◽  
S. F. Shariat ◽  
J. A. Eastham ◽  
P. T. Scardino ◽  
A. J. Vickers ◽  
...  

207 Background: PSA assays can be calibrated to either the WHO or the Hybritech standard. Studies of PSA-based prostate cancer screening have used Hybritech-standardized assays and prostate cancer risk calculators are based on these studies. Testing of patient samples with a WHO calibrated assay gives values that are 22% lower than from those with Hybritech-calibrated assays. Up to 60% of the labs in the US use WHO calibrated assays. We evaluated whether US urologists are aware of the different calibrators and the differences in PSA values. Methods: A random sample of 1,742 US urologists were invited by email to participate in a web-based survey of their knowledge and practices regarding PSA assay standardization. No mention was made of assays or calibration in the invitation. 419 responses were received. Results: Many (56%) US urologists thought that different standards may lead to clinically relevant differences in PSA values. Although 62% reported awareness of the two PSA calibrators, 67% did not know the difference between the two. Only 17% correctly reported the difference between the two standards. Nationally almost 60% of the labs use WHO standardized assays, but in this survey only 5% of the urologists thought that the hospital where they practice used a WHO standardized assay. The rest reported either not knowing the standard (46%) or use of the Hybritech standard (49%). The majority of urologists did not look at the reference range (64%) or for the PSA standard (74%) in the lab reports. Only 25% reported considering the PSA-calibration in their clinical decisions about prostate biopsy, but only a third of them correctly knew the difference between the calibrators. Conclusions: Many US urologists are unaware of the difference caused by WHO versus Hybritech based PSA-assay calibration. Although 60% of clinical laboratories use WHO-calibrated assays, only 5% of urologists are aware of this use in their practice, and a majority of urologists could not correctly explain the difference between the different calibrators. A greater awareness is needed amongst US urologists about the different PSA calibrators, the calibrator in use at their practice, and means to account for different calibrators in clinical decision making. [Table: see text]


2017 ◽  
Vol 16 (6) ◽  
pp. 767-776 ◽  
Author(s):  
Julie B. Schnur ◽  
Matthew J. Dillon ◽  
Rachel E. Goldsmith ◽  
Guy H. Montgomery

ABSTRACTObjective:Some 25% of women and 8% of men in the United States have experienced childhood sexual abuse (CSA) before the age of 18. For these individuals, healthcare visits and interactions can be retraumatizing due to perceived similarities to past abuse (e.g., pain, undressing, lack of control). However, no prior studies have provided formal qualitative analyses regarding CSA survivors' reactions to cancer treatment. Therefore, our study's objective was to identify key themes pertaining to CSA survivors' cancer treatment experiences.Method:Male and female members of the Amazon Mechanical Turk (N= 159, mean age = 44.27 years,SD= 10.02) participated in an anonymous online survey study. The inclusion criteria included reporting: history of CSA; a diagnosis of colorectal, gynecological, breast, or skin cancer; and experience of triggers and/or difficulties during cancer treatment. Participants' responses to open-ended questions were analyzed using inductive thematic analysis.Results:We identified two primary themes describing CSA survivors' experiences: Theme 1: treatment-related triggers (key subthemes: procedure-related, provider-related, and emotional triggers); and Theme 2: questioning the meaning of cumulative trauma (e.g., “Why me again?”).Significance of results:For CSA survivors, cancer and its treatment can trigger thoughts and emotions associated with the original abuse as well as negative evaluations of themselves, the world, and their future. Our findings are consistent with past research on CSA survivors' experiences in non-cancer healthcare settings and add to the literature by highlighting their struggles during cancer treatment. The present results can inform further research on trauma survivors' reactions to cancer treatment and give cancer care providers the context they need to understand and sensitively serve a substantial yet often overlooked patient group.


2019 ◽  
Author(s):  
Albert Sohn ◽  
William Speier ◽  
Esther Lan ◽  
Kymberly Aoki ◽  
Gregg Fonarow ◽  
...  

BACKGROUND Heart failure is a serious public health concern that afflicts millions of individuals in the United States. Development of behaviors that promote heart failure self-care may be imperative to reduce complications and avoid hospital re-admissions. Mobile health solutions, such as activity trackers and smartphone apps, could potentially help to promote self-care through remote tracking and issuing reminders. OBJECTIVE The objective of this study was to ascertain heart failure patients’ interest in a smartphone app to assist them in managing their treatment and symptoms and to determine factors that influence their interest in such an app. METHODS In the clinic waiting room on the day of their outpatient clinic appointments, 50 heart failure patients participated in a self-administered survey. The survey comprised 139 questions from previously published, institutional review board–approved questionnaires. The survey measured patients’ interest in and experience using technology as well as their function, heart failure symptoms, and heart failure self-care behaviors. The Minnesota Living with Heart Failure Questionnaire (MLHFQ) was among the 11 questionnaires and was used to measure the heart failure patients’ health-related quality of life through patient-reported outcomes. RESULTS Participants were aged 64.5 years on average, 32% (16/50) of the participants were women, and 91% (41/45) of the participants were determined to be New York Heart Association Class II or higher. More than 60% (30/50) of the survey participants expressed interest in several potential features of a smartphone app designed for heart failure patients. Participant age correlated negatively with interest in tracking, tips, and reminders in multivariate regression analysis (<italic>P</italic>&lt;.05). In contrast, MLHFQ scores (worse health status) produced positive correlations with these interests (<italic>P</italic>&lt;.05). CONCLUSIONS The majority of heart failure patients showed interest in activity tracking, heart failure symptom management tips, and reminder features of a smartphone app. Desirable features and an understanding of factors that influence patient interest in a smartphone app for heart failure self-care may allow researchers to address common concerns and to develop apps that demonstrate the potential benefits of mobile technology.


2020 ◽  
Author(s):  
Temiloluwa Prioleau

BACKGROUND There are still many unanswered questions about the novel coronavirus; however, a largely underutilized source of knowledge is the millions of people who have recovered after contracting the virus. This includes a majority of undocumented cases of COVID-19, which were classified as mild or moderate and received little to no clinical care during the course of illness. OBJECTIVE This study aims to document and glean insights from the experiences of individuals with a first-hand experience in dealing with COVID-19, especially the so-called mild-to-moderate cases that self-resolved while in isolation. METHODS This web-based survey study called C19 Insider Scoop recruited adult participants aged 18 years or older who reside in the United States and had tested positive for COVID-19 or antibodies. Participants were recruited through various methods, including online support groups for COVID-19 survivors, advertisement in local news outlets, as well as through professional and other networks. The main outcomes measured in this study included knowledge of contraction or transmission of the virus, symptoms, and personal experiences on the road to recovery. RESULTS A total of 72 participants (female, n=53; male, n=19; age range: 18-73 years; mean age: 41 [SD 14] years) from 22 US states were enrolled in this study. The top known source of how people contracted SARS-CoV-2, the virus known to cause COVID-19, was through a family or household member (26/72, 35%). This was followed by essential workers contracting the virus through the workplace (13/72, 18%). Participants reported up to 27 less-documented symptoms that they experienced during their illness, such as brain or memory fog, palpitations, ear pain or discomfort, and neurological problems. In addition, 47 of 72 (65%) participants reported that their symptoms lasted longer than the commonly cited 2-week period even for mild cases of COVID-19. The mean recovery time of the study participants was 4.5 weeks, and exactly one-half of participants (50%) still experienced lingering symptoms of COVID-19 after an average of 65 days following illness onset. Additionally, 37 (51%) participants reported that they experienced stigma associated with contracting COVID-19. CONCLUSIONS This study presents preliminary findings suggesting that emphasis on family or household spread of COVID-19 may be lacking and that there is a general underestimation of the recovery time even for mild cases of illness with the virus. Although a larger study is needed to validate these results, it is important to note that as more people experience COVID-19, insights from COVID-19 survivors can enable a more informed public, pave the way for others who may be affected by the virus, and guide further research.


2018 ◽  
Vol 20 (5) ◽  
pp. e187 ◽  
Author(s):  
Perrine Créquit ◽  
Ghizlène Mansouri ◽  
Mehdi Benchoufi ◽  
Alexandre Vivot ◽  
Philippe Ravaud

Background Crowdsourcing involves obtaining ideas, needed services, or content by soliciting Web-based contributions from a crowd. The 4 types of crowdsourced tasks (problem solving, data processing, surveillance or monitoring, and surveying) can be applied in the 3 categories of health (promotion, research, and care). Objective This study aimed to map the different applications of crowdsourcing in health to assess the fields of health that are using crowdsourcing and the crowdsourced tasks used. We also describe the logistics of crowdsourcing and the characteristics of crowd workers. Methods MEDLINE, EMBASE, and ClinicalTrials.gov were searched for available reports from inception to March 30, 2016, with no restriction on language or publication status. Results We identified 202 relevant studies that used crowdsourcing, including 9 randomized controlled trials, of which only one had posted results at ClinicalTrials.gov. Crowdsourcing was used in health promotion (91/202, 45.0%), research (73/202, 36.1%), and care (38/202, 18.8%). The 4 most frequent areas of application were public health (67/202, 33.2%), psychiatry (32/202, 15.8%), surgery (22/202, 10.9%), and oncology (14/202, 6.9%). Half of the reports (99/202, 49.0%) referred to data processing, 34.6% (70/202) referred to surveying, 10.4% (21/202) referred to surveillance or monitoring, and 5.9% (12/202) referred to problem-solving. Labor market platforms (eg, Amazon Mechanical Turk) were used in most studies (190/202, 94%). The crowd workers’ characteristics were poorly reported, and crowdsourcing logistics were missing from two-thirds of the reports. When reported, the median size of the crowd was 424 (first and third quartiles: 167-802); crowd workers’ median age was 34 years (32-36). Crowd workers were mainly recruited nationally, particularly in the United States. For many studies (58.9%, 119/202), previous experience in crowdsourcing was required, and passing a qualification test or training was seldom needed (11.9% of studies; 24/202). For half of the studies, monetary incentives were mentioned, with mainly less than US $1 to perform the task. The time needed to perform the task was mostly less than 10 min (58.9% of studies; 119/202). Data quality validation was used in 54/202 studies (26.7%), mainly by attention check questions or by replicating the task with several crowd workers. Conclusions The use of crowdsourcing, which allows access to a large pool of participants as well as saving time in data collection, lowering costs, and speeding up innovations, is increasing in health promotion, research, and care. However, the description of crowdsourcing logistics and crowd workers’ characteristics is frequently missing in study reports and needs to be precisely reported to better interpret the study findings and replicate them.


2017 ◽  
Author(s):  
Niam Yaraghi ◽  
Weiguang Wang ◽  
Guodong (Gordon) Gao ◽  
Ritu Agarwal

BACKGROUND In recent years, the information environment for patients to learn about physician quality is being rapidly changed by Web-based ratings from both commercial and government efforts. However, little is known about how various types of Web-based ratings affect individuals’ choice of physicians. OBJECTIVE The objective of this research was to measure the relative importance of Web-based quality ratings from governmental and commercial agencies on individuals’ choice of primary care physicians. METHODS In a choice-based conjoint experiment conducted on a sample of 1000 Amazon Mechanical Turk users in October 2016, individuals were asked to choose their preferred primary care physician from pairs of physicians with different ratings in clinical and nonclinical aspects of care provided by governmental and commercial agencies. RESULTS The relative log odds of choosing a physician increases by 1.31 (95% CI 1.26-1.37; P<.001) and 1.32 (95% CI 1.27-1.39; P<.001) units when the government clinical ratings and commercial nonclinical ratings move from 2 to 4 stars, respectively. The relative log odds of choosing a physician increases by 1.12 (95% CI 1.07-1.18; P<.001) units when the commercial clinical ratings move from 2 to 4 stars. The relative log odds of selecting a physician with 4 stars in nonclinical ratings provided by the government is 1.03 (95% CI 0.98-1.09; P<.001) units higher than a physician with 2 stars in this rating. The log odds of selecting a physician with 4 stars in nonclinical government ratings relative to a physician with 2 stars is 0.23 (95% CI 0.13-0.33; P<.001) units higher for females compared with males. Similar star increase in nonclinical commercial ratings increases the relative log odds of selecting the physician by female respondents by 0.15 (95% CI 0.04-0.26; P=.006) units. CONCLUSIONS Individuals perceive nonclinical ratings provided by commercial websites as important as clinical ratings provided by government websites when choosing a primary care physician. There are significant gender differences in how the ratings are used. More research is needed on whether patients are making the best use of different types of ratings, as well as the optimal allocation of resources in improving physician ratings from the government’s perspective.


Author(s):  
Perrine Créquit ◽  
Ghizlène Mansouri ◽  
Mehdi Benchoufi ◽  
Alexandre Vivot ◽  
Philippe Ravaud

BACKGROUND Crowdsourcing involves obtaining ideas, needed services, or content by soliciting Web-based contributions from a crowd. The 4 types of crowdsourced tasks (problem solving, data processing, surveillance or monitoring, and surveying) can be applied in the 3 categories of health (promotion, research, and care). OBJECTIVE This study aimed to map the different applications of crowdsourcing in health to assess the fields of health that are using crowdsourcing and the crowdsourced tasks used. We also describe the logistics of crowdsourcing and the characteristics of crowd workers. METHODS MEDLINE, EMBASE, and ClinicalTrials.gov were searched for available reports from inception to March 30, 2016, with no restriction on language or publication status. RESULTS We identified 202 relevant studies that used crowdsourcing, including 9 randomized controlled trials, of which only one had posted results at ClinicalTrials.gov. Crowdsourcing was used in health promotion (91/202, 45.0%), research (73/202, 36.1%), and care (38/202, 18.8%). The 4 most frequent areas of application were public health (67/202, 33.2%), psychiatry (32/202, 15.8%), surgery (22/202, 10.9%), and oncology (14/202, 6.9%). Half of the reports (99/202, 49.0%) referred to data processing, 34.6% (70/202) referred to surveying, 10.4% (21/202) referred to surveillance or monitoring, and 5.9% (12/202) referred to problem-solving. Labor market platforms (eg, Amazon Mechanical Turk) were used in most studies (190/202, 94%). The crowd workers’ characteristics were poorly reported, and crowdsourcing logistics were missing from two-thirds of the reports. When reported, the median size of the crowd was 424 (first and third quartiles: 167-802); crowd workers’ median age was 34 years (32-36). Crowd workers were mainly recruited nationally, particularly in the United States. For many studies (58.9%, 119/202), previous experience in crowdsourcing was required, and passing a qualification test or training was seldom needed (11.9% of studies; 24/202). For half of the studies, monetary incentives were mentioned, with mainly less than US $1 to perform the task. The time needed to perform the task was mostly less than 10 min (58.9% of studies; 119/202). Data quality validation was used in 54/202 studies (26.7%), mainly by attention check questions or by replicating the task with several crowd workers. CONCLUSIONS The use of crowdsourcing, which allows access to a large pool of participants as well as saving time in data collection, lowering costs, and speeding up innovations, is increasing in health promotion, research, and care. However, the description of crowdsourcing logistics and crowd workers’ characteristics is frequently missing in study reports and needs to be precisely reported to better interpret the study findings and replicate them.


2018 ◽  
Author(s):  
William Tsai ◽  
Daisy Zavala ◽  
Sol Gomez

BACKGROUND Ethnic minority cancer survivors remain an understudied and underrepresented population in cancer research, in part, due to the challenge of low participant recruitment rates. Therefore, identifying effective recruitment strategies is imperative for reducing cancer health disparities among this population. With the widespread use of social media, health researchers have turned to Facebook as a potential source of recruitment. OBJECTIVE We aimed to evaluate the feasibility and effectiveness of purchasing ads on Facebook to recruit Chinese, Korean, and Latinx cancer survivors residing in the United States. We assessed their experience with participating in a Web-based survey and their interest for future research. METHODS We showed 5 purchased ads in English, simplified Chinese, traditional Chinese, Korean, and Spanish on Facebook. Participants who clicked on the Facebook ad were directed to the study website and asked to submit their emails to receive the link to the 30-minute Web-based survey. Inclusion criteria included being of Asian or Latinx heritage, age ≥18 years, having a cancer diagnosis, and being within 5 years of cancer treatment. Participants who completed the survey were sent a US $10 Walmart eGiftcard. RESULTS The Facebook ads were shown for 48 consecutive days for a total spending of US $1200.46 (US $25/day budget). Overall, 11 East Asian and 15 Latinx cancer survivors completed the study, resulting in an average cost per participant of US $46.17. The East Asian and Latinx cancer survivors did not significantly differ in age, years lived in the United States, education level, generation status, and time since diagnosis. However, Latinx cancer survivors were marginally more likely to have limited English proficiency and lower annual income than East Asian cancer survivors. Both Latinx and East Asian cancer survivors reported that they enjoyed participating in this study and indicated an interest in participating in future psychosocial research studies. CONCLUSIONS The use of Facebook ads successfully resulted in the recruitment of East Asian and Latinx cancer survivors with different cancer diagnoses who reside in various geographic regions of the United States. We found that East Asian and Latinx cancer survivors recruited through Facebook were interested in participating in future psychosocial research, thereby providing support for the feasibility and effectiveness of using Facebook as a source of recruitment for ethnic minority cancer survivors.


2019 ◽  
Vol 10 ◽  
pp. 215013271985306 ◽  
Author(s):  
Lloyd N. Werk ◽  
Maria Carmen Diaz ◽  
Adriana Cadilla ◽  
James P. Franciosi ◽  
Md Jobayer Hossain

Objectives: In the United States, nonadherence to seasonal influenza vaccination guidelines for children and adolescents is common and results in unnecessary morbidity and mortality. We conducted a quality improvement project to improve vaccination rates and test effects of 2 interventions on vaccination guidelines adherence. Methods: We conducted a cluster randomized control trial with 11 primary care practices (PRACTICE) that provided care for 11 293 individual children and adolescents in a children’s health care system from September 2015 through April 2016. Practice sites (with their clinicians) were randomly assigned to 4 arms (no intervention [Control], computerized clinical decision support system [CCDSS], web-based training [WBT], or CCDSS and WBT [BOTH]). Results: During the study, 55.8% of children and adolescents received influenza vaccination, which improved modestly during the study period compared with the prior influenza season ( P = .009). Actual adherence to recommendations, including dosing, timeliness, and avoidance of missed opportunities, was 46.4% of patients cared for by the PRACTICE. The WBT was most effective in promoting adherence with vaccination recommendations with an estimated average odds ratio = 1.26, P < .05, to compare between preintervention and intervention periods. Over the influenza season, there was a significantly increasing trend in odds ratio in the WBT arm ( P < .05). Encouraging process improvements and providing longitudinal feedback on monthly rate of vaccination sparked some practice changes but limited impact on outcomes. Conclusions: Web-based training at the start of influenza season with monthly reports of adherence can improve correct dose and timing of influenza vaccination with modest impact on overall vaccination rate.


Sign in / Sign up

Export Citation Format

Share Document