scholarly journals Technical Metrics Used to Evaluate Health Care Chatbots: Scoping Review

10.2196/18301 ◽  
2020 ◽  
Vol 22 (6) ◽  
pp. e18301 ◽  
Author(s):  
Alaa Abd-Alrazaq ◽  
Zeineb Safi ◽  
Mohannad Alajlani ◽  
Jim Warren ◽  
Mowafa Househ ◽  
...  
Keyword(s):  
Health Care ◽  
Error Rate ◽  
Bibliographic Databases ◽  
Reference List ◽  
Word Error Rate ◽  
Classifier Performance ◽  
History Of ◽  
Usability Metrics ◽  
Survey Designs ◽  
Speed Response

Background Dialog agents (chatbots) have a long history of application in health care, where they have been used for tasks such as supporting patient self-management and providing counseling. Their use is expected to grow with increasing demands on health systems and improving artificial intelligence (AI) capability. Approaches to the evaluation of health care chatbots, however, appear to be diverse and haphazard, resulting in a potential barrier to the advancement of the field. Objective This study aims to identify the technical (nonclinical) metrics used by previous studies to evaluate health care chatbots. Methods Studies were identified by searching 7 bibliographic databases (eg, MEDLINE and PsycINFO) in addition to conducting backward and forward reference list checking of the included studies and relevant reviews. The studies were independently selected by two reviewers who then extracted data from the included studies. Extracted data were synthesized narratively by grouping the identified metrics into categories based on the aspect of chatbots that the metrics evaluated. Results Of the 1498 citations retrieved, 65 studies were included in this review. Chatbots were evaluated using 27 technical metrics, which were related to chatbots as a whole (eg, usability, classifier performance, speed), response generation (eg, comprehensibility, realism, repetitiveness), response understanding (eg, chatbot understanding as assessed by users, word error rate, concept error rate), and esthetics (eg, appearance of the virtual agent, background color, and content). Conclusions The technical metrics of health chatbot studies were diverse, with survey designs and global usability metrics dominating. The lack of standardization and paucity of objective measures make it difficult to compare the performance of health chatbots and could inhibit advancement of the field. We suggest that researchers more frequently include metrics computed from conversation logs. In addition, we recommend the development of a framework of technical metrics with recommendations for specific circumstances for their inclusion in chatbot studies.

scholarly journals Technical Metrics Used to Evaluate Health Care Chatbots: Scoping Review (Preprint)

2020 ◽  
Author(s):  
Alaa Abd-Alrazaq ◽  
Zeineb Safi ◽  
Mohannad Alajlani ◽  
Jim Warren ◽  
Mowafa Househ ◽  
...  
Keyword(s):  
Health Care ◽  
Error Rate ◽  
Bibliographic Databases ◽  
Reference List ◽  
Word Error Rate ◽  
Classifier Performance ◽  
History Of ◽  
Usability Metrics ◽  
Survey Designs ◽  
Speed Response

BACKGROUND Dialog agents (chatbots) have a long history of application in health care, where they have been used for tasks such as supporting patient self-management and providing counseling. Their use is expected to grow with increasing demands on health systems and improving artificial intelligence (AI) capability. Approaches to the evaluation of health care chatbots, however, appear to be diverse and haphazard, resulting in a potential barrier to the advancement of the field. OBJECTIVE This study aims to identify the technical (nonclinical) metrics used by previous studies to evaluate health care chatbots. METHODS Studies were identified by searching 7 bibliographic databases (eg, MEDLINE and PsycINFO) in addition to conducting backward and forward reference list checking of the included studies and relevant reviews. The studies were independently selected by two reviewers who then extracted data from the included studies. Extracted data were synthesized narratively by grouping the identified metrics into categories based on the aspect of chatbots that the metrics evaluated. RESULTS Of the 1498 citations retrieved, 65 studies were included in this review. Chatbots were evaluated using 27 technical metrics, which were related to chatbots as a whole (eg, usability, classifier performance, speed), response generation (eg, comprehensibility, realism, repetitiveness), response understanding (eg, chatbot understanding as assessed by users, word error rate, concept error rate), and esthetics (eg, appearance of the virtual agent, background color, and content). CONCLUSIONS The technical metrics of health chatbot studies were diverse, with survey designs and global usability metrics dominating. The lack of standardization and paucity of objective measures make it difficult to compare the performance of health chatbots and could inhibit advancement of the field. We suggest that researchers more frequently include metrics computed from conversation logs. In addition, we recommend the development of a framework of technical metrics with recommendations for specific circumstances for their inclusion in chatbot studies.

Stepping up detection, response, preparedness and readiness measures for “COVID-19”- A Pandemic

2020 ◽  
Vol 11 (SPL1) ◽  
pp. 1042-1047
Author(s):  
Khushbu Balsara ◽  
Deepankar Shukla
Keyword(s):  
Health Care ◽  
Health Care Workers ◽  
Clinical History ◽  
Frontline Workers ◽  
Diagnostic Assays ◽  
Care Workers ◽  
Short Period ◽  
History Of ◽  
Detection Response ◽  
Care Worker

In a very short period of time, “COVID-19” has seized the consciousness globally by making remarkable changes in our day to day living and has superintended as a public health emergency globally. It has high radar of transmission, affecting an individual at work to frontline workers. The measures and planning for a response plays a key role from drawing up an emergency committee and this follows an equation which broadly deals with epidemiological to clinical history of the patient, management steps from isolation, screening, diagnostic assays for identification and treatment. The application of an organized plan with secure structure aids in better performance, increases efficacy of management and saves time. Also saves time for a health care worker to g through routine levels of channels of administration if already a familiar way of operation is known for such situations. Thus, planning and developing a ‘blueprint of approach’ towards management of patient while facing such situation is a must. This review provides an insight to the measures for detection, response and preparedness of the hospital and health care workers should largely be inclusive of; also highlights the measures to be taken at every step after coming in contact with a positive case of “COVID-19”.

Health and Medicine in the Islamic Tradition

1990 ◽  
Vol 7 (1) ◽  
pp. 89-90
Author(s):  
Dennis Michael Warren
Keyword(s):  
Health Care ◽  
Medical Care ◽  
Human Health ◽  
Well Being ◽  
Islamic Tradition ◽  
Medical Tradition ◽  
Islamic Thought ◽  
History Of ◽  
Relationship Of ◽  
The Relationship

The late Dr. Fazlur Rahman, Harold H. Swift Distinguished Service Professor of Islamic Thought at the Oriental Institute of the University of Chicago, has written this book as number seven in the series on Health/Medicine and the Faith Traditions. This series has been sponsored as an interfaith program by The Park Ridge Center, an Institute for the study of health, faith, and ethics. Professor Rahman has stated that his study is "an attempt to portray the relationship of Islam as a system of faith and as a tradition to human health and health care: What value does Islam attach to human well-being-spiritual, mental, and physical-and what inspiration has it given Muslims to realize that value?" (xiii). Although he makes it quite clear that he has not attempted to write a history of medicine in Islam, readers will find considerable depth in his treatment of the historical development of medicine under the influence of Islamic traditions. The book begins with a general historical introduction to Islam, meant primarily for readers with limited background and understanding of Islam. Following the introduction are six chapters devoted to the concepts of wellness and illness in Islamic thought, the religious valuation of medicine in Islam, an overview of Prophetic Medicine, Islamic approaches to medical care and medical ethics, and the relationship of the concepts of birth, contraception, abortion, sexuality, and death to well-being in Islamic culture. The basis for Dr. Rahman's study rests on the explication of the concepts of well-being, illness, suffering, and destiny in the Islamic worldview. He describes Islam as a system of faith with strong traditions linking that faith with concepts of human health and systems for providing health care. He explains the value which Islam attaches to human spiritual, mental, and physical well-being. Aspects of spiritual medicine in the Islamic tradition are explained. The dietary Jaws and other orthodox restrictions are described as part of Prophetic Medicine. The religious valuation of medicine based on the Hadith is compared and contrasted with that found in the scientific medical tradition. The history of institutionalized medical care in the Islamic World is traced to awqaf, pious endowments used to support health services, hospices, mosques, and educational institutions. Dr. Rahman then describes the ...

scholarly journals Seroprevalence of SARS-CoV-2 Antibodies Among Health Care Personnel at a Health Care System in Pakistan

2021 ◽  
pp. 101053952110110
Author(s):  
Salma Abbas ◽  
Aun Raza ◽  
Ayesha Iftikhar ◽  
Aamir Khan ◽  
Shahzaib Khan ◽  
...  
Keyword(s):  
Health Care ◽  
Control Measures ◽  
Health Care Personnel ◽  
Test Results ◽  
Antibody Testing ◽  
Chain Reaction ◽  
Past Infection ◽  
Infection Control Measures ◽  
History Of ◽  
Polymerase Chain

Health care personnel (HCP) are at high risk for coronavirus disease-2019 acquisition. Serum antibodies against severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) indicate past infection. Our institution offered SARS-CoV-2 antibody testing to HCP. We surveyed HCP with positive test results to explore past exposure to SARS-CoV-2, details of symptoms during the preceding 6 months, and a history of SARS-CoV-2 polymerase chain reaction testing. A total of 2162 HCP underwent antibody testing. Eight hundred fifty-seven (39.6%) employees tested positive and, of these, 820 (95.7%) participated in the survey. When adjusted for age, males had higher odds of testing positive for SARS-CoV-2 antibodies compared with females (OR = 1.68; 95% CI = 1.37-2.05; P = .00) and clinical staff had higher odds of SARS-CoV-2 seropositivity compared with nonclinical staff (OR = 1.273; 95% CI = 1.06-1.53; P = .01). Implementation of effective infection control measures is essential to protect HCP from coronavirus disease-2019.

scholarly journals Canada’s history of failing to provide medical care for Indigenous children

2020 ◽  
Author(s):  
Rakhshan Kamran
Keyword(s):  
Health Care ◽  
Human Rights ◽  
First Nations ◽  
Social Services ◽  
Current Status ◽  
House Of Commons ◽  
Canadian Government ◽  
Indigenous Children ◽  
History Of ◽  
Services And Supports

Abstract In December 2007, the House of Commons unanimously supported Jordan’s Principle, a commitment that all First Nations children would receive the health care products, social services, and supports, and education they need, in memory of Jordan River Anderson. However, the process of applying for Jordan’s Principle was convoluted and not transparent, leaving several cases not being responded to. The Canadian Human Rights Tribunal found the definition and implementation of Jordan’s Principle to be racist and discriminatory in 2016, ordering the Canadian government to make immediate changes. Failing to make changes to Jordan’s Principle, the Canadian government was found to be noncompliant with the Canadian Human Rights Tribunal orders in 2018. This article provides one case example of Jordan’s Principle that was not responded to, details on the current status of Jordan’s Principle, and information on the recent implementation of the Act respecting First Nations, Inuit and Métis children, youth and families.

Roles of Psychological Distress and Social Support in the Relationship Between Childhood Maltreatment and Perceived Needs for Mental Health Care

2021 ◽  
pp. 088626052110063
Author(s):  
Tingting Gao ◽  
Songli Mei ◽  
Muzi Li ◽  
Carl D’ Arcy ◽  
Xiangfei Meng
Keyword(s):  
Mental Health ◽  
Social Support ◽  
Health Care ◽  
Psychological Distress ◽  
Mental Health Care ◽  
Childhood Maltreatment ◽  
Perceived Needs ◽  
Perceived Need ◽  
History Of ◽  
The Relationship

Childhood maltreatment is a major public health issue worldwide. It increases a range of health-risk behaviors, psychological and physical problems, which are associated with an increased need for mental health services in adulthood. Identification of mediating factors in the relationship between maltreatment and seeking mental health care may help attenuate the negative consequences of childhood maltreatment and promote more appropriate treatment. This study aims to examine whether the relationship between childhood maltreatment and perceived need for mental health care is mediated by psychological distress and/or moderated by social support. Data from the Canadian Community Health Survey-Mental Health 2012 are analyzed. A total of 8,993 participants, who had complete information on childhood maltreatment and diagnoses of mental disorders or psychological distress, are included in this study. Structural equation modeling and the PROCESS macro were used to identify relationships among childhood maltreatment, perceived needs for mental health care, and psychological distress. Hierarchical linear regression was then used to verify the moderated mediation model. We found that psychological distress partially mediated the effect of childhood maltreatment on perceived needs for mental health care in adulthood. Social support played an important role in terms of moderating the relationship between maltreatment and perceived needs for care. For those with a history of childhood maltreatment, those who perceived a low level of social support were more likely to have higher levels of psychological distress and perceived need for mental health care. This is the first study to identify the separate and combined roles of psychological distress and social support in the relationship between childhood maltreatment and perceived need for mental health care. Selective prevention strategies should focus on social support to improve mental health services among people with a history of childhood maltreatment.

Hypnotherapy as an Adjunct to Medical Treatment: Highlighting Effectiveness and Identifying Barriers to Further Integrative Treatment

2021 ◽  
pp. 097206342110115
Author(s):  
Feryad A. Hussain
Keyword(s):  
Health Care ◽  
Medical Treatment ◽  
Health Care Services ◽  
Well Being ◽  
Medical Treatments ◽  
Care Services ◽  
Secondary Health Care ◽  
History Of ◽  
Clinical Hypnosis ◽  
The Mind

Integrative models of health care have garnered increasing attention over the years and are currently being employed within acute and secondary health care services to support medical treatments in a range of specialities. Clinical hypnosis has a history of working in partnership with medical treatments quite apart from its psychiatric associations. It aims to mobilise the mind–body connection in order to identify and overcome obstacles to managing symptoms of ill health, resulting in overall improved emotional and physical well-being. This article aims to encourage the use of hypnotherapy in physical health care by highlighting the effectiveness of hypnosis as an adjunct to medical treatment and identifying barriers preventing further integrative treatments.

scholarly journals Dynamic Acoustic Unit Augmentation with BPE-Dropout for Low-Resource End-to-End Speech Recognition

Sensors ◽  
2021 ◽  
Vol 21 (9) ◽  
pp. 3063
Author(s):  
Aleksandr Laptev ◽  
Andrei Andrusenko ◽  
Ivan Podluzhny ◽  
Anton Mitrofanov ◽  
Ivan Medennikov ◽  
...  
Keyword(s):  
Speech Recognition ◽  
Error Rate ◽  
Rapid Development ◽  
Computational Cost ◽  
Vocabulary Size ◽  
Word Error Rate ◽  
Low Resource ◽  
Steady Improvement ◽  
End To End ◽  
Asr System

With the rapid development of speech assistants, adapting server-intended automatic speech recognition (ASR) solutions to a direct device has become crucial. For on-device speech recognition tasks, researchers and industry prefer end-to-end ASR systems as they can be made resource-efficient while maintaining a higher quality compared to hybrid systems. However, building end-to-end models requires a significant amount of speech data. Personalization, which is mainly handling out-of-vocabulary (OOV) words, is another challenging task associated with speech assistants. In this work, we consider building an effective end-to-end ASR system in low-resource setups with a high OOV rate, embodied in Babel Turkish and Babel Georgian tasks. We propose a method of dynamic acoustic unit augmentation based on the Byte Pair Encoding with dropout (BPE-dropout) technique. The method non-deterministically tokenizes utterances to extend the token’s contexts and to regularize their distribution for the model’s recognition of unseen words. It also reduces the need for optimal subword vocabulary size search. The technique provides a steady improvement in regular and personalized (OOV-oriented) speech recognition tasks (at least 6% relative word error rate (WER) and 25% relative F-score) at no additional computational cost. Owing to the BPE-dropout use, our monolingual Turkish Conformer has achieved a competitive result with 22.2% character error rate (CER) and 38.9% WER, which is close to the best published multilingual system.

scholarly journals Carcinoembryonic Antigen Elevation in a Patient with SARS-CoV-2 and a History of Colon Cancer

2021 ◽  
pp. 802-805
Author(s):  
Taha Alrifai ◽  
Anita Sharko ◽  
William T. Leslie
Keyword(s):  
Health Care ◽  
Colon Cancer ◽  
Severe Acute Respiratory Syndrome ◽  
Carcinoembryonic Antigen ◽  
Health Care Delivery ◽  
Care Delivery ◽  
Prognostic Biomarkers ◽  
Diagnosis And Treatment ◽  
Routine Surveillance ◽  
History Of

The severe acute respiratory syndrome-coronavirus-2 (SARS-CoV-2) pandemic continues to affect millions of people and impact health-care delivery worldwide. New data and insights into the diagnosis and treatment of SARS-CoV-2 are emerging rapidly. Several prognostic biomarkers have been studied in patients with SARS-CoV-2. Among those biomarkers is the carcinoembryonic antigen (CEA). We report the case of a 46-year-old male with a history of colon cancer who was found to have an elevated CEA level during routine surveillance. Further workup confirmed a diagnosis of SARS-CoV-2 infection. The CEA level normalized with the resolution of the infection.

Sign in / Sign up

Export Citation Format

Share Document