A Delphi approach to acquiring knowledge from single and multiple experts

1990 ◽  
Author(s):  
Roberta M. Roth ◽  
William C. Wood
Keyword(s):  
Delphi Approach ◽  
Multiple Experts

Antimicrobial resistance in cystic fibrosis: A Delphi approach to defining best practices

2020 ◽  
Vol 19 (3) ◽  
pp. 370-375 ◽  
Author(s):  
Edith Zemanick ◽  
Pierre–Régis Burgel ◽  
Giovanni Taccetti ◽  
Alison Holmes ◽  
Felix Ratjen ◽  
...  
Keyword(s):  
Cystic Fibrosis ◽  
Antimicrobial Resistance ◽  
Best Practices ◽  
Delphi Approach

A Novel Thyroid Ultrasound Proficiency Metric Designed Through a Multidisciplinary Delphi Approach

2021 ◽  
pp. 000313482110111
Author(s):  
Yinin Hu ◽  
Alex D. Michaels ◽  
Rachita Khot ◽  
Worthington G. Schenk ◽  
John B. Hanks ◽  
...  
Keyword(s):  
Learning Curves ◽  
Delphi Panel ◽  
Thyroid Ultrasound ◽  
Point Scale ◽  
Image Capture ◽  
Delphi Approach ◽  
Inflection Points ◽  
Operative Planning ◽  
Positioning Technique ◽  
Formally Trained

Background Thyroid ultrasounds extend surgeons’ outpatient capabilities and are essential for operative planning. However, most residents are not formally trained in thyroid ultrasound. The purpose of this study was to create a novel thyroid ultrasound proficiency metric through a collaborative Delphi approach. Methods Clinical faculty experienced in thyroid ultrasound participated on a Delphi panel to design the thyroid Ultrasound Proficiency Scale (UPS-Thyroid). Participants proposed items under the categories of Positioning, Technique, Image Capture, Measurement, and Interpretation. In subsequent rounds, participants voted to retain, revise, or exclude each item. The process continued until all items had greater than 70% consensus for retention. The UPS-Thyroid was pilot tested across 5 surgery residents with moderate ultrasound experience. Learning curves were assessed with cumulative sum. Results Three surgeons and 4 radiologists participated on the Delphi panel. Following 3 iterative Delphi rounds, the panel arrived at >70% consensus to retain 14 items without further revisions or additions. The metric included the following items on a 3-point scale for a maximum of 42 points: Positioning (1 item), Technique (4 items), Image Capture (2 items), Measurement (2 items), and Interpretation (5 items). A pilot group of 5 residents was scored against a proficiency threshold of 36 points. Learning curve inflection points were noted at between 4 to 7 repetitions. Conclusions A multidisciplinary Delphi approach generated consensus for a thyroid ultrasound proficiency metric (UPS-Thyroid). Among surgery residents with moderate ultrasound experience, basic proficiency at thyroid ultrasound is feasible within 10 repetitions.

scholarly journals Dealing with multiple experts and non-stationarity in inverse reinforcement learning: an application to real-life problems

2021 ◽  
Author(s):  
Amarildo Likmeta ◽  
Alberto Maria Metelli ◽  
Giorgia Ramponi ◽  
Andrea Tirinzoni ◽  
Matteo Giuliani ◽  
...  
Keyword(s):  
Reinforcement Learning ◽  
Real World ◽  
Real Life ◽  
User Preferences ◽  
Inverse Reinforcement Learning ◽  
Water Release ◽  
Reward Function ◽  
Model Free ◽  
Conflicting Objectives ◽  
Multiple Experts

AbstractIn real-world applications, inferring the intentions of expert agents (e.g., human operators) can be fundamental to understand how possibly conflicting objectives are managed, helping to interpret the demonstrated behavior. In this paper, we discuss how inverse reinforcement learning (IRL) can be employed to retrieve the reward function implicitly optimized by expert agents acting in real applications. Scaling IRL to real-world cases has proved challenging as typically only a fixed dataset of demonstrations is available and further interactions with the environment are not allowed. For this reason, we resort to a class of truly batch model-free IRL algorithms and we present three application scenarios: (1) the high-level decision-making problem in the highway driving scenario, and (2) inferring the user preferences in a social network (Twitter), and (3) the management of the water release in the Como Lake. For each of these scenarios, we provide formalization, experiments and a discussion to interpret the obtained results.

scholarly journals A consensus approach toward the standardization of spinal stiffness measurement using a loaded rolling wheel device: results of a Delphi study

2021 ◽  
Vol 22 (1) ◽  
Author(s):  
Maliheh Hadizadeh ◽  
Greg Kawchuk ◽  
Simon French
Keyword(s):  
Delphi Study ◽  
Best Practice ◽  
Standard Protocol ◽  
Research Experience ◽  
Multicenter Studies ◽  
Stiffness Measurement ◽  
Delphi Approach ◽  
Spinal Stiffness ◽  
Human Participants ◽  
Rolling Wheel

Abstract Background Spinal stiffness assessment has the potential to become an important clinical measure. Various spinal stiffness-testing devices are available to help researchers objectively evaluate the spine and patient complaints. One of these is VerteTrack, a device capable of measuring posteroanterior displacement values over an entire spinal region. This study aimed to develop a best-practice protocol for evaluating spinal stiffness in human participants using VerteTrack. Methods Twenty-five individuals with research experience in measuring spinal stiffness, or who were trained in spinal stiffness measurement using the VerteTrack device, were invited to participate in this 3-Round Delphi study. Answers to open-ended questions in Round 1 were thematically analyzed and translated into statements about VerteTrack operation for spinal stiffness measurements. Participants then rated their level of agreement with these statements using a 5-point Likert scale in Rounds 2 and 3. A descriptive statistical analysis was performed. Consensus was achieved when at least 70% of the participants either strongly agreed, agreed, (or strongly disagreed, disagreed) to include a statement in the final protocol. Results Twenty participants completed Round 1 (80%). All these participants completed Rounds 2 and 3. In total, the pre-defined consensus threshold was reached for 67.2% (123/183) of statements after three rounds of surveys. From this, a best-practice protocol was created. Conclusions Using a Delphi approach, a consensus-based protocol for measuring spinal stiffness using the VerteTrack was developed. This standard protocol will help to improve the accuracy, efficiency, and safety of spinal stiffness measurements, facilitate the training of new operators, increase consistency of these measurements in multicenter studies, and provide the synergy and potential for data comparison between spine studies internationally. Although specific to VerteTrack, the resulting standard protocol could be modified for use with other devices designed to collect spinal stiffness measures.

scholarly journals Consensus-based approach to managing opioids, including opioid misuse and opioid use disorder, in patients with serious illness: protocol for a modified Delphi process

BMJ Open ◽  
2021 ◽  
Vol 11 (5) ◽  
pp. e045402
Author(s):  
Caroline King ◽  
Robert Arnold ◽  
Emily Dao ◽  
Jennifer Kapo ◽  
Jane Liebschutz ◽  
...  
Keyword(s):  
Palliative Care ◽  
Opioid Therapy ◽  
Opioid Use Disorder ◽  
Opioid Misuse ◽  
Opioid Use ◽  
Serious Illness ◽  
Modified Delphi ◽  
Delphi Approach ◽  
Institutional Review ◽  
Management Guidance

IntroductionManagement of opioid misuse and opioid use disorder (OUD) among individuals with serious illness is an important yet understudied issue. Palliative care clinicians caring for individuals with serious illness, many of whom may live for months or years, describe a complex tension between weighing the benefits of opioids, which are considered a cornerstone of pain management in serious illness, and serious opioid-related harms like opioid misuse and OUD. And yet, little literature exists to inform the management of opioid misuse and OUDs among individuals with serious illness. Our objective is to provide evidence-based management guidance to clinicians caring for individuals with serious illness who develop opioid misuse or OUD.Methods and analysisWe chose a modified Delphi approach, which is appropriate when empirical evidence is lacking and expert input must be used to shape clinical guidance. We sought to recruit 60 clinicians with expertise in palliative care, addiction or both to participate in this study. We created seven patient cases that capture important management challenges in individuals with serious illness prescribed opioid therapy. We used ExpertLens, an online platform for conducting modified Delphi panels. Participants completed three rounds of data collection. In round 1, they rated and commented on the appropriateness of management choices for cases. In round 2, participants reviewed and discussed their own and other participants’ round 1 numerical responses and comments. In round 3 (currently ongoing), participants again reviewed rounds 1 and 2, and are allowed to change their final numerical responses. We used ExpertLens to automatically identify whether there is consensus, or disagreement, among responses in panels. Only round 3 responses will be used to assess final consensus and disagreement.Ethics and disseminationThis project received ethical approval from the University of Pittsburgh’s Institutional Review Board (study 19110301) and the RAND Institutional Research Board (study 2020-0142). Guidance from this work will be disseminated through national stakeholder networks to gain buy-in and endorsement. This study will also form the basis of an implementation toolkit for clinicians caring for individuals with serious illness who are at risk of opioid misuse or OUD.

scholarly journals Building a neuroinfectious disease consensus curriculum

Neurology ◽  
2019 ◽  
Vol 93 (5) ◽  
pp. 208-216
Author(s):  
Arun Venkatesan ◽  
Felicia C. Chow ◽  
Allen Aksamit ◽  
Russell Bartt ◽  
Thomas P. Bleck ◽  
...  
Keyword(s):  
American Academy ◽  
Delphi Technique ◽  
Fellowship Training ◽  
Consensus Process ◽  
Diverse Range ◽  
Modified Delphi ◽  
Delphi Approach ◽  
Modified Delphi Technique ◽  
Comprehensive Curriculum ◽  
American Academy Of Neurology

ObjectiveTo delineate a comprehensive curriculum for fellowship training in neuroinfectious diseases, we conducted a modified Delphi approach to reach consensus among 11 experts in the field.MethodsThe authors invited a diverse range of experts from the American Academy of Neurology Neuro-Infectious Diseases (AAN Neuro-ID) Section to participate in a consensus process using a modified Delphi technique.ResultsA comprehensive list of topics was generated with 101 initial items. Through 3 rounds of voting and discussion, a curriculum with 83 items reached consensus.ConclusionsThe modified Delphi technique provides an efficient and rigorous means to reach consensus on topics requiring expert opinion. The AAN Neuro-ID section provided the pool of diverse experts, the infrastructure, and the community through which to accomplish the consensus project successfully. This process could be applied to other subspecialties and sections at the AAN.

scholarly journals Entrustable Professional Activities in Hematopathology Pathology Fellowship Training: Consensus Design and Proposal

2021 ◽  
Vol 8 ◽  
pp. 237428952199082
Author(s):  
Kristie White ◽  
Julianne Qualtieri ◽  
Elizabeth L. Courville ◽  
Rose C. Beck ◽  
Bachir Alobeid ◽  
...  
Keyword(s):  
Medical Education ◽  
Graduate Medical Education ◽  
Fellowship Training ◽  
Entrustable Professional Activities ◽  
Guidance Document ◽  
Accreditation Council ◽  
Graduate Medical ◽  
Professional Activities ◽  
Constructive Feedback ◽  
Delphi Approach

Hematopathology fellowship education has grown in complexity as patient-centered treatment plans have come to depend on integration of clinical, morphologic, immunophenotypic, molecular, and cytogenetic variables. This complexity is in competition with the need for timely hematopathology care with stewardship of patient, laboratory, and societal resources. Accreditation Council for Graduate Medical Education Milestones provide a guidance document for hematopathology training, but fellows and their educators are in need of a simple framework that allows assessment and feedback of growth toward independent hematopathology practice. Entrustable professional activities provide one such framework, and herein, we provide proposed Hematopathology Fellowship Entrustable Professional Activities based on review of pertinent guidelines and literature, with multiple rounds of expert and stakeholder input utilizing a modified mini-Delphi approach. Ten core entrustable professional activities deemed essential for graduating hematopathology fellows were developed together with skills and knowledge statements, example scenarios, and corresponding Accreditation Council for Graduate Medical Education Milestones. Application of these entrustable professional activities in program design, fellow evaluation, and decisions regarding level of supervision is discussed with consideration of benefits and barriers to implementation. These entrustable professional activities may be used by hematopathology fellowship directors and faculty to provide fellows with timely constructive feedback, determine entrustment decisions, provide the Clinical Competency Committee with granular data to support Milestone evaluations, and provide insight into areas of potential improvement in fellowship training. Fellows will benefit from a clear roadmap to independent hematopathology practice with concrete and timely feedback.

scholarly journals Selecting contextually appropriate performance indicators in a circumpolar context: a modified Delphi approach

2021 ◽  
Vol 21 (1) ◽  
Author(s):  
Rebecca Rich ◽  
Thomsen D’Hont ◽  
Kellie E. Murphy ◽  
Jeremy Veillard ◽  
Susan Chatwood
Keyword(s):  
Child Health ◽  
Performance Measurement ◽  
Performance Indicators ◽  
Maternity Care ◽  
Cultural Safety ◽  
Determinants Of Health ◽  
Maternal Child Health ◽  
Modified Delphi ◽  
Delphi Approach ◽  
Maternal Child

Abstract Background Meaningful performance measurement requires indicators to be scientifically robust and strategically focused. For many circumpolar states, indicators aligned with national strategies may ignore the priorities of northern, remote, or Indigenous populations. The aim of this project was to identify contextually appropriate performance indicators for maternity care in circumpolar regions. Methods Fourteen maternity care and health systems experts participated in a modified Delphi consensus process. The list of proposed indicators was derived from a previously published scoping review. Fourteen participants rated each proposed indicator according to importance, circumpolar relevance, validity, and reliability and suggested additional indicators for consideration. Results Consensus was achieved after two rounds, as measured by a Cronbach’s alpha of 0.87. Eleven indicators, many of which represented physical health outcomes, were ranked highly on all four criteria. Twenty-nine additional indicators, largely focused on social determinants of health, health care responsiveness, and accessibility, were identified for further research. Travel for care, cultural safety and upstream structural determinants of health were identified as important themes. Conclusions This study identified the important gaps between current performance measurement strategies and the context and values that permeate maternal-child health in circumpolar regions. The indicators identified in this study provide an important foundation for ongoing work. We recommend that future work encompass an appreciation for the intersectoral nature of social, structural, and colonial determinants of maternal-child health in circumpolar regions.

scholarly journals Caregiver-Centered Care Health Workforce Competencies: Developing Consistent Support for Family Caregivers

2020 ◽  
Vol 4 (Supplement_1) ◽  
pp. 15-15
Author(s):  
Jasneet Parmar ◽  
Sharon Anderson ◽  
Lisa Poole ◽  
Wendy Duggleby ◽  
Jayna Holyroyd-Leduc ◽  
...  
Keyword(s):  
Health System ◽  
Family Caregivers ◽  
Health Providers ◽  
Care Providers ◽  
Modified Delphi ◽  
Healthcare Settings ◽  
Delphi Approach ◽  
People With Dementia ◽  
Health And Social Care ◽  
Centered Care

Abstract Family caregivers [FCGs] are the backbone of the health system. They provide over 80% of the care for people with dementia, chronic illnesses and impairments. Despite evidence of their contributions and consequences of caregiving, support for FCGs has not been a health system priority. Education to prepare health providers to effectively identify, engage, assess, and support FCGs throughout the care trajectory is an innovative approach in addressing inconsistent system of supports for FCGs. We report on development and validation of the Caregiver-Centered Care Competency Framework to help with curricular design and subsequent evaluation of effectiveness of care providers working within healthcare settings to engage and support FCGs. We used a three round modified Delphi approach. An expert panel of 42 international, national, and provincial stakeholders agreed to participate. In the first 2 rounds, multi-level, interdisciplinary participants, rated the indicators in terms of importance and relevance. In the 3rd round consensus meeting, participants validated the six competency domains, including indicators in small group sessions. Thirty-four experts (81%) participated in the round 1, 36 (85.7%) in round 2, and 42 people (100%) in round 3. There was stable consensus across all three rounds, 96.07% of participants rated the indicators as essential or important (Round 1, 95.81%; Round 2, 94.15; Round 3, 98.23%). FCG research has been primarily focussed on educating FCGs to provide care. These competencies will shape the design of educational curricula and interdisciplinary training programs aimed at supporting the health and social care workforce to provide caregiver-centered care.

Sign in / Sign up

Export Citation Format

Share Document