scholarly journals Guidelines for clinical trial protocols for interventions involving artificial intelligence: the SPIRIT-AI Extension

BMJ ◽  
2020 ◽  
pp. m3210 ◽  
Author(s):  
Samantha Cruz Rivera ◽  
Xiaoxuan Liu ◽  
An-Wen Chan ◽  
Alastair K Denniston ◽  
Melanie J Calvert
Keyword(s):  
Artificial Intelligence ◽  
Clinical Trial ◽  
Expert Consultation ◽  
Consensus Process ◽  
Output Data ◽  
Trial Protocols ◽  
General Readership ◽  
Peer Reviewers ◽  
Multi Stakeholder ◽  
Analysis Of Error

Abstract The SPIRIT 2013 (The Standard Protocol Items: Recommendations for Interventional Trials) statement aims to improve the completeness of clinical trial protocol reporting, by providing evidence-based recommendations for the minimum set of items to be addressed. This guidance has been instrumental in promoting transparent evaluation of new interventions. More recently, there is a growing recognition that interventions involving artificial intelligence need to undergo rigorous, prospective evaluation to demonstrate their impact on health outcomes. The SPIRIT-AI extension is a new reporting guideline for clinical trials protocols evaluating interventions with an AI component. It was developed in parallel with its companion statement for trial reports: CONSORT-AI. Both guidelines were developed using a staged consensus process, involving a literature review and expert consultation to generate 26 candidate items, which were consulted on by an international multi-stakeholder group in a 2-stage Delphi survey (103 stakeholders), agreed on in a consensus meeting (31 stakeholders) and refined through a checklist pilot (34 participants). The SPIRIT-AI extension includes 15 new items, which were considered sufficiently important for clinical trial protocols of AI interventions. These new items should be routinely reported in addition to the core SPIRIT 2013 items. SPIRIT-AI recommends that investigators provide clear descriptions of the AI intervention, including instructions and skills required for use, the setting in which the AI intervention will be integrated, considerations around the handling of input and output data, the human-AI interaction and analysis of error cases. SPIRIT-AI will help promote transparency and completeness for clinical trial protocols for AI interventions. Its use will assist editors and peer-reviewers, as well as the general readership, to understand, interpret and critically appraise the design and risk of bias for a planned clinical trial.

scholarly journals Guidelines for clinical trial protocols for interventions involving artificial intelligence: the SPIRIT-AI extension

2020 ◽  
Vol 26 (9) ◽  
pp. 1351-1363 ◽  
Author(s):  
Samantha Cruz Rivera ◽  
◽  
Xiaoxuan Liu ◽  
An-Wen Chan ◽  
Alastair K. Denniston ◽  
...  
Keyword(s):  
Artificial Intelligence ◽  
Clinical Trial ◽  
Expert Consultation ◽  
Consensus Process ◽  
Output Data ◽  
Trial Protocols ◽  
General Readership ◽  
Peer Reviewers ◽  
Multi Stakeholder ◽  
Analysis Of Error

AbstractThe SPIRIT 2013 statement aims to improve the completeness of clinical trial protocol reporting by providing evidence-based recommendations for the minimum set of items to be addressed. This guidance has been instrumental in promoting transparent evaluation of new interventions. More recently, there has been a growing recognition that interventions involving artificial intelligence (AI) need to undergo rigorous, prospective evaluation to demonstrate their impact on health outcomes. The SPIRIT-AI (Standard Protocol Items: Recommendations for Interventional Trials–Artificial Intelligence) extension is a new reporting guideline for clinical trial protocols evaluating interventions with an AI component. It was developed in parallel with its companion statement for trial reports: CONSORT-AI (Consolidated Standards of Reporting Trials–Artificial Intelligence). Both guidelines were developed through a staged consensus process involving literature review and expert consultation to generate 26 candidate items, which were consulted upon by an international multi-stakeholder group in a two-stage Delphi survey (103 stakeholders), agreed upon in a consensus meeting (31 stakeholders) and refined through a checklist pilot (34 participants). The SPIRIT-AI extension includes 15 new items that were considered sufficiently important for clinical trial protocols of AI interventions. These new items should be routinely reported in addition to the core SPIRIT 2013 items. SPIRIT-AI recommends that investigators provide clear descriptions of the AI intervention, including instructions and skills required for use, the setting in which the AI intervention will be integrated, considerations for the handling of input and output data, the human–AI interaction and analysis of error cases. SPIRIT-AI will help promote transparency and completeness for clinical trial protocols for AI interventions. Its use will assist editors and peer reviewers, as well as the general readership, to understand, interpret and critically appraise the design and risk of bias for a planned clinical trial.

scholarly journals Reporting guidelines for clinical trial reports for interventions involving artificial intelligence: the CONSORT-AI Extension

BMJ ◽  
2020 ◽  
pp. m3164 ◽  
Author(s):  
Xiaoxuan Liu ◽  
Samantha Cruz Rivera ◽  
David Moher ◽  
Melanie J Calvert ◽  
Alastair K Denniston
Keyword(s):  
Artificial Intelligence ◽  
Clinical Trial ◽  
Clinical Trials ◽  
Clinical Trial Design ◽  
Trial Protocols ◽  
General Readership ◽  
Peer Reviewers ◽  
Multi Stakeholder ◽  
Analysis Of Error

Abstract The CONSORT 2010 (Consolidated Standards of Reporting Trials) statement provides minimum guidelines for reporting randomised trials. Its widespread use has been instrumental in ensuring transparency when evaluating new interventions. More recently, there has been a growing recognition that interventions involving artificial intelligence (AI) need to undergo rigorous, prospective evaluation to demonstrate impact on health outcomes. The CONSORT-AI extension is a new reporting guideline for clinical trials evaluating interventions with an AI component. It was developed in parallel with its companion statement for clinical trial protocols: SPIRIT-AI. Both guidelines were developed through a staged consensus process, involving a literature review and expert consultation to generate 29 candidate items, which were assessed by an international multi-stakeholder group in a two-stage Delphi survey (103 stakeholders), agreed on in a two-day consensus meeting (31 stakeholders) and refined through a checklist pilot (34 participants). The CONSORT-AI extension includes 14 new items, which were considered sufficiently important for AI interventions, that they should be routinely reported in addition to the core CONSORT 2010 items. CONSORT-AI recommends that investigators provide clear descriptions of the AI intervention, including instructions and skills required for use, the setting in which the AI intervention is integrated, the handling of inputs and outputs of the AI intervention, the human-AI interaction and providing analysis of error cases. CONSORT-AI will help promote transparency and completeness in reporting clinical trials for AI interventions. It will assist editors and peer-reviewers, as well as the general readership, to understand, interpret and critically appraise the quality of clinical trial design and risk of bias in the reported outcomes.

scholarly journals Reporting guidelines for clinical trial reports for interventions involving artificial intelligence: the CONSORT-AI extension

2020 ◽  
Vol 26 (9) ◽  
pp. 1364-1374 ◽  
Author(s):  
Xiaoxuan Liu ◽  
◽  
Samantha Cruz Rivera ◽  
David Moher ◽  
Melanie J. Calvert ◽  
...  
Keyword(s):  
Artificial Intelligence ◽  
Clinical Trial ◽  
Clinical Trials ◽  
Clinical Trial Design ◽  
Trial Protocols ◽  
General Readership ◽  
Peer Reviewers ◽  
Multi Stakeholder ◽  
Analysis Of Error

AbstractThe CONSORT 2010 statement provides minimum guidelines for reporting randomized trials. Its widespread use has been instrumental in ensuring transparency in the evaluation of new interventions. More recently, there has been a growing recognition that interventions involving artificial intelligence (AI) need to undergo rigorous, prospective evaluation to demonstrate impact on health outcomes. The CONSORT-AI (Consolidated Standards of Reporting Trials–Artificial Intelligence) extension is a new reporting guideline for clinical trials evaluating interventions with an AI component. It was developed in parallel with its companion statement for clinical trial protocols: SPIRIT-AI (Standard Protocol Items: Recommendations for Interventional Trials–Artificial Intelligence). Both guidelines were developed through a staged consensus process involving literature review and expert consultation to generate 29 candidate items, which were assessed by an international multi-stakeholder group in a two-stage Delphi survey (103 stakeholders), agreed upon in a two-day consensus meeting (31 stakeholders) and refined through a checklist pilot (34 participants). The CONSORT-AI extension includes 14 new items that were considered sufficiently important for AI interventions that they should be routinely reported in addition to the core CONSORT 2010 items. CONSORT-AI recommends that investigators provide clear descriptions of the AI intervention, including instructions and skills required for use, the setting in which the AI intervention is integrated, the handling of inputs and outputs of the AI intervention, the human–AI interaction and provision of an analysis of error cases. CONSORT-AI will help promote transparency and completeness in reporting clinical trials for AI interventions. It will assist editors and peer reviewers, as well as the general readership, to understand, interpret and critically appraise the quality of clinical trial design and risk of bias in the reported outcomes.

scholarly journals Reporting guidelines for clinical trials of artificial intelligence interventions: the SPIRIT-AI and CONSORT-AI guidelines

Trials ◽  
2021 ◽  
Vol 22 (1) ◽  
Author(s):  
Hussein Ibrahim ◽  
Xiaoxuan Liu ◽  
Samantha Cruz Rivera ◽  
David Moher ◽  
An-Wen Chan ◽  
...  
Keyword(s):  
Artificial Intelligence ◽  
Clinical Trial ◽  
Clinical Trials ◽  
International Standards ◽  
Health Interventions ◽  
Reporting Guidelines ◽  
Traditional Health ◽  
Trial Protocols ◽  
International Experts

Abstract Background The application of artificial intelligence (AI) in healthcare is an area of immense interest. The high profile of ‘AI in health’ means that there are unusually strong drivers to accelerate the introduction and implementation of innovative AI interventions, which may not be supported by the available evidence, and for which the usual systems of appraisal may not yet be sufficient. Main text We are beginning to see the emergence of randomised clinical trials evaluating AI interventions in real-world settings. It is imperative that these studies are conducted and reported to the highest standards to enable effective evaluation because they will potentially be a key part of the evidence that is used when deciding whether an AI intervention is sufficiently safe and effective to be approved and commissioned. Minimum reporting guidelines for clinical trial protocols and reports have been instrumental in improving the quality of clinical trials and promoting completeness and transparency of reporting for the evaluation of new health interventions. The current guidelines—SPIRIT and CONSORT—are suited to traditional health interventions but research has revealed that they do not adequately address potential sources of bias specific to AI systems. Examples of elements that require specific reporting include algorithm version and the procedure for acquiring input data. In response, the SPIRIT-AI and CONSORT-AI guidelines were developed by a multidisciplinary group of international experts using a consensus building methodological process. The extensions include a number of new items that should be reported in addition to the core items. Each item, where possible, was informed by challenges identified in existing studies of AI systems in health settings. Conclusion The SPIRIT-AI and CONSORT-AI guidelines provide the first international standards for clinical trials of AI systems. The guidelines are designed to ensure complete and transparent reporting of clinical trial protocols and reports involving AI interventions and have the potential to improve the quality of these clinical trials through improvements in their design and delivery. Their use will help to efficiently identify the safest and most effective AI interventions and commission them with confidence for the benefit of patients and the public.

scholarly journals Guidelines for clinical trial protocols for interventions involving artificial intelligence: the SPIRIT-AI extension

2020 ◽  
Vol 2 (10) ◽  
pp. e549-e560 ◽  
Author(s):  
Samantha Cruz Rivera ◽  
Xiaoxuan Liu ◽  
An-Wen Chan ◽  
Alastair K Denniston ◽  
Melanie J Calvert ◽  
...  
Keyword(s):  
Artificial Intelligence ◽  
Clinical Trial ◽  
Trial Protocols

scholarly journals ‘Give Us The Tools!’: development of knowledge transfer tools to support the involvement of patient partners in the development of clinical trial protocols with patient-reported outcomes (PROs), in accordance with SPIRIT-PRO Extension

BMJ Open ◽  
2021 ◽  
Vol 11 (6) ◽  
pp. e046450
Author(s):  
Samantha Cruz Rivera ◽  
Richard Stephens ◽  
Rebecca Mercieca-Bebber ◽  
Ameeta Retzer ◽  
Claudia Rutherford ◽  
...  
Keyword(s):  
Clinical Trial ◽  
Public Involvement ◽  
Patient And Public Involvement ◽  
Patient Reported Outcome ◽  
Flow Diagram ◽  
Additional Patient ◽  
The Public ◽  
Patient Reported ◽  
Trial Protocols ◽  
User Friendly

Objectives(a) To adapt the Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT)-patient-reported outcome (PRO) Extension guidance to a user-friendly format for patient partners and (b) to codesign a web-based tool to support the dissemination and uptake of the SPIRIT-PRO Extension by patient partners.DesignA 1-day patient and public involvement session.ParticipantsSeven patient partners.MethodsA patient partner produced an initial lay summary of the SPIRIT-PRO guideline and a glossary. We held a 1-day PPI session in November 2019 at the University of Birmingham. Five patient partners discussed the draft lay summary, agreed on the final wording, codesigned and agreed the final content for both tools. Two additional patient partners were involved in writing the manuscript. The study compiled with INVOLVE guidelines and was reported according to the Guidance for Reporting Involvement of Patients and the Public 2 checklist.ResultsTwo user-friendly tools were developed to help patients and members of the public be involved in the codesign of clinical trials collecting PROs. The first tool presents a lay version of the SPIRIT-PRO Extension guidance. The second depicts the most relevant points, identified by the patient partners, of the guidance through an interactive flow diagram.ConclusionsThese tools have the potential to support the involvement of patient partners in making informed contributions to the development of PRO aspects of clinical trial protocols, in accordance with the SPIRIT-PRO Extension guidelines. The involvement of patient partners ensured the tools focused on issues most relevant to them.

Artificial intelligence and mass personalization of communication content—An ethical and literacy perspective

2021 ◽  
pp. 146144482110227
Author(s):  
Erik Hermann
Keyword(s):  
Artificial Intelligence ◽  
Conceptual Analysis ◽  
Need Satisfaction ◽  
Ethical Challenges ◽  
Social Good ◽  
Communication Content ◽  
Basic Understanding ◽  
Stakeholder Perspective ◽  
Multi Stakeholder ◽  
Mass Personalization

Artificial intelligence (AI) is (re)shaping communication and contributes to (commercial and informational) need satisfaction by means of mass personalization. However, the substantial personalization and targeting opportunities do not come without ethical challenges. Following an AI-for-social-good perspective, the authors systematically scrutinize the ethical challenges of deploying AI for mass personalization of communication content from a multi-stakeholder perspective. The conceptual analysis reveals interdependencies and tensions between ethical principles, which advocate the need of a basic understanding of AI inputs, functioning, agency, and outcomes. By this form of AI literacy, individuals could be empowered to interact with and treat mass-personalized content in a way that promotes individual and social good while preventing harm.

scholarly journals Adaptive Semantic Tag Mining from Heterogeneous Clinical Research Texts

2015 ◽  
Vol 54 (02) ◽  
pp. 164-170 ◽  
Author(s):  
T. Hao ◽  
C. Weng
Keyword(s):  
Clinical Trial ◽  
Clinical Research ◽  
Architectural Design ◽  
Semantic Equivalence ◽  
Clinical Text ◽  
Mining Algorithm ◽  
Prevalence Trends ◽  
Trial Protocols ◽  
Mining Methods ◽  
Frequency Threshold

SummaryObjectives: To develop an adaptive approach to mine frequent semantic tags (FSTs) from heterogeneous clinical research texts.Methods: We develop a “plug-n-play” framework that integrates replaceable un-supervised kernel algorithms with formatting, functional, and utility wrappers for FST mining. Temporal information identification and semantic equivalence detection were two example functional wrappers. We first compared this approach’s recall and efficiency for mining FSTs from ClinicalTrials.gov to that of a recently published tag-mining algorithm. Then we assessed this approach’s adaptability to two other types of clinical research texts: clinical data requests and clinical trial protocols, by comparing the prevalence trends of FSTs across three texts.Results: Our approach increased the average recall and speed by 12.8% and 47.02% respectively upon the baseline when mining FSTs from ClinicalTrials.gov, and maintained an overlap in relevant FSTs with the baseline ranging between 76.9% and 100% for varying FST frequency thresholds. The FSTs saturated when the data size reached 200 documents. Consistent trends in the prevalence of FST were observed across the three texts as the data size or frequency threshold changed.Conclusions: This paper contributes an adaptive tag-mining framework that is scalable and adaptable without sacrificing its recall. This component-based architectural design can be potentially generalizable to improve the adaptability of other clinical text mining methods.

Sign in / Sign up

Export Citation Format

Share Document