scholarly journals Review of study reporting guidelines for clinical studies using artificial intelligence in healthcare

2021 ◽  
Vol 28 (1) ◽  
pp. e100385
Author(s):  
Susan Cheng Shelmerdine ◽  
Owen J Arthurs ◽  
Alastair Denniston ◽  
Neil J Sebire
Keyword(s):  
Artificial Intelligence ◽  
Diagnostic Accuracy ◽  
Prediction Model ◽  
Clinical Investigation ◽  
Health Interventions ◽  
Reporting Guidelines ◽  
High Quality Research ◽  
Specific Element ◽  
Model Studies ◽  
Study Protocols

High-quality research is essential in guiding evidence-based care, and should be reported in a way that is reproducible, transparent and where appropriate, provide sufficient detail for inclusion in future meta-analyses. Reporting guidelines for various study designs have been widely used for clinical (and preclinical) studies, consisting of checklists with a minimum set of points for inclusion. With the recent rise in volume of research using artificial intelligence (AI), additional factors need to be evaluated, which do not neatly conform to traditional reporting guidelines (eg, details relating to technical algorithm development). In this review, reporting guidelines are highlighted to promote awareness of essential content required for studies evaluating AI interventions in healthcare. These include published and in progress extensions to well-known reporting guidelines such as Standard Protocol Items: Recommendations for Interventional Trials-AI (study protocols), Consolidated Standards of Reporting Trials-AI (randomised controlled trials), Standards for Reporting of Diagnostic Accuracy Studies-AI (diagnostic accuracy studies) and Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis-AI (prediction model studies). Additionally there are a number of guidelines that consider AI for health interventions more generally (eg, Checklist for Artificial Intelligence in Medical Imaging (CLAIM), minimum information (MI)-CLAIM, MI for Medical AI Reporting) or address a specific element such as the ‘learning curve’ (Developmental and Exploratory Clinical Investigation of Decision-AI) . Economic evaluation of AI health interventions is not currently addressed, and may benefit from extension to an existing guideline. In the face of a rapid influx of studies of AI health interventions, reporting guidelines help ensure that investigators and those appraising studies consider both the well-recognised elements of good study design and reporting, while also adequately addressing new challenges posed by AI-specific elements.

scholarly journals Reporting guidelines for clinical trials of artificial intelligence interventions: the SPIRIT-AI and CONSORT-AI guidelines

Trials ◽  
2021 ◽  
Vol 22 (1) ◽  
Author(s):  
Hussein Ibrahim ◽  
Xiaoxuan Liu ◽  
Samantha Cruz Rivera ◽  
David Moher ◽  
An-Wen Chan ◽  
...  
Keyword(s):  
Artificial Intelligence ◽  
Clinical Trial ◽  
Clinical Trials ◽  
International Standards ◽  
Health Interventions ◽  
Reporting Guidelines ◽  
Traditional Health ◽  
Trial Protocols ◽  
International Experts

Abstract Background The application of artificial intelligence (AI) in healthcare is an area of immense interest. The high profile of ‘AI in health’ means that there are unusually strong drivers to accelerate the introduction and implementation of innovative AI interventions, which may not be supported by the available evidence, and for which the usual systems of appraisal may not yet be sufficient. Main text We are beginning to see the emergence of randomised clinical trials evaluating AI interventions in real-world settings. It is imperative that these studies are conducted and reported to the highest standards to enable effective evaluation because they will potentially be a key part of the evidence that is used when deciding whether an AI intervention is sufficiently safe and effective to be approved and commissioned. Minimum reporting guidelines for clinical trial protocols and reports have been instrumental in improving the quality of clinical trials and promoting completeness and transparency of reporting for the evaluation of new health interventions. The current guidelines—SPIRIT and CONSORT—are suited to traditional health interventions but research has revealed that they do not adequately address potential sources of bias specific to AI systems. Examples of elements that require specific reporting include algorithm version and the procedure for acquiring input data. In response, the SPIRIT-AI and CONSORT-AI guidelines were developed by a multidisciplinary group of international experts using a consensus building methodological process. The extensions include a number of new items that should be reported in addition to the core items. Each item, where possible, was informed by challenges identified in existing studies of AI systems in health settings. Conclusion The SPIRIT-AI and CONSORT-AI guidelines provide the first international standards for clinical trials of AI systems. The guidelines are designed to ensure complete and transparent reporting of clinical trial protocols and reports involving AI interventions and have the potential to improve the quality of these clinical trials through improvements in their design and delivery. Their use will help to efficiently identify the safest and most effective AI interventions and commission them with confidence for the benefit of patients and the public.

scholarly journals Protocol for development of a reporting guideline (TRIPOD-AI) and risk of bias tool (PROBAST-AI) for diagnostic and prognostic prediction model studies based on artificial intelligence

BMJ Open ◽  
2021 ◽  
Vol 11 (7) ◽  
pp. e048008
Author(s):  
Gary S Collins ◽  
Paula Dhiman ◽  
Constanza L Andaur Navarro ◽  
Ji Ma ◽  
Lotty Hooft ◽  
...  
Keyword(s):  
Artificial Intelligence ◽  
Machine Learning ◽  
Prediction Model ◽  
Assessment Tool ◽  
Risk Of Bias ◽  
Reporting Guideline ◽  
Machine Learning Techniques ◽  
Quality Of Reporting ◽  
Model Studies ◽  
Equator Network

IntroductionThe Transparent Reporting of a multivariable prediction model of Individual Prognosis Or Diagnosis (TRIPOD) statement and the Prediction model Risk Of Bias ASsessment Tool (PROBAST) were both published to improve the reporting and critical appraisal of prediction model studies for diagnosis and prognosis. This paper describes the processes and methods that will be used to develop an extension to the TRIPOD statement (TRIPOD-artificial intelligence, AI) and the PROBAST (PROBAST-AI) tool for prediction model studies that applied machine learning techniques.Methods and analysisTRIPOD-AI and PROBAST-AI will be developed following published guidance from the EQUATOR Network, and will comprise five stages. Stage 1 will comprise two systematic reviews (across all medical fields and specifically in oncology) to examine the quality of reporting in published machine-learning-based prediction model studies. In stage 2, we will consult a diverse group of key stakeholders using a Delphi process to identify items to be considered for inclusion in TRIPOD-AI and PROBAST-AI. Stage 3 will be virtual consensus meetings to consolidate and prioritise key items to be included in TRIPOD-AI and PROBAST-AI. Stage 4 will involve developing the TRIPOD-AI checklist and the PROBAST-AI tool, and writing the accompanying explanation and elaboration papers. In the final stage, stage 5, we will disseminate TRIPOD-AI and PROBAST-AI via journals, conferences, blogs, websites (including TRIPOD, PROBAST and EQUATOR Network) and social media. TRIPOD-AI will provide researchers working on prediction model studies based on machine learning with a reporting guideline that can help them report key details that readers need to evaluate the study quality and interpret its findings, potentially reducing research waste. We anticipate PROBAST-AI will help researchers, clinicians, systematic reviewers and policymakers critically appraise the design, conduct and analysis of machine learning based prediction model studies, with a robust standardised tool for bias evaluation.Ethics and disseminationEthical approval has been granted by the Central University Research Ethics Committee, University of Oxford on 10-December-2020 (R73034/RE001). Findings from this study will be disseminated through peer-review publications.PROSPERO registration numberCRD42019140361 and CRD42019161764.

scholarly journals Uniformity in measuring adherence to reporting guidelines: the example of TRIPOD for assessing completeness of reporting of prediction model studies

BMJ Open ◽  
2019 ◽  
Vol 9 (4) ◽  
pp. e025611 ◽  
Author(s):  
Pauline Heus ◽  
Johanna A A G Damen ◽  
Romin Pajouheshnia ◽  
Rob J P M Scholten ◽  
Johannes B Reitsma ◽  
...  
Keyword(s):  
Prediction Model ◽  
Scoring Rules ◽  
Reporting Guideline ◽  
Reporting Guidelines ◽  
Future Studies ◽  
Model Studies ◽  
Adherence Assessment ◽  
Different Types ◽  
Prognostic Prediction ◽  
Completeness Of Reporting

To promote uniformity in measuring adherence to the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) statement, a reporting guideline for diagnostic and prognostic prediction model studies, and thereby facilitate comparability of future studies assessing its impact, we transformed the original 22 TRIPOD items into an adherence assessment form and defined adherence scoring rules. TRIPOD specific challenges encountered were the existence of different types of prediction model studies and possible combinations of these within publications. More general issues included dealing with multiple reporting elements, reference to information in another publication, and non-applicability of items. We recommend our adherence assessment form to be used by anyone (eg, researchers, reviewers, editors) evaluating adherence to TRIPOD, to make these assessments comparable. In general, when developing a form to assess adherence to a reporting guideline, we recommend formulating specific adherence elements (if needed multiple per reporting guideline item) using unambiguous wording and the consideration of issues of applicability in advance.

scholarly journals Prediction Models for Public Health Containment Measures on COVID-19 Using Artificial Intelligence and Machine Learning: A Systematic Review

2021 ◽  
Vol 18 (9) ◽  
pp. 4499
Author(s):  
Anil Babu Payedimarri ◽  
Diego Concina ◽  
Luigi Portinale ◽  
Massimo Canonico ◽  
Deborah Seys ◽  
...  
Keyword(s):  
Public Health ◽  
Artificial Intelligence ◽  
Machine Learning ◽  
Systematic Review ◽  
Public Transportation ◽  
Prediction Models ◽  
Positive Impact ◽  
Health Interventions ◽  
Public Health Interventions ◽  
Commercial Activities

Artificial Intelligence (AI) and Machine Learning (ML) have expanded their utilization in different fields of medicine. During the SARS-CoV-2 outbreak, AI and ML were also applied for the evaluation and/or implementation of public health interventions aimed to flatten the epidemiological curve. This systematic review aims to evaluate the effectiveness of the use of AI and ML when applied to public health interventions to contain the spread of SARS-CoV-2. Our findings showed that quarantine should be the best strategy for containing COVID-19. Nationwide lockdown also showed positive impact, whereas social distancing should be considered to be effective only in combination with other interventions including the closure of schools and commercial activities and the limitation of public transportation. Our findings also showed that all the interventions should be initiated early in the pandemic and continued for a sustained period. Despite the study limitation, we concluded that AI and ML could be of help for policy makers to define the strategies for containing the COVID-19 pandemic.

Artificial intelligence reporting guidelines: what the pediatric radiologist needs to know

2021 ◽  
Author(s):  
Riwa Meshaka ◽  
Daniel Pinto Dos Santos ◽  
Owen J. Arthurs ◽  
Neil J. Sebire ◽  
Susan C. Shelmerdine
Keyword(s):  
Artificial Intelligence ◽  
Reporting Guidelines ◽  
Pediatric Radiologist

scholarly journals Systematic review protocol to assess artificial intelligence diagnostic accuracy performance in detecting acute ischaemic stroke and large-vessel occlusions on CT and MR medical imaging

BMJ Open ◽  
2021 ◽  
Vol 11 (3) ◽  
pp. e043665
Author(s):  
Srinivasa Rao Kundeti ◽  
Manikanda Krishnan Vaidyanathan ◽  
Bharath Shivashankar ◽  
Sankar Prasad Gorthi
Keyword(s):  
Artificial Intelligence ◽  
Systematic Review ◽  
Diagnostic Accuracy ◽  
Ischaemic Stroke ◽  
Study Protocol ◽  
Systematic Reviews ◽  
Acute Ischaemic Stroke ◽  
Meta Analysis ◽  
Large Vessel ◽  
Sufficient Data

IntroductionThe use of artificial intelligence (AI) to support the diagnosis of acute ischaemic stroke (AIS) could improve patient outcomes and facilitate accurate tissue and vessel assessment. However, the evidence in published AI studies is inadequate and difficult to interpret which reduces the accountability of the diagnostic results in clinical settings. This study protocol describes a rigorous systematic review of the accuracy of AI in the diagnosis of AIS and detection of large-vessel occlusions (LVOs).Methods and analysisWe will perform a systematic review and meta-analysis of the performance of AI models for diagnosing AIS and detecting LVOs. We will adhere to the Preferred Reporting Items for Systematic Reviews and Meta-analyses Protocols guidelines. Literature searches will be conducted in eight databases. For data screening and extraction, two reviewers will use a modified Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies checklist. We will assess the included studies using the Quality Assessment of Diagnostic Accuracy Studies guidelines. We will conduct a meta-analysis if sufficient data are available. We will use hierarchical summary receiver operating characteristic curves to estimate the summary operating points, including the pooled sensitivity and specificity, with 95% CIs, if pooling is appropriate. Furthermore, if sufficient data are available, we will use Grading of Recommendations, Assessment, Development and Evaluations profiler software to summarise the main findings of the systematic review, as a summary of results.Ethics and disseminationThere are no ethical considerations associated with this study protocol, as the systematic review focuses on the examination of secondary data. The systematic review results will be used to report on the accuracy, completeness and standard procedures of the included studies. We will disseminate our findings by publishing our analysis in a peer-reviewed journal and, if required, we will communicate with the stakeholders of the studies and bibliographic databases.PROSPERO registration numberCRD42020179652.

scholarly journals Designing the next generation of implementation research training for learners in low- and middle-income countries

2021 ◽  
Vol 17 (1) ◽  
Author(s):  
Michael J. Penkunas ◽  
Shiau Yun Chong ◽  
Emma L. M. Rhule ◽  
Evangelia Berdou ◽  
Pascale Allotey
Keyword(s):  
Implementation Research ◽  
Research Training ◽  
Training Model ◽  
Learning By Doing ◽  
Health Interventions ◽  
System Level ◽  
High Quality Research ◽  
Middle Income ◽  
Middle Income Countries ◽  
Low And Middle Income

AbstractEfficacious health interventions tested through controlled trials often fail to show desired impacts when implemented at scale. These challenges can be particularly pervasive in low- and middle-income settings where health systems often lack the capacity and mechanisms required for high-quality research and evidence translation. Implementation research is a powerful tool for identifying and addressing the bottlenecks impeding the success of proven health interventions. Implementation research training initiatives, although growing in number, remain out of reach for many investigators in low- and middle-income settings, who possess the knowledge required to contextualize challenges and potential solutions in light of interacting community- and system-level features. We propose a realigned implementation research training model that centers on team-based learning, tailored didactic opportunities, learning-by-doing, and mentorship.

Generation of a Complete Profile for Porosity Log While Drilling Complex Lithology by Employing the Artificial Intelligence

2021 ◽  
Author(s):  
Ahmed Al-Sabaa ◽  
Hany Gamal ◽  
Salaheldin Elkatatny
Keyword(s):  
Artificial Intelligence ◽  
Prediction Model ◽  
Real Time ◽  
Storage Capacity ◽  
Data Set ◽  
Drilling Parameters ◽  
Unseen Data ◽  
Rock Porosity ◽  
Data Points ◽  
Logging Tool

Abstract The formation porosity of drilled rock is an important parameter that determines the formation storage capacity. The common industrial technique for rock porosity acquisition is through the downhole logging tool. Usually logging while drilling, or wireline porosity logging provides a complete porosity log for the section of interest, however, the operational constraints for the logging tool might preclude the logging job, in addition to the job cost. The objective of this study is to provide an intelligent prediction model to predict the porosity from the drilling parameters. Artificial neural network (ANN) is a tool of artificial intelligence (AI) and it was employed in this study to build the porosity prediction model based on the drilling parameters as the weight on bit (WOB), drill string rotating-speed (RS), drilling torque (T), stand-pipe pressure (SPP), mud pumping rate (Q). The novel contribution of this study is to provide a rock porosity model for complex lithology formations using drilling parameters in real-time. The model was built using 2,700 data points from well (A) with 74:26 training to testing ratio. Many sensitivity analyses were performed to optimize the ANN model. The model was validated using unseen data set (1,000 data points) of Well (B), which is located in the same field and drilled across the same complex lithology. The results showed the high performance for the model either for training and testing or validation processes. The overall accuracy for the model was determined in terms of correlation coefficient (R) and average absolute percentage error (AAPE). Overall, R was higher than 0.91 and AAPE was less than 6.1 % for the model building and validation. Predicting the rock porosity while drilling in real-time will save the logging cost, and besides, will provide a guide for the formation storage capacity and interpretation analysis.

Sign in / Sign up

Export Citation Format

Share Document