scholarly journals Major Mistakes and Errors in the Use of Trial Sequential Analysis in Systematic Reviews or Meta-analyses – Protocol for a Systematic Review

Author(s):  
Christian Gunge Riberholt ◽  
Markus Harboe Olsen ◽  
Joachim Birch Milan ◽  
Christian Gluud

Abstract Background: Adequately conducted systematic reviews with meta-analyses are considered the highest level of evidence and thus directly defines many clinical guidelines. However, the risk of type I and II errors in meta-analyses are substantial. Trial Sequential Analysis is a method for controlling these risks. Erroneous use of the method might lead to research waste or misleading conclusions. Methods: The current protocol describes a systematic review aimed to identify common and major mistakes and errors in the use of Trial Sequential Analysis by evaluating published systematic reviews and meta-analyses that include this method. We plan to include all studies using Trial Sequential Analysis published from 2018 to 2021, an estimated 400 to 600 publications. We will search Medical Literature Analysis and Retrieval System Online (MEDLINE) and the Cochrane Database of Systematic Reviews (CDSR), including studies with all types of participants, interventions, and outcomes. The search will begin in July 2021. Two independent reviewers will screen titles and abstracts, include relevant full text articles, extract data from the studies into a predefined checklist, and evaluate the methodological quality of the study using the AMSTAR 2 (Assessing the methodological quality of systematic reviews). Discussion: This protocol follows the Preferred Reporting Items for Systematic Reviews and Meta-Analysis Protocols (PRISMA-P). The identified mistakes and errors will form the basis of a reviewed guideline for the use of Trial Sequential Analysis. Appropriately controlling for type I and II errors might reduce research waste and improve quality and precision of the evidence that clinical guidelines are based upon.

2021 ◽  
Vol 10 (1) ◽  
Author(s):  
Faiza Siddiqui ◽  
Marija Barbateskovic ◽  
Sophie Juul ◽  
Kiran Kumar Katakam ◽  
Klaus Munkholm ◽  
...  

Abstract Background Major depression significantly impairs quality of life, increases the risk of suicide, and poses tremendous economic burden on individuals and societies. Duloxetine, a serotonin norepinephrine reuptake inhibitor, is a widely prescribed antidepressant. The effects of duloxetine have, however, not been sufficiently assessed in earlier systematic reviews and meta-analyses. Methods/design A systematic review will be performed including randomised clinical trials comparing duloxetine with ‘active’ placebo, placebo or no intervention for adults with major depressive disorder. Bias domains will be assessed, an eight-step procedure will be used to assess if the thresholds for clinical significance are crossed. We will conduct meta-analyses. Trial sequential analysis will be conducted to control random errors, and the certainty of the evidence will be assessed using GRADE. To identify relevant trials, we will search Cochrane Central Register of Controlled Trials, Medical Literature Analysis and Retrieval System Online, Excerpta Medica database, PsycINFO, Science Citation Index Expanded, Social Sciences Citation Index, Conference Proceedings Citation Index—Science and Conference Proceedings Citation Index—Social Science & Humanities. We will also search Chinese databases and Google Scholar. We will search all databases from their inception to the present. Two review authors will independently extract data and perform risk of bias assessment. Primary outcomes will be the difference in mean depression scores on Hamilton Depression Rating Scale between the intervention and control groups and serious adverse events. Secondary outcomes will be suicide, suicide-attempts, suicidal ideation, quality of life and non-serious adverse events. Discussion No former systematic review has systematically assessed the beneficial and harmful effects of duloxetine taking into account both the risks of random errors and the risks of systematic errors. Our review will help clinicians weigh the benefits of prescribing duloxetine against its adverse effects and make informed decisions. Systematic review registration PROSPERO 2016 CRD42016053931


2020 ◽  
pp. 219256822090681 ◽  
Author(s):  
Muthu Sathish ◽  
Ramakrishnan Eswar

Study Design: Systematic review. Objectives: To assess the methodological quality of systematic reviews and meta-analyses in spine surgery over the past 2 decades. Materials and Methods: We conducted independent and in duplicate systematic review of the published systematic reviews and meta-analyses between 2000 and 2019 from PubMed Central and Cochrane Database pertaining to spine surgery involving surgical intervention. We searched bibliographies to identify additional relevant studies. Methodological quality was evaluated with AMSTAR score and graded with AMSTAR 2 criteria. Results: A total of 96 reviews met the eligibility criteria, with mean AMSTAR score of 7.51 (SD = 1.98). Based on AMSTAR 2 criteria, 13.5% (n = 13) and 18.7% (n = 18) of the studies had high and moderate level of confidence of results, respectively, without any critical flaws. A total of 29.1% (n = 28) of the studies had at least 1 critical flaw and 38.5% (n = 37) of the studies had more than 1 critical flaw, so that their results have low and critically low confidence, respectively. Failure to analyze the conflict of interest of authors of primary studies included in review and lack of list of excluded studies with justification were the most common critical flaw. Regression analysis demonstrated that studies with funding and studies published in recent years were significantly associated with higher methodological quality. Conclusion: Despite improvement in methodological quality of systematic reviews and meta-analyses in spine surgery in current decade, a substantial proportion continue to show critical flaws. With increasing number of review articles in spine surgery, stringent measures must be taken to adhere to methodological quality by following PRISMA and AMSTAR guidelines to attain higher standards of evidence in published literature.


2021 ◽  
Vol 23 (Supplement_2) ◽  
pp. ii51-ii52
Author(s):  
A M George ◽  
S Gupta ◽  
S M Keshwara ◽  
M A Mustafa ◽  
C S Gillespie ◽  
...  

Abstract BACKGROUND Systematic reviews and meta-analyses constitute the highest level of research evidence and for a disease with limited clinical trial activity, are often relied upon to help inform clinical practice. This review of reviews evaluates both the reporting & methodological quality of meningioma evidence syntheses. MATERIAL AND METHODS Potentially eligible meningioma reviews published between 1st January 1990 and 31st December 2020 were identified from eight electronic databases. Inclusion required the study to meet the Cochrane guideline definition of a systematic review or meta-analysis. Reviews concerning neurofibromatosis type 2, spinal and pediatric meningiomas were excluded. The reporting and methodological quality of articles were assessed against the following modified guidelines: Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA), A MeaSurement Tool to Assess Systematic Reviews (AMSTAR2) and the Risk of Bias in Systematic Reviews (ROBIS) guidelines. RESULTS 117 systematic reviews were identified, 57 of which included meta-analysis (48.7%). The number of meningioma systematic reviews published each year has increased with 63 studies (53.9%) published between 01/2018 and 12/2020. A median of 17 studies (IQR 9–29) were included per review. Impact factor of journals publishing a systematic review with or without a meta-analysis was similar (median 2.3 vs 1.8, P=0.397). The mean PRISMA scores for systematic reviews with a meta-analysis was 21.11 (SD 4.1, 78% adherence) and without was 13.89 (SD 3.4, 63% adherence). Twenty-nine systematic reviews with meta-analysis (51%) and 11 without meta-analysis (18%) achieved greater than 80% adherence to PRISMA recommendations. Methodological quality assessment using AMSTAR2 revealed one study (0.9%) as high quality whilst 111 (94.8%) studies were graded as critically low. One hundred and two articles (87.2%) did not utilize a comprehensive search strategy as defined by the AMSTAR2 tool. Ninety-nine studies (84.6%) obtained a high level of concern for potential bias as per the ROBIS assessment. One hundred and eight articles (92.3%) failed to present information that a protocol had been established prior to study commencement and 76 articles (65.0%) did not conduct a risk of bias assessment. Across the three tools, domains relating to the establishment of a protocol prior to review commencement and conducting appropriate risk of bias assessments were frequently low scoring. CONCLUSION Overall reporting and methodological quality of meningioma systematic reviews was sub-optimal. Established critical appraisal tools and reporting guidelines should be utilized a priori to assist in producing high-quality systematic reviews.


Author(s):  
Morteza Arab-Zozani ◽  
Zahra Heidarifard ◽  
Efat Jabarpour

Context: The number of studies on health is increasing rapidly worldwide and in Iran. Systematic review studies, meta-analyses, and economic evaluation are of great importance in evidence-based decision making because of their standing in the evidence-based pyramid. The purpose of this study was to evaluate the reporting and methodological quality of Iranian systematic reviews, meta-analysis studies and economic evaluations on healthcare. Evidence Acquisition: PubMed and Scopus databases were searched to find considered studies, including systematic reviews, meta analyses and economic evaluations published from 2005 to 2015. Because of the high volume of review studies, 10% of all systematic reviews and meta-analyses were selected as a random sample. Also, all economic evaluations were included. Articles were evaluated using checklists, including PRISMA, AMSTAR and QHES with a maximum score of 27, 11 and 100, respectively. The quality score for each criterion as well as the epidemiological and descriptive characteristics of all articles was determined. Data were analyzed using SPSS V. 16 software. Results: After searching the databases, 1084 systematic reviews and meta-analyses were obtained, 10% of which were included in the study. A total of 41 economic evaluations were also included. The mean scores of systematic reviews and meta-analyses based on PRISMA and AMSTAR checklists were 17.04 (5.35) and 5.42 (1.97), respectively, and 68.21 (12.44) for economic evaluations based on QHES. Only three systematic reviews and meta-analysis articles had recorded protocols and 85% of the studies included the terms “systematic review” and “meta-analysis” in their titles. Only one study had been updated. In addition, 81% of the systematic reviews and meta-analyses were published in specialized journals and 47% in Iranian journals. Financial resources and conflict of interests had been mentioned in 33% and 66% of the studies, respectively. Of the selected studies, 60% had evaluated the quality of the articles and 35% of the studies had assessed publication bias. In economic evaluations, 56% had used CEA analysis, 22% CUA analysis, 12% CBA analysis, and one study had used CMA analysis. Of these studies, 54% were model-based health economic studies and 12% were trial-based. The economic perspective was the health care system in most studies. Forty-four percent of the studies had a short time horizon of one year or less, whereas 33% had a lifetime horizon. Moreover, 68% of the studies showed sensitivity analysis and only 5 included the magnitude and direction of the bias. Conclusions: Overall, the reporting and methodological quality of the selected studies were estimated at a moderate level. Based on these results, it is recommended to adopt strategies to reduce preventable errors in studies. Having a primary plan and protocol and registering it as a systematic review can be an important factor in improving the quality of studies. Economic evaluations should also focus on issues, such as economic perspective, time horizon, available bias, and sensitivity analysis.


BMJ Open ◽  
2020 ◽  
Vol 10 (4) ◽  
pp. e030503 ◽  
Author(s):  
Martijn S. Marsman ◽  
Jorn Wetterslev ◽  
Patrick W.H.E. Vriens ◽  
Ronald L.A.W. Bleys ◽  
Abdelkarime Kh. Jahrome ◽  
...  

IntroductionTraditional carotid endarterectomy is considered to be the standard technique for prevention of a new stroke in patients with a symptomatic carotid stenosis. Use of patch angioplasty to restore the arterial wall after longitudinal endarterectomy is, to date, not unequivocally proven to be superior to eversion technique. A systematic review is needed for evaluation of benefits and harms of the eversion technique versus the traditional endarterectomy with patch angioplasty in patients with symptomatic carotid stenosis.Methods and outcomesThe review will be conducted according to this protocol following the recommendations of the ‘Cochrane Handbook for Systematic Reviews’ and reported according to Preferred Reporting Items for Systematic Reviews and Meta-Analyses. Randomised clinical trials comparing eversion technique versus endarterectomy with patch angioplasty in patients with a symptomatic stenosis of the internal carotid artery will be included. Primary outcomes are all-cause mortality rate, health-related quality of life and serious adverse events. Secondary outcomes are 30-day stroke and mortality rate, symptomatic arterial restenosis or occlusion and non-serious adverse events. The databases Cochrane Central Register of Controlled Trials, PubMed/MEDLINE and EMBASE will be searched (November 2019). We will primarily base our conclusions on meta-analyses of trials with overall low-risk of bias. We will use trial sequential analysis to assist the evaluation of imprecision in Grading of Recommendations, Assessment, Development and Evaluation. However, if pooled point estimates of all trials are similar to pooled point estimates of trials with overall low risk of bias and there is lack of a statistical significant interaction between estimates from trials with overall high risk of bias and trials with overall low risk of bias we will consider the trial sequential analysis adjusted precision of the estimate achieved in all trials as the result of our meta-analyses.Ethics and disseminationThe proposed systematic review will collect and analyse data from published studies, therefore, ethical approval is not required. The results of the review will be disseminated by publication in a peer-review journal and submitted for presentation at conferences.PROSPERO registration numberCRD42019119361.


Author(s):  
Christine Baldwin ◽  
Rosemary Smith ◽  
Michelle Gibbs ◽  
C Elizabeth Weekes ◽  
Peter W Emery

ABSTRACT There is considerable heterogeneity across the findings of systematic reviews of oral nutritional supplement (ONS) interventions, presenting difficulties for healthcare decision-makers and patients alike. It is not known whether heterogeneity arises from differences in patient populations or relates to methodological rigor. This overview aimed to collate and compare findings from systematic reviews of ONSs compared with routine care in adult patients who were malnourished or at risk of malnutrition with any clinical condition and to examine their methodological quality. Three electronic databases were searched to July 2019, supplemented with hand-searching. Data on all outcomes were extracted and review methodological quality assessed using A MeaSurement Tool for Assessment of systematic Reviews (AMSTAR). Twenty-two reviews were included, 11 in groups from mixed clinical backgrounds and 11 in specific clinical conditions. Ninety-one meta-analyses were identified for 12 different outcomes but there was discordance between results. Significant benefits of ONSs were reported in 4 of 4 analyses of energy intake, 7 of 11 analyses of body weight, 7 of 22 analyses of mortality, 10 of 17 analyses of complications (total and infectious), 1 of 3 analyses of muscle strength, 4 of 9 analyses of body composition/nutritional status, 2 of 14 analyses of length of stay, and 2 of 5 analyses of hospital readmissions. Ten reviews were high quality (AMSTAR scores 8–11), 9 moderate (AMSTAR scores 3–8), and 3 poor (AMSTAR scores 0–3). Methodological deficiencies were limitations to searches, poor reporting of heterogeneity, and failure to incorporate quality of evidence into any recommendations. Discordance between reviews was not markedly reduced when only high-quality reviews were considered. Evidence for the effects of ONS in malnourished patients or those who are at risk of malnutrition is uncertain, and discordance in results can arise from differences in clinical background of patients or the etiological basis of malnutrition.


2020 ◽  
Vol 2020 ◽  
pp. 1-7
Author(s):  
Jinke Huang ◽  
Manli Wu ◽  
Simin Liang ◽  
Xiaohui Qin ◽  
Min Shen ◽  
...  

Objectives. Acupuncture has increasingly been used for insomnia relief after stroke. We aimed to evaluate the methodological quality and summarize the evidence regarding the effectiveness of acupuncture for poststroke insomnia (PSI) from systematic reviews/meta-analyses (SRs/MAs). Methods. Eight databases were searched from inception through August 23, 2020. SRs/MAs on acupuncture treatment for PSI were included. Methodological quality assessment was performed using Assessing the Methodological Quality of Systematic Reviews 2 (AMSTAR-2), and evidence quality assessment was performed using the Grading of Recommendations, Assessment, Development, and Evaluation (GRADE). Results. Six SRs/MAs on acupuncture treatment for PSI were included. The AMSTAR-2 showed that the methodological quality of all included SRs/MAs was rated as critically low. According to the evaluation results of GRADE, 38.9% (7/18) of outcomes were rated as very low-quality evidence, 22.2% (4/18) were low-quality evidence, and 8.9% (7/18) were moderate-quality evidence. Descriptive analysis results showed that acupuncture was an effective treatment modality for PSI. Conclusions. All included reviews indicated that acupuncture was more effective than the control group for the treatment of PSI, but the credibility of the results is limited owing to the generally low methodological and evidence quality of the included SRs/MAs. More high-quality evidence is needed to determine whether acupuncture is more effective than other treatments.


2015 ◽  
Vol 18 (3) ◽  
pp. A31
Author(s):  
J. Chambers ◽  
H. Naci ◽  
O. Wouters ◽  
J. Pyo ◽  
S. Gunjal ◽  
...  

Sign in / Sign up

Export Citation Format

Share Document