A Simple, Principled Approach to Combining Evidence From Meta-Analysis and High-Quality Replications

2018 ◽  
Vol 1 (2) ◽  
pp. 174-185 ◽  
Author(s):  
Evan C. Carter ◽  
Michael E. McCullough
Keyword(s):  
Meta Analysis ◽  
Ego Depletion ◽  
Practical Significance ◽  
High Quality ◽  
Questionable Research Practices ◽  
Combining Evidence ◽  
Meta Analyses ◽  
Work Done ◽  
Combining Information

Recent discussions of the influence of publication bias and questionable research practices on psychological science have increased researchers’ interest in both bias-correcting meta-analytic techniques and preregistered replication. Both approaches have their strengths: For example, meta-analyses can quantitatively characterize the full body of work done in the field of interest, and preregistered replications can be immune to bias. Both approaches also have clear weaknesses: Decisions about which meta-analytic estimates to interpret tend to be controversial, and replications can be discounted for failing to address important methodological heterogeneity. Using the experimental literature on ego depletion as a case study, we illustrate a principled approach to combining information from meta-analysis with information from subsequently conducted high-quality replications. This approach (a) compels researchers to explicate their beliefs in meta-analytic conclusions (and also, when controversy arises, to defend the basis for those beliefs), (b) encourages consideration of practical significance, and (c) facilitates the process of planning replications by specifying the sample sizes necessary to have a reasonable chance of changing the minds of other researchers.

Evidence-based approaches to psychiatry

2017 ◽  
pp. 119-134
Author(s):  
Paul Harrison ◽  
Philip Cowen ◽  
Tom Burns ◽  
Mina Fazel
Keyword(s):  
Patient Experience ◽  
Meta Analysis ◽  
Psychiatric Research ◽  
Evidence Based ◽  
High Quality ◽  
Qualitative Methodologies ◽  
Meta Analyses ◽  
Based Medicine ◽  
Quality Evidence

‘Evidence-based approaches to psychiatry’ describes the application of evidence-based medicine (EBM) to psychiatric practice. The chapter covers the key processes in EBM, including the formulation of a clinically relevant question, the systematic search for high-quality evidence and the meta-analytic synthesis of data. It demonstrates how evidence-based approaches to psychiatry have led to important developments showing quantitative effects of different treatments through advanced meta-analysis of data from randomized trials. This has underpinned the development of clinical guidelines that have the aim of improving the reliability and quality of treatments that patients receive. The chapter also describes how meta-analyses should be critically reviewed, as well as their problems and limitations. Not all relevant questions in psychiatric research are susceptible to the quantitative approach offered by EBM, and the chapter also outlines how qualitative methodologies can play a key role in answering important questions related, for example, to the patient experience.

Medical visit time and predictors in health facilities: a mega systematic review and meta-analysis

2019 ◽  
Vol 12 (5) ◽  
pp. 373-402 ◽  
Author(s):  
Sakineh Hajebrahimi ◽  
Ali Janati ◽  
Morteza Arab-Zozani ◽  
Mobin Sokhanvar ◽  
Elaheh Haghgoshayie ◽  
...  
Keyword(s):  
Systematic Review ◽  
Meta Analysis ◽  
Visit Time ◽  
Medical Visit ◽  
High Quality ◽  
Content Type ◽  
Face To Face ◽  
Significant Difference ◽  
Meta Analyses

Purpose Visit time is a crucial aspect of patient–physician interaction; its inadequacy can negatively impact the efficiency of treatment and diagnosis. In addition, visit time is a fundamental demand of patients, and it is one of the rights of every patient. The purpose of this paper is to determine factors influencing the consultation length of physicians and to compare consultation length in different countries. Design/methodology/approach MEDLINE (PubMed), Web of Science, Cochrane, ProQuest, Scopus, and Google Scholar were searched. In addition, references of references were checked, and publication lists of individual scholars in the field were examined. We used data sources up to June 2018, without language restriction. We used a random-effects model for the meta-analyses. Meta-analyses were conducted using Comprehensive Meta-Analysis Version (CMA) 3.0. Findings Of 16,911 identified studies, 189 studies were assessed of which 125 cases (67 percent) have been conducted in the USA. A total of 189 studies, 164 (86.77 percent) involved face-to face-consultations. The effects of three variables, physician gender, patient gender, and type of consultation were analyzed. According to moderate and strong evidence studies, no significant difference was found in the consultation lengths of female and male doctors (Q=42.72, df=8, I2=81.27, p=0.891) and patients’ gender (Q=55.98, df=11, I2=80.35, p=0.314). In addition, no significant difference was found in the telemedicine or face-to-face visits (Q=41.25, df=5, I2=87.88, p=0.170). Originality/value In this systematic review and meta-analysis, all of physicians’ visits in 34 countries were surveyed. The evidence suggests that specified variables do not influence the length of consultations. Good relationship is essential to a safe and high-quality consultation and referral process. A high-quality consultation can improve decisions and quality of visits, treatment effectiveness, efficiency of service, quality of care, patient safety and physician and patient satisfaction.

scholarly journals Predicting suicidal behaviours using clinical instruments: Systematic review and meta-analysis of positive predictive values for risk scales

2017 ◽  
Vol 210 (6) ◽  
pp. 387-395 ◽  
Author(s):  
Gregory Carter ◽  
Allison Milner ◽  
Katie McGill ◽  
Jane Pirkis ◽  
Nav Kapur ◽  
...  
Keyword(s):  
Systematic Review ◽  
High Risk ◽  
Meta Analysis ◽  
Modifiable Risk Factors ◽  
High Quality ◽  
Policy Makers ◽  
Predictive Values ◽  
Effective Interventions ◽  
Self Harm ◽  
Meta Analyses

BackgroundPrediction of suicidal behaviour is an aspirational goal for clinicians and policy makers; with patients classified as ‘high risk’ to be preferentially allocated treatment. Clinical usefulness requires an adequate positive predictive value (PPV).AimsTo identify studies of predictive instruments and to calculate PPV estimates for suicidal behaviours.MethodA systematic review identified studies of predictive instruments. A series of meta-analyses produced pooled estimates of PPV for suicidal behaviours.ResultsFor all scales combined, the pooled PPVs were: suicide 5.5% (95% CI 3.9–7.9%), self-harm 26.3% (95% CI 21.8–31.3%) and self-harm plus suicide 35.9% (95% CI 25.8–47.4%). Subanalyses on self-harm found pooled PPVs of 16.1% (95% CI 11.3–22.3%) for high-quality studies, 32.5% (95% CI 26.1–39.6%) for hospital-treated self-harm and 26.8% (95% CI 19.5–35.6%) for psychiatric in-patients.ConclusionsNo ‘high-risk’ classification was clinically useful. Prevalence imposes a ceiling on PPV. Treatment should reduce exposure to modifiable risk factors and offer effective interventions for selected subpopulations and unselected clinical populations.

scholarly journals Deep learning for robust meta-analytic estimation

2019 ◽  
Author(s):  
Evan C Carter
Keyword(s):  
Deep Learning ◽  
High Performance ◽  
Meta Analysis ◽  
Statistical Simulation ◽  
Questionable Research Practices ◽  
Scientific Debate ◽  
Hierarchy Of Evidence ◽  
Novel Approach ◽  
Meta Analyses ◽  
Analytic Estimation

Meta-analysis represents the promise of cumulative science--that each successive study brings us greater understanding of a given phenomenon. As such, meta-analyses are highly influential and gaining in popularity. However, there are well-known threats to the validity of meta-analytic results, such as processes like publication bias and questionable research practices which can cause researchers to massively overestimate the evidence in support of a claim. There are many statistical methods to correct for such bias, but no single method has been found to be robust in all realistic conditions. Here, I describe a method that merges statistical simulation and deep learning to achieve an unprecedented level of robust meta-analytic estimation in the face of numerous forms of bias and other historically problematic conditions. Furthermore, the resulting estimator, called DeepMA, has the unique property that it can easily evolve: As new conditions for which robustness is needed are identified, DeepMA can be re-trained to maintain high performance. Given the weaknesses that have been identified for meta-analysis, the current consensus is that it should serve as simply another data point, rather than residing at the top of the hierarchy of evidence. The novel approach I describe, however, holds the potential to eliminate these weaknesses, possibly solidifying meta-analysis as the platinum standard in scientific debate.

scholarly journals 919Step-by-step guidance on prospective meta-analyses

2021 ◽  
Vol 50 (Supplement_1) ◽  
Author(s):  
Anna Lene Seidler ◽  
Kylie Hunter ◽  
Saskia Cheyne ◽  
Jesse Berlin ◽  
Davina Ghersi ◽  
...  
Keyword(s):  
Systematic Reviews ◽  
Meta Analysis ◽  
Analysis Committee ◽  
Medical Databases ◽  
Expert Opinions ◽  
Use Of Data ◽  
Recent Case Study ◽  
Technical Advances ◽  
Meta Analyses

Abstract Focus of Presentation In a prospective meta-analysis (PMA), studies are included before their results are known. This can reduce risk of publication bias and selective outcome reporting, and enables researchers to harmonise their research efforts. Despite rising numbers, there is little guidance on how to conduct PMA. We, the Cochrane PMA Methods Group, developed step-by-step guidance based on a scoping review, and expert opinions and experiences. Each step is illustrated with a recent case study. Findings We describe seven steps for conducting PMA. After developing a protocol (Steps 1), a systematic search for eligible planned/ongoing studies should be conducted, including a search of registries, medical databases and contacting stakeholders (Step 2-3). These studies are then invited to form a collaboration (Step 4), ideally including a steering and data analysis committee. Next, important study features such as common core outcomes and confounders are agreed upon (Step 5). This reduces heterogeneity and increases the number of available outcomes for meta-analysis. Certainty of evidence is assessed by adapting tools such as GRADE (Step 6). Results should be reported using adapted versions of reporting tools such as PRISMA (Step 7). Conclusions/Implications PMA reduce many problems of traditional retrospective systematic reviews and meta-analysis. Updated guidance and recent technical advances will help increase their numbers further. Key messages PMA are ‘next generation systematic reviews’ that allow for greatly improved use of data, whilst reducing bias and research waste. This step-by-step guidance will enable more researchers to conduct successful PMA.

Statistical Models for the Meta-analysis of Nonindependent Data

2013 ◽  
Author(s):  
Kerrie Mengersen ◽  
Michael D. Jennions ◽  
Christopher H. Schmid
Keyword(s):  
Statistical Models ◽  
Single Case ◽  
Meta Analysis ◽  
Dry Weight ◽  
Multiple Treatments ◽  
The Individual ◽  
Meta Analyses ◽  
Size Estimates ◽  
Over Time

In many meta-analyses, independence is questionable because there are several effect estimates per study and/or some of the individual studies included in the meta-analysis might not provide independent estimates of the effect. Within-study nonindependence can arise due to multiple measures of the same effect on the same experimental units being made over time, multiple treatments being compared to the same set of control individuals, or different measures being taken (e.g., plant height, dry weight, and photosynthesis rate) from the same experimental units to generate several different effect size estimates. This chapter discusses nonindependence among effect sizes both within and among studies. It focuses on four commonplace situations where nonindependence can occur in ecology and evolution meta-analyses. Each of these four situations is illustrated with a single case study.

P14.71 An assessment of the reporting and methodological quality of meningioma systematic reviews and meta-analyses

2021 ◽  
Vol 23 (Supplement_2) ◽  
pp. ii51-ii52
Author(s):  
A M George ◽  
S Gupta ◽  
S M Keshwara ◽  
M A Mustafa ◽  
C S Gillespie ◽  
...  
Keyword(s):  
Systematic Review ◽  
Systematic Reviews ◽  
Methodological Quality ◽  
Meta Analysis ◽  
Neurofibromatosis Type ◽  
Risk Of Bias ◽  
Measurement Tool ◽  
High Quality ◽  
Meta Analyses

Abstract BACKGROUND Systematic reviews and meta-analyses constitute the highest level of research evidence and for a disease with limited clinical trial activity, are often relied upon to help inform clinical practice. This review of reviews evaluates both the reporting & methodological quality of meningioma evidence syntheses. MATERIAL AND METHODS Potentially eligible meningioma reviews published between 1st January 1990 and 31st December 2020 were identified from eight electronic databases. Inclusion required the study to meet the Cochrane guideline definition of a systematic review or meta-analysis. Reviews concerning neurofibromatosis type 2, spinal and pediatric meningiomas were excluded. The reporting and methodological quality of articles were assessed against the following modified guidelines: Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA), A MeaSurement Tool to Assess Systematic Reviews (AMSTAR2) and the Risk of Bias in Systematic Reviews (ROBIS) guidelines. RESULTS 117 systematic reviews were identified, 57 of which included meta-analysis (48.7%). The number of meningioma systematic reviews published each year has increased with 63 studies (53.9%) published between 01/2018 and 12/2020. A median of 17 studies (IQR 9–29) were included per review. Impact factor of journals publishing a systematic review with or without a meta-analysis was similar (median 2.3 vs 1.8, P=0.397). The mean PRISMA scores for systematic reviews with a meta-analysis was 21.11 (SD 4.1, 78% adherence) and without was 13.89 (SD 3.4, 63% adherence). Twenty-nine systematic reviews with meta-analysis (51%) and 11 without meta-analysis (18%) achieved greater than 80% adherence to PRISMA recommendations. Methodological quality assessment using AMSTAR2 revealed one study (0.9%) as high quality whilst 111 (94.8%) studies were graded as critically low. One hundred and two articles (87.2%) did not utilize a comprehensive search strategy as defined by the AMSTAR2 tool. Ninety-nine studies (84.6%) obtained a high level of concern for potential bias as per the ROBIS assessment. One hundred and eight articles (92.3%) failed to present information that a protocol had been established prior to study commencement and 76 articles (65.0%) did not conduct a risk of bias assessment. Across the three tools, domains relating to the establishment of a protocol prior to review commencement and conducting appropriate risk of bias assessments were frequently low scoring. CONCLUSION Overall reporting and methodological quality of meningioma systematic reviews was sub-optimal. Established critical appraisal tools and reporting guidelines should be utilized a priori to assist in producing high-quality systematic reviews.

scholarly journals Effectiveness of Olympic Combat Sports on Balance, Fall Risk or Falls in Older Adults: A Systematic Review

Biology ◽  
2022 ◽  
Vol 11 (1) ◽  
pp. 74
Author(s):  
Pablo Valdés-Badilla ◽  
Rodrigo Ramirez-Campillo ◽  
Tomás Herrera-Valenzuela ◽  
Braulio Henrique Magnani Branco ◽  
Eduardo Guzmán-Muñoz ◽  
...  
Keyword(s):  
Systematic Review ◽  
Older Adults ◽  
Fall Risk ◽  
Meta Analysis ◽  
Combat Sports ◽  
High Quality ◽  
Grade Scale ◽  
Falls In Older Adults ◽  
Final Manuscript ◽  
Meta Analyses

This systematic review and meta-analysis aimed to assess the available body of published peer-reviewed articles related to the effects of Olympic combat sports (OCS), compared with active/passive controls, on balance, fall risk, or falls in older adults. The TESTEX and GRADE scales assessed the methodological quality and certainty of the evidence. The protocol was registered in PROSPERO (code: CRD42020204034). From 1496 records, eight studies were included, involving 322 older adults (64% female; mean age = 71.1 years). The TESTEX scale revealed all studies with a score ≥ 60% (moderate-high quality). The GRADE scale indicated all studies with at least some concerns, up to a high risk of bias (i.e., was rated very low). Meta-analyses were planned, although the reduced number of studies precluded its incorporation in the final manuscript. Only two from six studies that assessed balance found improvements after OCS compared to controls. No differences were found between OCS vs. control groups for fall risk or falls. The available evidence does not allow a definitive recommendation for or against OCS interventions as an effective strategy to improve balance and reduce the fall risk or falls in older adults. Therefore, more high-quality studies are required to draw definitive conclusions.

scholarly journals The Relationship Between Epilepsy and Marijuana

2018 ◽  
Vol 5 (1) ◽  
Author(s):  
Quinn Sully
Keyword(s):  
Meta Analysis ◽  
Health Benefits ◽  
Treatment Resistant ◽  
Positive Health ◽  
High Quality ◽  
The World ◽  
Resistant Form ◽  
Relationship Of ◽  
The Relationship

Epilepsy is a disorder in which several recurrent seizures occur, and despite the fact that there are over twenty anti-seizure drugs available, more than 30% of people with epilepsy continue to have seizures (Friedman & Devinsky, 2015; Kolb & Whishaw, 2009). Many researchers have turned to marijuana, specifically the constituent cannabidiol (CBD), as they search for new solutions to effectively help this treatment-resistant form of epilepsy. The purpose of this paper is to provide an assessment between the relationship of marijuana and epilepsy. I will review a total of six studies, including one case study, and one meta-analysis. A considerable amount of controversy surrounds this topic, as marijuana is illegal in many parts of the world, and many researchers are undecided as to whether its legalization will be beneficial or not. In spite of this disagreement, most researchers believe that marijuana, specifically CBD, has shown some evidence in regard to the positive health benefits and reduction of seizures in epilepsy. Future analysis requires high quality and reliable studies which can continue to further our understanding of the relationship between marijuana and epilepsy.

scholarly journals AMSTAR 2 Appraisal of Systematic Reviews and Meta-analyses in the Field of Heart Failure from High-impact Journals

2021 ◽  
Author(s):  
LIN LI ◽  
IRIAGBONSE ROTIMI ASEMOTA ◽  
BOLUN LIU ◽  
JAVIER GOMEZ-VALENZIA ◽  
LIFENG LIN ◽  
...  
Keyword(s):  
Heart Failure ◽  
Systematic Reviews ◽  
Meta Analysis ◽  
A Priori ◽  
Compliance Rate ◽  
High Impact ◽  
Measurement Tool ◽  
High Quality ◽  
Critical Appraisal Tool ◽  
Meta Analyses

Abstract Background: The MeaSurement Tool to Assess systematic Reviews (AMSTAR) 2 is a critical appraisal tool for systematic reviews (SRs) and meta-analyses (MAs) of interventions. We aimed to perform the first AMSTAR 2-based quality assessment of heart failure-related studies. Methods: Eleven high-impact journals were searched from 2009 to 2019. The included studies were assessed on the basis of 16 domains. Seven domains were deemed critical for high-quality studies. On the basis of the performance in these 16 domains with different weights, overall ratings were generated and the quality was determined to be “high,” “moderate,” “low,” or “critically low.” Results: Eighty-one heart failure-related SRs with MAs were included. Overall, 79 studies were of “critically low quality” and two were of “low quality.” These findings were attributed to insufficiency in the following critical domains: a priori protocols (compliance rate, 5%), complete list of exclusions with justification (5%), risk of bias assessment (69%), meta-analysis methodology (78%), and investigation of publication bias (60%).Conclusions: The low ratings for these potential high-quality heart failure-related SRs and MAs challenge the discrimination capacity of AMSTAR 2. In addition to identifying certain areas of insufficiency, these findings indicate the need to justify or modify AMSTAR 2’s rating rules.

Sign in / Sign up

Export Citation Format

Share Document