A Topical and Methodological Systematic Review of Meta-Analyses Published in the Educational Measurement Literature

This systematic review investigated the topics studied and reporting practices of published meta-analyses in educational measurement. Our findings indicated that meta-analysis is not a highly utilized methodological tool in educational measurement, as on average, less than one meta-analysis has been published per year over the last 30 years (28 meta-analyses were published between 1986 to 2016). Within the field, researchers have utilized meta-analysis to study three primary subject areas: test format effects, test accommodations, and predictive validity of operational testing programs. In regard to reporting practices, authors often failed to provide descriptive details of both their search strategy and sample characteristics limiting reproducibility and generalizability of findings, respectively. Furthermore, diagnostic analyses of outliers, publication bias, and statistical power were not provided for the majority of studies, putting into question the validity of inferences made from the meta-analyses sampled. The lack of transparent and replicable practices of meta-analyses in educational measurement are concerns for generating credible research syntheses that can assist the field in improving evidence-based practices. Recommendations are provided for improving training and editorial standards of meta-analytic research.

Download Full-text

What components of smoking cessation care during pregnancy are implemented by health providers? A systematic review and meta-analysis

BMJ Open ◽

10.1136/bmjopen-2018-026037 ◽

2019 ◽

Vol 9 (8) ◽

pp. e026037 ◽

Cited By ~ 2

Author(s):

Gillian Sandra Gould ◽

Laura Twyman ◽

Leah Stevenson ◽

Gabrielle R Gribbin ◽

Billie Bonevski ◽

...

Keyword(s):

Systematic Review ◽

Smoking Cessation ◽

Data Extraction ◽

Meta Analysis ◽

Self Report ◽

Health Providers ◽

Eligibility Criteria ◽

Reporting Practices ◽

Registration Number ◽

Meta Analyses

BackgroundPregnancy is an opportunity for health providers to support women to stop smoking.ObjectivesIdentify the pooled prevalence for health providers in providing components of smoking cessation care to women who smoke during pregnancy.DesignA systematic review synthesising original articles that reported on (1) prevalence of health providers’ performing the 5As (‘Ask’, ‘Advise’, ‘Assess’, ‘Assist’, ‘Arrange’), prescribing nicotine replacement therapy (NRT) and (2) factors associated with smoking cessation care.Data sourcesMEDLINE, EMBASE, CINAHL and PsycINFO databases searched using ‘smoking’, ‘pregnancy’ and ‘health provider practices’.Eligibility criteria for selecting studiesStudies included any design except interventions (self-report, audit, observed consultations and women’s reports), in English, with no date restriction, up to June 2017.ParticipantsHealth providers of any profession.Data extraction, appraisal and analysisData were extracted, then appraised with the Hawker tool. Meta-analyses pooled percentages for performing each of the 5As and prescribing NRT, using, for example, ‘often/always’ and ‘always/all’. Meta-regressions were performed of 5As for ‘often/always’.ResultsOf 3933 papers, 54 were included (n=29 225 participants): 33 for meta-analysis. Health providers included general practitioners, obstetricians, midwives and others from 10 countries. Pooled percentages of studies reporting practices ‘often/always’ were: ‘Ask’ (n=9) 91.6% (95% CI 88.2% to 95%); ‘Advise’ (n=7) 90% (95% CI 72.5% to 99.3%), ‘Assess’ (n=3) 79.2% (95% CI 76.5% to 81.8%), ‘Assist (cessation support)’ (n=5) 59.1% (95% CI 56% to 62.2%), ‘Arrange (referral)’ (n=6) 33.3% (95% CI 20.4% to 46.2%) and ‘prescribing NRT’ (n=6) 25.4% (95% CI 12.8% to 38%). Heterogeneity (I2) was 95.9%–99.1%. Meta-regressions for ‘Arrange’ were significant for year (p=0.013) and country (p=0.037).ConclusionsHealth providers ‘Ask’, ‘Advise’ and ‘Assess’ most pregnant women about smoking. ‘Assist’, ‘Arrange’ and ‘prescribing NRT’ are reported at lower rates: strategies to improve these should be considered.PROSPERO registration numberCRD42015029989.

Download Full-text

P-P39 Perioperative Interventions to Reduce Pancreatic Fistula following Pancreaticoduodenectomy: A Systematic Review and Meta-Analysis

British Journal of Surgery ◽

10.1093/bjs/znab430.261 ◽

2021 ◽

Vol 108 (Supplement_9) ◽

Author(s):

James Halle-Smith ◽

Rupaly Pande ◽

Lewis Hall ◽

James Hodson ◽

Keith J Roberts ◽

...

Keyword(s):

Systematic Review ◽

Pancreatic Fistula ◽

Statistical Power ◽

Fundamental Problem ◽

Meta Analysis ◽

New Directions ◽

Randomised Controlled ◽

Meta Analyses ◽

Post Hoc ◽

Pancreatic Duct Drainage

Abstract Background Many studies evaluate interventions to reduce POPF following PD, but often report conflicting results. Previous meta-analyses have generally included non-randomised trials and not considered novel interventions. Aim To evaluate interventions to reduce postoperative pancreatic fistula (POPF) following pancreatoduodenectomy (PD) with level 1 data. Methods A systematic review and meta-analysis assessed randomised controlled trials (RCTs) evaluating interventions to reduce All-POPF or clinically relevant (CR)-POPF after PD. A post-hoc analysis of negative RCTs assessed whether these had appropriate levels of statistical power. Results Among 22 interventions (n = 7,512 patients, 55 studies), 12 were assessed by multiple studies, and subject to meta-analysis. Of these, external pancreatic duct drainage was the only intervention found to be associated with significantly reduced rates of CR- and all-POPF. In addition, Ulinastatin was associated with significantly reduced rates of CR-POPF, whilst invagination (vs duct to mucosa) pancreatojejunostomy was associated with significantly reduced rates of all-POPF. Review of negative RCTs found the majority to be underpowered, with post-hoc power calculations indicating that interventions would need to reduce the POPF rate to ≤ 1% in order to achieve 80% power in 16/34 (All-POPF) and 19/25 (CR-POPF) studies, respectively. Conclusions Meta-analysis supports a role for several interventions to reduce POPF after PD, although data is often inconsistent and/or based on small trials. Systematic review identifies other interventions which may benefit from further study. However, underpowered trials appear to be a fundamental problem, inherently more so with CR-POPF. Larger trials, or new directions for research are required to further understanding in this field.

Download Full-text

The Contribution of “Individual Participant Data” Meta-Analyses of Psychotherapies for Depression to the Development of Personalized Treatments: A Systematic Review

Journal of Personalized Medicine ◽

10.3390/jpm12010093 ◽

2022 ◽

Vol 12 (1) ◽

pp. 93

Author(s):

Pim Cuijpers ◽

Marketa Ciharova ◽

Soledad Quero ◽

Clara Miguel ◽

Ellen Driessen ◽

...

Keyword(s):

Systematic Review ◽

Statistical Power ◽

Randomized Trials ◽

Meta Analysis ◽

Primary Data ◽

Individual Participant Data ◽

Psychological Treatments ◽

Gender Education ◽

Individual Participant ◽

Meta Analyses

While randomized trials typically lack sufficient statistical power to identify predictors and moderators of outcome, “individual participant data” (IPD) meta-analyses, which combine primary data of multiple randomized trials, can increase the statistical power to identify predictors and moderators of outcome. We conducted a systematic review of IPD meta-analyses on psychological treatments of depression to provide an overview of predictors and moderators identified. We included 10 (eight pairwise and two network) IPD meta-analyses. Six meta-analyses showed that higher baseline depression severity was associated with better outcomes, and two found that older age was associated with better outcomes. Because power was high in most IPD meta-analyses, non-significant findings are also of interest because they indicate that these variables are probably not relevant as predictors and moderators. We did not find in any IPD meta-analysis that gender, education level, or relationship status were significant predictors or moderators. This review shows that IPD meta-analyses on psychological treatments can identify predictors and moderators of treatment effects and thereby contribute considerably to the development of personalized treatments of depression.

Download Full-text

Mindfulness related changes in grey matter: a systematic review and meta‐analysis

Brain Imaging and Behavior ◽

10.1007/s11682-021-00453-4 ◽

2021 ◽

Author(s):

Cyril R. Pernet ◽

Nikolai Belov ◽

Arnaud Delorme ◽

Alison Zammit

Keyword(s):

Systematic Review ◽

Statistical Power ◽

Grey Matter ◽

Mindfulness Meditation ◽

Meta Analysis ◽

Likelihood Estimation ◽

Anterior Cingulate ◽

Magnetic Resonance Imaging Mri ◽

The Right ◽

Meta Analyses

AbstractKnowing target regions undergoing strfuncti changes caused by behavioural interventions is paramount in evaluating the effectiveness of such practices. Here, using a systematic review approach, we identified 25 peer-reviewed magnetic resonance imaging (MRI) studies demonstrating grey matter changes related to mindfulness meditation. An activation likelihood estimation (ALE) analysis (n = 16) revealed the right anterior ventral insula as the only significant region with consistent effect across studies, whilst an additional functional connectivity analysis indicates that both left and right insulae, and the anterior cingulate gyrus with adjacent paracingulate gyri should also be considered in future studies. Statistical meta-analyses suggest medium to strong effect sizes from Cohen’s d ~ 0.8 in the right insula to ~ 1 using maxima across the whole brain. The systematic review revealed design issues with selection, information, attrition and confirmation biases, in addition to weak statistical power. In conclusion, our analyses show that mindfulness meditation practice does induce grey matter changes but also that improvements in methodology are needed to establish mindfulness as a therapeutic intervention.

Download Full-text

Mindfulness related changes in grey matter: A Systematic Review and Meta-analysis

10.1101/2020.03.31.20049510 ◽

2020 ◽

Author(s):

Cyril R. Pernet ◽

Nikolai Belov ◽

Arnaud Delorme ◽

Alison Zammit

Keyword(s):

Systematic Review ◽

Statistical Power ◽

Grey Matter ◽

Structural Changes ◽

Mindfulness Meditation ◽

Meta Analysis ◽

Likelihood Estimation ◽

Anterior Cingulate ◽

The Right ◽

Meta Analyses

AbstractKnowing target regions undergoing structural changes caused by behavioural interventions is paramount in evaluating the effectiveness of such practices. Here, using a systematic review approach, we identified 25 peer-reviewed magnetic resonance imaging (MRI) studies demonstrating grey matter changes related to mindfulness meditation. An activation likelihood estimation (ALE) analysis (n=16) revealed the right anterior ventral insula as the only significant region with consistent effect across studies, whilst an additional functional connectivity analysis indicates that both left and right insulae, and the anterior cingulate gyrus with adjacent paracingulate gyri should also be considered in future studies. Statistical meta-analyses suggest medium to strong effect sizes from Cohen’s d ∼0.8 in the right insula to ∼1 using maxima across the whole brain. The systematic review revealed design issues with selection, information, attrition and confirmation biases, in addition to weak statistical power. In conclusion, our analyses show that mindfulness meditation practice does induce grey matter changes but also that improvements in methodology are needed to establish mindfulness as a therapeutic intervention.

Download Full-text

Power-Enhanced Funnel Plots for Meta-Analysis

Zeitschrift für Psychologie ◽

10.1027/2151-2604/a000392 ◽

2020 ◽

Vol 228 (1) ◽

pp. 43-49 ◽

Cited By ~ 1

Author(s):

Michael Kossmeier ◽

Ulrich S. Tran ◽

Martin Voracek

Keyword(s):

Statistical Power ◽

Meta Analysis ◽

Relevant Information ◽

Selective Reporting ◽

Graphical Display ◽

Funnel Plot ◽

Evidential Value ◽

Graphical Displays ◽

Funnel Plots ◽

Meta Analyses

Abstract. Currently, dedicated graphical displays to depict study-level statistical power in the context of meta-analysis are unavailable. Here, we introduce the sunset (power-enhanced) funnel plot to visualize this relevant information for assessing the credibility, or evidential value, of a set of studies. The sunset funnel plot highlights the statistical power of primary studies to detect an underlying true effect of interest in the well-known funnel display with color-coded power regions and a second power axis. This graphical display allows meta-analysts to incorporate power considerations into classic funnel plot assessments of small-study effects. Nominally significant, but low-powered, studies might be seen as less credible and as more likely being affected by selective reporting. We exemplify the application of the sunset funnel plot with two published meta-analyses from medicine and psychology. Software to create this variation of the funnel plot is provided via a tailored R function. In conclusion, the sunset (power-enhanced) funnel plot is a novel and useful graphical display to critically examine and to present study-level power in the context of meta-analysis.

Download Full-text

Impact of Chemotherapy in the Adjuvant Setting of Early Stage Uterine Leiomyosarcoma: A Systematic Review and Updated Meta-Analysis

Cancers ◽

10.3390/cancers12071899 ◽

2020 ◽

Vol 12 (7) ◽

pp. 1899

Author(s):

Alessandro Rizzo ◽

Margherita Nannini ◽

Annalisa Astolfi ◽

Valentina Indio ◽

Pierandrea De Iaco ◽

...

Keyword(s):

Systematic Review ◽

Recurrence Rate ◽

Locoregional Recurrence ◽

Early Stage ◽

Meta Analysis ◽

Distant Recurrence ◽

Uterine Leiomyosarcoma ◽

Primary Surgery ◽

Locoregional Recurrence Rate ◽

Meta Analyses

Background: Although the use of adjuvant chemotherapy (AC) appears to be increasing over the past few years, several clinical trials and previous meta-analyses failed to determine whether AC could improve clinical outcomes in uterine leiomyosarcoma (uLMS). The aim of this systematic review and meta-analysis was to compare AC (with or without radiotherapy) versus observation (obs) after primary surgery in early stage uLMS. Materials and Methods: Randomized controlled (RCTs) and non-randomized studies (NRSs) were retrieved. Outcomes of interest were as follows: distant recurrence rate, locoregional recurrence rate and overall recurrence rate. Results about distant recurrence rate, locoregional recurrence rate and overall recurrence rate were compared by calculating odds ratios (ORs) with 95% confidence intervals (CIs); ORs were combined with Mantel–Haenszel method. Results: Nine studies were included in the analysis, involving 545 patients (AC: 252, obs: 293). Compared with obs, AC did not reduce locoregional and distant recurrence rate, with a pooled OR of 1.36 and 0.63, respectively. Similarly, administration of AC did not decrease overall recurrence rate in comparison to obs. Conclusion: According to our results, AC (with or without radiotherapy) did not decrease recurrence rate in early stage uLMS; thus, the role of AC in this setting remains unclear.

Download Full-text

Systematic Review: Allogenic Use of Stromal Vascular Fraction (SVF) and Decellularized Extracellular Matrices (ECM) as Advanced Therapy Medicinal Products (ATMP) in Tissue Regeneration

International Journal of Molecular Sciences ◽

10.3390/ijms21144982 ◽

2020 ◽

Vol 21 (14) ◽

pp. 4982 ◽

Cited By ~ 1

Author(s):

Pietro Gentile ◽

Aris Sterodimas ◽

Jacopo Pizzicannella ◽

Laura Dionisi ◽

Domenico De Fazio ◽

...

Keyword(s):

Systematic Review ◽

Meta Analysis ◽

Stromal Vascular Fraction ◽

Lateral Epicondylitis ◽

Medicinal Products ◽

Advanced Therapy Medicinal Products ◽

Advanced Therapy ◽

Meta Analyses ◽

Clinical Experiments

Stromal vascular fraction (SVF) containing adipose stem cells (ASCs) has been used for many years in regenerative plastic surgery for autologous applications, without any focus on their potential allogenic role. Allogenic SVF transplants could be based on the possibility to use decellularized extracellular matrix (ECM) as a scaffold from a donor then re-cellularized by ASCs of the recipient, in order to develop the advanced therapy medicinal products (ATMP) in fully personalized clinical approaches. A systematic review of this field has been realized in accordance with the Preferred Reporting for Items for Systematic Reviews and Meta-Analyses-Protocols (PRISMA-P) guidelines. Multistep research of the PubMed, Embase, MEDLINE, Pre-MEDLINE, PsycINFO, CINAHL, Clinicaltrials.gov, Scopus database, and Cochrane databases has been conducted to identify articles and investigations on human allogenic ASCs transplant for clinical use. Of the 341 articles identified, 313 were initially assessed for eligibility on the basis of the abstract. Of these, only 29 met all the predetermined criteria for inclusion according to the PICOS (patients, intervention, comparator, outcomes, and study design) approach, and 19 have been included in quantitative synthesis (meta-analysis). Ninety-one percent of the studies previously screened (284 papers) were focused on the in vitro results and pre-clinical experiments. The allogenic use regarded the treatment of perianal fistulas, diabetic foot ulcers, knee osteoarthritis, acute respiratory distress syndrome, refractory rheumatoid arthritis, pediatrics disease, fecal incontinence, ischemic heart disease, autoimmune encephalomyelitis, lateral epicondylitis, and soft tissue defects. The information analyzed suggested the safety and efficacy of allogenic ASCs and ECM transplants without major side effects.

Download Full-text

Indications and outcomes of enterovesical and colovesical fistulas: systematic review of the literature and meta-analysis of prevalence

BMC Surgery ◽

10.1186/s12893-021-01272-6 ◽

2021 ◽

Vol 21 (1) ◽

Author(s):

Stefano Granieri ◽

Francesco Sessa ◽

Alessandro Bonomi ◽

Sissi Paleino ◽

Federica Bruno ◽

...

Keyword(s):

Systematic Review ◽

Clinical Symptoms ◽

Rare Complication ◽

Meta Analysis ◽

Treatment Options ◽

Palliative Surgery ◽

Review Of The Literature ◽

Colovesical Fistula ◽

Level Of Evidence ◽

Meta Analyses

Abstract Background Entero-colovesical fistula is a rare complication of various benign and malignant diseases. The diagnosis is prominently based on clinical symptoms; imaging studies are necessary not only to confirm the presence of the fistula, but more importantly to demonstrate the extent and the nature of the fistula. There is still a lack of consensus regarding the if, when and how to repair the fistula. The aim of the study is to review the different surgical treatment options, focus on surgical indications, and explore cumulative recurrence, morbidity, and mortality rates of entero-vesical and colo-vesical fistula patients. Methods A systematic review of the literature was conducted according to PRISMA guidelines. Random effects meta-analyses of proportions were developed to assess primary and secondary endpoints. I2 statistic and Cochran’s Q test were computed to assess inter-studies’ heterogeneity. Results Twenty-two studies were included in the analysis with a total of 861 patients. Meta-analyses of proportions pointed out 5, 22.2, and 4.9% rates for recurrence, complications, and mortality respectively. A single-stage procedure was performed in 75.5% of the cases, whereas a multi-stage operation in 15.5% of patients. Palliative surgery was performed in 6.2% of the cases. In 2.3% of the cases, the surgical procedure was not specified. Simple and advanced repair of the bladder was performed in 84.3% and 15.6% of the cases respectively. Conclusions Although burdened by a non-negligible rate of complications, surgical repair of entero-colovesical fistula leads to excellent results in terms of primary healing. Our review offers opportunities for significant further research in this field. Level of Evidence Level III according to ELIS (SR/MA with up to two negative criteria).

Download Full-text

Pre-Procedural Lumbar Neuraxial Ultrasound—A Systematic Review of Randomized Controlled Trials and Meta-Analysis

Healthcare ◽

10.3390/healthcare9040479 ◽

2021 ◽

Vol 9 (4) ◽

pp. 479

Author(s):

Tatiana Sidiropoulou ◽

Kalliopi Christodoulaki ◽

Charalampos Siristatidis

Keyword(s):

Systematic Review ◽

Lumbar Spine ◽

Randomized Controlled Trials ◽

Success Rate ◽

Meta Analysis ◽

Controlled Trials ◽

Technical Failure ◽

Obese Patients ◽

Randomized Controlled ◽

Meta Analyses

A pre-procedural ultrasound of the lumbar spine is frequently used to facilitate neuraxial procedures. The aim of this review is to examine the evidence sustaining the utilization of pre-procedural neuraxial ultrasound compared to conventional methods. We perform a systematic review of randomized controlled trials with meta-analyses. We search the electronic databases Medline, Cochrane Central, Science Direct and Scopus up to 1 June 2019. We include trials comparing a pre-procedural lumbar spine ultrasound to a non-ultrasound-assisted method. The primary endpoints are technical failure rate, first-attempt success rate, number of needle redirections and procedure time. We retrieve 32 trials (3439 patients) comparing pre-procedural lumbar ultrasounds to palpations for neuraxial procedures in various clinical settings. Pre-procedural ultrasounds decrease the overall risk of technical failure (Risk Ratio (RR) 0.69 (99% CI, 0.43 to 1.10), p = 0.04) but not in obese and difficult spinal patients (RR 0.53, p = 0.06) and increase the first-attempt success rate (RR 1.5 (99% CI, 1.22 to 1.86), p < 0.0001, NNT = 5). In difficult spines and obese patients, the RR is 1.84 (99% CI, 1.44 to 2.3; p < 0.0001, NNT = 3). The number of needle redirections is lower with pre-procedural ultrasounds (SMD = −0.55 (99% CI, −0.81 to −0.29), p < 0.0001), as is the case in difficult spines and obese patients (SMD = −0.85 (99% CI, −1.08 to −0.61), p < 0.0001). No differences are observed in procedural times. Ιn conclusion, a pre-procedural ultrasound provides significant benefit in terms of technical failure, number of needle redirections and first attempt-success rate. Τhe effect of pre-procedural ultrasound scanning of the lumbar spine is more significant in a subgroup analysis of difficult spines and obese patients.

Download Full-text