scholarly journals A meta-review of transparency and reproducibility-related reporting practices in published meta-analyses on clinical psychological interventions (2000–2020)

2021 ◽  
Author(s):  
Rubén López-Nicolás ◽  
José Antonio López-López ◽  
María Rubio-Aparicio ◽  
Julio Sánchez-Meca
Keyword(s):  
Clinical Psychology ◽  
Scientific Method ◽  
Meta Analysis ◽  
Psychological Interventions ◽  
Reporting Practices ◽  
Analysis Techniques ◽  
Key Points ◽  
Meta Review ◽  
Code Sharing ◽  
Meta Analyses

AbstractMeta-analysis is a powerful and important tool to synthesize the literature about a research topic. Like other kinds of research, meta-analyses must be reproducible to be compliant with the principles of the scientific method. Furthermore, reproducible meta-analyses can be easily updated with new data and reanalysed applying new and more refined analysis techniques. We attempted to empirically assess the prevalence of transparency and reproducibility-related reporting practices in published meta-analyses from clinical psychology by examining a random sample of 100 meta-analyses. Our purpose was to identify the key points that could be improved, with the aim of providing some recommendations for carrying out reproducible meta-analyses. We conducted a meta-review of meta-analyses of psychological interventions published between 2000 and 2020. We searched PubMed, PsycInfo and Web of Science databases. A structured coding form to assess transparency indicators was created based on previous studies and existing meta-analysis guidelines. We found major issues concerning: completely reproducible search procedures report, specification of the exact method to compute effect sizes, choice of weighting factors and estimators, lack of availability of the raw statistics used to compute the effect size and of interoperability of available data, and practically total absence of analysis script code sharing. Based on our findings, we conclude with recommendations intended to improve the transparency, openness, and reproducibility-related reporting practices of meta-analyses in clinical psychology and related areas.

scholarly journals Transparency and Reproducibility-related Reporting Practices in Meta-analyses on the Effectiveness of Psychological Interventions

2021 ◽  
Author(s):  
Rubén López-Nicolás ◽  
José A López-López ◽  
María Rubio Aparicio ◽  
Julio Sánchez-Meca
Keyword(s):  
Clinical Psychology ◽  
Scientific Method ◽  
Meta Analysis ◽  
Psychological Interventions ◽  
Reporting Practices ◽  
Analysis Techniques ◽  
Key Points ◽  
Meta Review ◽  
Code Sharing ◽  
Meta Analyses

Meta-analysis is a powerful and important tool to synthesize the literature about a research topic. Like other kinds of research, meta-analyses must be reproducible to be compliant with the principles of the scientific method. Furthermore, reproducible meta-analyses can be easily updated with new data and reanalysed applying new and more refined analysis techniques. We attempted to empirically assess the prevalence of transparency and reproducibility-related reporting practices in published meta-analyses from clinical psychology by examining a random sample of 100 meta-analyses. Our purpose was to identify the key points that could be improved with the aim to provide some recommendations to carry out reproducible meta-analyses. We conducted a meta-review of meta-analyses of psychological interventions published between 2000 and 2019. We searched PubMed, PsycInfo and Web of Science databases. A structured coding form to assess transparency indicators was created based on previous studies and existing meta-analysis guidelines. We found major issues concerning: completely reproducible search procedures report, specification of the exact method to compute effect sizes, choice of weighting factors and estimators, lack of availability of the raw statistics used to compute the effect size and of interoperability of available data, and practically total absence of analysis script code sharing. Based on our findings, we conclude with recommendations intended to improve the transparency, openness and reproducibility-related reporting practices of meta-analyses in clinical psychology and related areas.

Transparency and Reproducibility of Meta-Analyses in Psychology: A Meta-Review

2020 ◽  
Vol 15 (4) ◽  
pp. 1026-1041 ◽  
Author(s):  
Joshua R. Polanin ◽  
Emily A. Hennessy ◽  
Sho Tsuji
Keyword(s):  
Effect Size ◽  
Best Practice ◽  
Meta Analysis ◽  
Eligibility Criteria ◽  
Reporting Practices ◽  
Transparent Reporting ◽  
Meta Review ◽  
Meta Analyses ◽  
High Degree ◽  
Specific Search

Systematic review and meta-analysis are possible as viable research techniques only through transparent reporting of primary research; thus, one might expect meta-analysts to demonstrate best practice in their reporting of results and have a high degree of transparency leading to reproducibility of their work. This assumption has yet to be fully tested in the psychological sciences. We therefore aimed to assess the transparency and reproducibility of psychological meta-analyses. We conducted a meta-review by sampling 150 studies from Psychological Bulletin to extract information about each review’s transparent and reproducible reporting practices. The results revealed that authors reported on average 55% of criteria and that transparent reporting practices increased over the three decades studied ( b = 1.09, SE = 0.24, t = 4.519, p < .001). Review authors consistently reported eligibility criteria, effect-size information, and synthesis techniques. Review authors, however, on average, did not report specific search results, screening and extraction procedures, and most importantly, effect-size and moderator information from each individual study. Far fewer studies provided statistical code required for complete analytical replication. We argue that the field of psychology and research synthesis in general should require review authors to report these elements in a transparent and reproducible manner.

scholarly journals A systematic review and network meta-analysis of current and investigational treatments for active ankylosing spondylitis

2020 ◽  
Vol 39 (8) ◽  
pp. 2307-2315 ◽  
Author(s):  
A. Deodhar ◽  
S. D. Chakravarty ◽  
C. Cameron ◽  
S. Peterson ◽  
R. Hensman ◽  
...  
Keyword(s):  
Ankylosing Spondylitis ◽  
Meta Analysis ◽  
Placebo Response ◽  
Baseline Risk ◽  
C Reactive Protein ◽  
Investigational Agent ◽  
Reactive Protein ◽  
Key Points ◽  
Investigational Therapies ◽  
Meta Analyses

Abstract Objective To compare the relative efficacy of current and investigational biologic and oral small molecule (OSM) treatments for active ankylosing spondylitis (AS). Methods A systematic literature review was conducted to identify all phase 2/3 randomized trials of interest in patients with AS. Outcomes assessed were ≥ 20% improvement in the Assessment of Spondyloarthritis International Society Criteria (ASAS20) and change from baseline in Bath Ankylosing Spondylitis Functional Index (BASFI) and C-reactive protein (CRP) at weeks 12–16. Bayesian network meta-analyses were conducted for outcomes using a random effects model. Baseline-risk adjustment was also conducted to account for differences in placebo response across studies. Surface Under the Cumulative Ranking curve (SUCRA) values are reported, reflecting the relative probability that intervention was the best of all interventions. Results The investigational agent tofacitinib 5 mg was the top-ranked treatment (SUCRA, 93%) for ASAS20 response, followed by intravenous (IV) golimumab 2 mg/kg (90%). Golimumab IV 2 mg/kg and infliximab 5 mg/kg were the top two ranked treatments for change from baseline in BASFI (golimumab IV, 81%; infliximab, 80%) and change from baseline in CRP (infliximab, 90%; golimumab IV, 82%). Conclusions Two approved therapies (golimumab IV, infliximab) and one investigational product ranked highest for efficacy in AS. Key Points• Although golimumab IV, infliximab, and tofacitinib ranked highest for efficacy in AS, differences in efficacy between approved and investigational therapies were not statistically significant.

scholarly journals What components of smoking cessation care during pregnancy are implemented by health providers? A systematic review and meta-analysis

BMJ Open ◽  
2019 ◽  
Vol 9 (8) ◽  
pp. e026037 ◽  
Author(s):  
Gillian Sandra Gould ◽  
Laura Twyman ◽  
Leah Stevenson ◽  
Gabrielle R Gribbin ◽  
Billie Bonevski ◽  
...  
Keyword(s):  
Systematic Review ◽  
Smoking Cessation ◽  
Data Extraction ◽  
Meta Analysis ◽  
Self Report ◽  
Health Providers ◽  
Eligibility Criteria ◽  
Reporting Practices ◽  
Registration Number ◽  
Meta Analyses

BackgroundPregnancy is an opportunity for health providers to support women to stop smoking.ObjectivesIdentify the pooled prevalence for health providers in providing components of smoking cessation care to women who smoke during pregnancy.DesignA systematic review synthesising original articles that reported on (1) prevalence of health providers’ performing the 5As (‘Ask’, ‘Advise’, ‘Assess’, ‘Assist’, ‘Arrange’), prescribing nicotine replacement therapy (NRT) and (2) factors associated with smoking cessation care.Data sourcesMEDLINE, EMBASE, CINAHL and PsycINFO databases searched using ‘smoking’, ‘pregnancy’ and ‘health provider practices’.Eligibility criteria for selecting studiesStudies included any design except interventions (self-report, audit, observed consultations and women’s reports), in English, with no date restriction, up to June 2017.ParticipantsHealth providers of any profession.Data extraction, appraisal and analysisData were extracted, then appraised with the Hawker tool. Meta-analyses pooled percentages for performing each of the 5As and prescribing NRT, using, for example, ‘often/always’ and ‘always/all’. Meta-regressions were performed of 5As for ‘often/always’.ResultsOf 3933 papers, 54 were included (n=29 225 participants): 33 for meta-analysis. Health providers included general practitioners, obstetricians, midwives and others from 10 countries. Pooled percentages of studies reporting practices ‘often/always’ were: ‘Ask’ (n=9) 91.6% (95% CI 88.2% to 95%); ‘Advise’ (n=7) 90% (95% CI 72.5% to 99.3%), ‘Assess’ (n=3) 79.2% (95% CI 76.5% to 81.8%), ‘Assist (cessation support)’ (n=5) 59.1% (95% CI 56% to 62.2%), ‘Arrange (referral)’ (n=6) 33.3% (95% CI 20.4% to 46.2%) and ‘prescribing NRT’ (n=6) 25.4% (95% CI 12.8% to 38%). Heterogeneity (I2) was 95.9%–99.1%. Meta-regressions for ‘Arrange’ were significant for year (p=0.013) and country (p=0.037).ConclusionsHealth providers ‘Ask’, ‘Advise’ and ‘Assess’ most pregnant women about smoking. ‘Assist’, ‘Arrange’ and ‘prescribing NRT’ are reported at lower rates: strategies to improve these should be considered.PROSPERO registration numberCRD42015029989.

An Empirical Review of Research and Reporting Practices in Psychological Meta-Analyses

2020 ◽  
Vol 24 (3) ◽  
pp. 195-209
Author(s):  
Richard E. Hohn ◽  
Kathleen L. Slaney ◽  
Donna Tafreshi
Keyword(s):  
Meta Analysis ◽  
Reporting Standards ◽  
Exclusion Criteria ◽  
Research Practices ◽  
Reporting Practices ◽  
Reporting Decisions ◽  
Research Findings ◽  
State Of Research ◽  
Meta Analyses ◽  
Made In

As meta-analytic studies have come to occupy a sizable contingent of published work in the psychological sciences, clarity in the research and reporting practices of such work is crucial to the interpretability and reproducibility of research findings. The present study examines the state of research and reporting practices within a random sample of 384 published psychological meta-analyses across several important dimensions (e.g., search methods, exclusion criteria, statistical techniques). In addition, we surveyed the first authors of the meta-analyses in our sample to ask them directly about the research practices employed and reporting decisions made in their studies, including the assessments and procedures they conducted and the guidelines or materials they relied on. Upon cross-validating the first author responses with what was reported in their published meta-analyses, we identified numerous potential gaps in reporting and research practices. In addition to providing a survey of recent reporting practices, our findings suggest that (a) there are several research practices conducted by meta-analysts that are ultimately not reported; (b) some aspects of meta-analysis research appear to be conducted at disappointingly low rates; and (c) the adoption of the reporting standards, including the Meta-Analytic Reporting Standards (MARS), has been slow to nonexistent within psychological meta-analytic research.

Can Psychological Interventions Reduce Perfectionism? A Systematic Review and Meta-analysis

2014 ◽  
Vol 43 (6) ◽  
pp. 705-731 ◽  
Author(s):  
Samantha Lloyd ◽  
Ulrike Schmidt ◽  
Mizanur Khondoker ◽  
Kate Tchanturia
Keyword(s):  
Systematic Review ◽  
Psychiatric Disorders ◽  
Meta Analysis ◽  
The Self ◽  
Effect Sizes ◽  
Psychological Interventions ◽  
Personal Standards ◽  
Socially Prescribed Perfectionism ◽  
Concern Over Mistakes ◽  
Meta Analyses

Background:Perfectionism is implicated in a range of psychiatric disorders, impedes treatment and is associated with poorer treatment outcomes.Aims:The aim of this systematic review and meta-analysis was to summarize the existing evidence for psychological interventions targeting perfectionism in individuals with psychiatric disorders associated with perfectionism and/or elevated perfectionism.Method: Eight studies were identified and were analysed in meta-analyses. Meta-analyses were carried out for the Personal Standards and Concern over Mistakes subscales of the Frost Multi-Dimensional Perfectionism Scale (FMPS) and the Self Orientated Perfectionism and Socially Prescribed Perfectionism subscales of the Hewitt and Flett MPS (HMPS) in order to investigate change between pre and postintervention.Results:Large pooled effect sizes were found for the Personal Standards and Concern over Mistakes subscales of the FMPS and the Self Orientated Perfectionism subscale of the HMPS, whilst a medium sized effect was found for change in Socially Prescribed Perfectionism. Medium pooled effect sizes were also found for symptoms of anxiety and depression.Conclusions:There is some support that it is possible to significantly reduce perfectionism in individuals with clinical disorders associated with perfectionism and/or clinical levels of perfectionism. There is also some evidence that such interventions are associated with decreases in anxiety, depression, eating disorder and obsessive compulsive symptoms. Further research is needed in order to investigate the optimal dosage and format of such interventions as well as into specific disorders where there is a lack of evidence for their effectiveness.

Selection and Evaluation of Empirical Research in Technology Assessment

1989 ◽  
Vol 5 (4) ◽  
pp. 521-536 ◽  
Author(s):  
Thomas C. Chalmers ◽  
Peg Hewett ◽  
Dinah Reitman ◽  
Henry S. Sacks
Keyword(s):  
Clinical Trials ◽  
Technology Assessment ◽  
Scientific Method ◽  
Meta Analysis ◽  
Sensitivity Analyses ◽  
Controlled Trials ◽  
Data Bases ◽  
Randomized Controlled ◽  
Practice Of Medicine ◽  
Meta Analyses

Technology assessment involves application of the scientific method to the practice of medicine. Finding all of the assessment reports in a given field is not an easy task. Proper evaluation of those assessments requires the conduct of a prospective experiment in which the sources and results are blinded when the choice is made of papers to exclude and to include, and the process should be carried in duplicate. There are several available data bases for carrying out the search, but because of problems they should be supplemented by reference to the bibliographies of pertinent published articles. Clinical trials included in meta-analyses should be graded by quality and thus facilitate sensitivity analyses. Attention must be paid to the possibility of publication bias. Finally, the advent of meta-analysis makes it desirable to begin randomized controlled trials in areas of uncertainty, even when there is no possibility that individual investigators will encounter enough patients to draw valid conclusions.

scholarly journals Internet- and Mobile-Based Interventions for Mental and Somatic Conditions in Children and Adolescents

2020 ◽  
Vol 48 (1) ◽  
pp. 33-46 ◽  
Author(s):  
Matthias Domhardt ◽  
Lena Steubl ◽  
Harald Baumeister
Keyword(s):  
Mental Disorders ◽  
Children And Adolescents ◽  
Effect Size ◽  
Treatment Effects ◽  
Meta Analysis ◽  
Mental Healthcare ◽  
Quality Ratings ◽  
Meta Review ◽  
Mean Differences ◽  
Meta Analyses

Abstract. This meta-review integrates the current meta-analysis literature on the efficacy of internet- and mobile-based interventions (IMIs) for mental disorders and somatic diseases in children and adolescents. Further, it summarizes the moderators of treatment effects in this age group. Using a systematic literature search of PsycINFO and MEDLINE/PubMed, we identified eight meta-analyses (N = 8,417) that met all inclusion criteria. Current meta-analytical evidence of IMIs exists for depression (range of standardized mean differences, SMDs = .16 to .76; 95 % CI: –.12 to 1.12; k = 3 meta-analyses), anxiety (SMDs = .30 to 1.4; 95 % CI: –.53 to 2.44; k = 5) and chronic pain (SMD = .41; 95 % CI: .07 to .74; k = 1) with predominantly nonactive control conditions (waiting-list; placebo). The effect size for IMIs across mental disorders reported in one meta-analysis is SMD = 1.27 (95 % CI: .96 to 1.59; k = 1), the effect size of IMIs for different somatic conditions is SMD = .49 (95 % CI: .33 to .64; k = 1). Moderators of treatment effects are age (k = 3), symptom severity (k = 1), and source of outcome assessment (k = 1). Quality ratings with the AMSTAR-2-checklist indicate acceptable methodological rigor of meta-analyses included. Taken together, this meta-review suggests that IMIs are efficacious in some health conditions in youths, with evidence existing primarily for depression and anxiety so far. The findings point to the potential of IMIs to augment evidence based mental healthcare for children and adolescents.

A hidden pandemic? An umbrella review of global evidence on mental health in the time of COVID-19

2022 ◽  
Author(s):  
Marlee Bower ◽  
Scarlett Smout ◽  
Amarina Donohoe-Bales ◽  
Lily Teesson ◽  
Eleisha Lauria ◽  
...  
Keyword(s):  
Mental Health ◽  
General Population ◽  
Health Outcomes ◽  
Systematic Reviews ◽  
Meta Analysis ◽  
Mental Health Outcomes ◽  
Eligibility Criteria ◽  
Pooled Prevalence ◽  
Meta Review ◽  
Meta Analyses

Introduction: Vast available international evidence has investigated the mental health impacts of the COVID-19 pandemic. This review aims to synthesise evidence, identifying populations and characteristics associated with poor mental health.Methods: A meta-review of pooled prevalence of anxiety and depression, with subgroup analyses for the general population, healthcare workers (HCW) and COVID-19 patients; and a meta-synthesis of systematic reviews to collate evidence on associated factors and further mental disorders. Databases searched included Scopus, Embase, PsycINFO, and MEDLINE dated to May 2021. Eligibility criteria included systematic reviews and/or meta-analyses, published post-November 2019, reporting data in English on mental health outcomes during the pandemic.Results: Eighty-one systematic reviews were included, 51 of which incorporated meta-analysis. Meta-review overall anxiety prevalence was 29% (95%CI: 27–31%, I2: 99.83%), with subgroup prevalence as 35% (95%CI: 23–47%, I2: 97.4%) in COVID-19 patients, 29% in HCW (95%CI: 25– 32, I2: 99.8%) and 28% in the general population (95%CI: 25–31%, I2: 99.9%). Meta-review overall depression prevalence was 28% (95%CI: 26–30%, I2: 99.7), with subgroup prevalence as 30% (95%CI: 7–60%, I2: 99.8%) in COVID-19 patients, 28% (95%CI: 25–31%, I2: 99.7%) in HCW and 27% (95%CI: 25–30, I2: 99.8%) in the general population. Meta-synthesis found many experienced psychological distress and PTSD/PTSS during COVID-19, but pooled prevalence ranged substantially. Fear of, proximity to, or confirmed COVID-19 infection; undergoing quarantine; and COVID-19-related news exposure were associated with adverse mental health outcomes. Amongst other factors, people who are younger, female, LGBTIQ, pregnant, parents or experiencing low social support, financial issues or socio-economic disadvantage, tended to have poorer mental health during the pandemic period.Conclusions: Despite high volumes of reviews, the diversity of findings and dearth of longitudinal studies within reviews means clear links between COVID-19 and mental health are not available, although existing evidence indicates probable associations.

Recommendations for Reviewing Meta-Analyses in Organizational Research

2020 ◽  
pp. 109442812096708
Author(s):  
Justin A. DeSimone ◽  
Michael T. Brannick ◽  
Ernest H. O’Boyle ◽  
Ji Woon Ryu
Keyword(s):  
Data Analysis ◽  
Review Process ◽  
Meta Analysis ◽  
Sufficient Information ◽  
Organizational Research ◽  
Data Presentation ◽  
Reporting Practices ◽  
Additional Information ◽  
Transparent Reporting ◽  
Meta Analyses

This article encourages transparency in the reporting of meta-analytic procedures. Specifically, we highlight aspects of meta-analytic search, coding, data presentation, and data analysis where published meta-analyses often fall short in presenting sufficient information to allow replication. We identify opportunities where reviewers can request additional information or analyses that will enhance transparent reporting practices and facilitate the evaluation of quality in meta-analytic reporting. We focus on concerns specific to (or prevalent in) meta-analyses conducted in organizational research. In doing so, we reference a number of existing and emerging techniques, highlighting their contribution to meta-analysis while emphasizing key information reviewers may request. Our focus is primarily on meta-analyses, but secondary uses of meta-analytic data are also considered. We conclude by providing a checklist for reviewers in an effort to facilitate the review process as it pertains to the goals of transparency and replicability.

Sign in / Sign up

Export Citation Format

Share Document