scholarly journals 10 easy steps to conducting transparent, reproducible meta-analyses for infant researchers

2021 ◽  
Author(s):  
Loretta Gasparini ◽  
Sho Tsuji ◽  
Christina Bergmann
Keyword(s):  
Developmental Psychology ◽  
Evidence Synthesis ◽  
Meta Analysis ◽  
Research Question ◽  
The Body ◽  
Effect Sizes ◽  
Robust Methods ◽  
Meta Analyses ◽  
Analytic Models

Meta-analyses provide researchers with an overview of the body of evidence in a topic, with quantified estimates of effect sizes and the role of moderators, and weighting studies according to their precision. We provide a guide for conducting a transparent and reproducible meta-analysis in the field of developmental psychology within the framework of the MetaLab platform, in 10 steps: 1) Choose a topic for your meta-analysis, 2) Formulate your research question and specify inclusion criteria, 3) Preregister and carefully document all stages of your meta-analysis, 4) Conduct the literature search, 5) Collect and screen records, 6) Extract data from eligible studies, 7) Read the data into analysis software and compute effect sizes, 8) Create meta-analytic models to assess the strength of the effect and investigate possible moderators, 9) Visualize your data, 10) Write up and promote your meta-analysis. Meta-analyses can inform future studies, through power calculations, by identifying robust methods and exposing research gaps. By adding a new meta-analysis to MetaLab, datasets across multiple topics of developmental psychology can be synthesized, and the dataset can be maintained as a living, community-augmented meta-analysis to which researchers add new data, allowing for a cumulative approach to evidence synthesis.

scholarly journals Examining Psychological Science through Systematic Meta-Method Analysis: A Call for Research

2019 ◽  
Author(s):  
Malte Elson
Keyword(s):  
Data Collection ◽  
Meta Analysis ◽  
Research Synthesis ◽  
Effect Sizes ◽  
Psychological Science ◽  
Meta Analyses ◽  
The Relationship ◽  
Method Analysis

Research synthesis is based on the assumption that when the same association between constructs is observed repeatedly in a field, the relationship is probably real, even if its exact magnitude can be debated. Yet this probability is not only a function of recurring results, but also of the quality and consistency in the empirical procedures that produced those results and that any meta-analysis necessarily inherits. Standardized protocols in data collection, analysis, and interpretation are important empirical properties, and a healthy sign of a discipline's maturity.This manuscript proposes that meta-analyses as typically applied in psychology benefit from complementing their aggregates of observed effect sizes by systematically examining the standardization of methodology that deterministically produced them. Potential units of analyses are described and two examples are offered to illustrate the benefits of such efforts. Ideally, this synergetic approach emphasizes the role of methods in advancing theory by improving the quality of meta-analytic inferences.

scholarly journals Exploring the Role of Persuasive Design in Unguided ICBT for Depression and Anxiety: Systematic Review, Meta-Analysis, and Meta-Regression (Preprint)

2021 ◽  
Author(s):  
Hugh Cameron McCall ◽  
Heather D Hadjistavropoulos ◽  
Christopher Richard Francis Sundström
Keyword(s):  
Systematic Review ◽  
Meta Analysis ◽  
Design Principles ◽  
Effect Sizes ◽  
Controlled Trials ◽  
Depression And Anxiety ◽  
Randomized Controlled ◽  
Meta Regression ◽  
Meta Analyses

BACKGROUND Internet-delivered cognitive behavioural therapy (ICBT) is an effective treatment that can overcome barriers to care. Various research groups have suggested that unguided ICBT (i.e., ICBT without therapist support) and other eHealth interventions can be designed in ways that enhance user engagement and thus outcomes. The persuasive systems design framework captures most design recommendations for eHealth interventions, but there is little empirical support that persuasive design is related to clinical outcomes in unguided ICBT. OBJECTIVE One objective of this study was to provide an updated meta-analysis of randomized controlled trials of unguided ICBT for depression and anxiety. Another objective was to describe the use of persuasive design in such interventions. A third objective was to employ meta-regression to explore whether a greater number of persuasive design elements predicts efficacy in unguided ICBT for depression and anxiety. METHODS We conducted a systematic review of five databases to find randomized controlled trials of unguided ICBT for depression and anxiety. We conducted separate random-effects meta-analyses and separate meta-regressions for depression and anxiety interventions. Each meta-regression included two steps. The first step included, as a predictor, whether each intervention was transdiagnostic. For the meta-regression of ICBT for depression, the first step also included the type of control condition. The number of persuasive design principles identified for each intervention was added as a predictor in the second step to reveal the additional variance in effect sizes explained by persuasive design. RESULTS We identified 4,471 articles in our search, 46 of which were eligible for inclusion in our analyses. Our meta-analyses showed effect sizes (Hedges’ g) ranging from 0.22 to 0.31 for depression interventions, depending on the measures taken to account for bias in the results. We found a mean effect size of 0.45 for anxiety interventions, with no evidence that results were inflated by bias. Included interventions were identified as employing an average of 4.88 persuasive design principles. The meta-regressions showed that a greater number of persuasive design principles predicted greater efficacy in ICBT for depression (R2 change=.27, B=0.04, P=.02) but not anxiety (R2 change=.05, B=0.03, P=.17). CONCLUSIONS These findings provide preliminary support for the proposition that more persuasively designed ICBT interventions are more efficacious, at least in the treatment of depression. Further research is needed to clarify the role of persuasive design in ICBT. CLINICALTRIAL PROSPERO (https://www.crd.york.ac.uk/prospero/) CRD42020153466

Systematic reviews and meta-analysis

2021 ◽  
pp. 193-206
Author(s):  
Nandi Siegfried ◽  
Lawrence Mbuagbaw
Keyword(s):  
Systematic Reviews ◽  
Evidence Synthesis ◽  
Meta Analysis ◽  
Research Question ◽  
Similar Data ◽  
Inaccurate Information ◽  
Healthcare Decision Making ◽  
Scoping Reviews ◽  
Meta Analyses

Systematic reviews play an important role in healthcare decision-making. When conducted correctly, they provide up-to-date, comprehensive, and replicable summaries of evidence. Authors of systematic reviews are expected to develop a protocol that outlines the research question and key methodological features of their review. A comprehensive and exhaustive search should be conducted, followed by screening to capture studies that meet the prespecified inclusion criteria. Once the relevant studies have been identified, data will be extracted, using a dedicated tool that permits the review authors to confirm the eligibility of the study and collect information on its design, risk of bias, and results. Sufficiently similar data may be pooled using meta-analytic techniques or synthesized narratively. A summary of the overall quality of evidence for each outcome is an essential component of a systematic review. The main concerns with systematic reviews are (1) selection bias: systematic exclusion of relevant studies due to publication status or language; (2) indexing bias: failure to identify relevant studies because they are not indexed accurately; and (3) information bias: missing or inaccurate information in the included studies. Other approaches to evidence synthesis include mapping the evidence with scoping reviews; conducting overviews of systematic reviews; using individual patient data; conducting network meta-analyses for multiple comparisons; conducting rapid reviews when evidence is needed urgently; synthesis of diagnostic accuracy data; and synthesis of qualitative data. Systematic reviews often inform clinical guidelines and require careful planning and execution by teams with content and methodological expertise.

Methodological Guidance Paper: High-Quality Meta-Analysis in a Systematic Review

2019 ◽  
Vol 90 (1) ◽  
pp. 24-46 ◽  
Author(s):  
Terri D. Pigott ◽  
Joshua R. Polanin
Keyword(s):  
Systematic Review ◽  
Best Practice ◽  
Meta Analysis ◽  
Research Question ◽  
Effect Sizes ◽  
Statistical Techniques ◽  
High Quality ◽  
Systematic Review Methods ◽  
Study Results ◽  
Meta Analyses

This methodological guidance article discusses the elements of a high-quality meta-analysis that is conducted within the context of a systematic review. Meta-analysis, a set of statistical techniques for synthesizing the results of multiple studies, is used when the guiding research question focuses on a quantitative summary of study results. In this guidance article, we discuss the systematic review methods that support high-quality meta-analyses and outline best practice meta-analysis methods for describing the distribution of effect sizes in a set of eligible studies. We also provide suggestions for transparently reporting the methods and results of meta-analyses to influence practice and policy. Given the increasing use of meta-analysis for important policy decisions, the methods and results of meta-analysis should be both transparent and reproducible.

scholarly journals Examining Psychological Science through Systematic Meta-Method Analysis: A Call for Research

2019 ◽  
Author(s):  
Malte Elson
Keyword(s):  
Data Collection ◽  
Meta Analysis ◽  
Research Synthesis ◽  
Effect Sizes ◽  
Psychological Science ◽  
Meta Analyses ◽  
The Relationship ◽  
Method Analysis

Research synthesis is based on the assumption that when the same association between constructs is observed repeatedly in a field, the relationship is probably real, even if its exact magnitude can be debated. Yet this probability is not only a function of recurring results, but also of the quality and consistency in the empirical procedures that produced those results and that any meta-analysis necessarily inherits. Standardized protocols in data collection, analysis, and interpretation are important empirical properties, and a healthy sign of a discipline's maturity.This manuscript proposes that meta-analyses as typically applied in psychology benefit from complementing their aggregates of observed effect sizes by systematically examining the standardization of methodology that deterministically produced them. Potential units of analyses are described and two examples are offered to illustrate the benefits of such efforts. Ideally, this synergetic approach emphasizes the role of methods in advancing theory by improving the quality of meta-analytic inferences.

The Orchard Plot: Cultivating a Forest Plot for Use in Ecology, Evolution and Beyond

2019 ◽  
Author(s):  
Shinichi Nakagawa ◽  
Malgorzata Lagisz ◽  
Rose E O'Dea ◽  
Joanna Rutkowska ◽  
Yefeng Yang ◽  
...  
Keyword(s):  
Meta Analysis ◽  
R Package ◽  
Effect Sizes ◽  
Forest Plot ◽  
Point Estimates ◽  
Aggregate Effect ◽  
The Individual ◽  
Meta Analyses ◽  
Heterogeneous Effect ◽  
Intuitive Interpretation

‘Classic’ forest plots show the effect sizes from individual studies and the aggregate effect from a meta-analysis. However, in ecology and evolution meta-analyses routinely contain over 100 effect sizes, making the classic forest plot of limited use. We surveyed 102 meta-analyses in ecology and evolution, finding that only 11% use the classic forest plot. Instead, most used a ‘forest-like plot’, showing point estimates (with 95% confidence intervals; CIs) from a series of subgroups or categories in a meta-regression. We propose a modification of the forest-like plot, which we name the ‘orchard plot’. Orchard plots, in addition to showing overall mean effects and CIs from meta-analyses/regressions, also includes 95% prediction intervals (PIs), and the individual effect sizes scaled by their precision. The PI allows the user and reader to see the range in which an effect size from a future study may be expected to fall. The PI, therefore, provides an intuitive interpretation of any heterogeneity in the data. Supplementing the PI, the inclusion of underlying effect sizes also allows the user to see any influential or outlying effect sizes. We showcase the orchard plot with example datasets from ecology and evolution, using the R package, orchard, including several functions for visualizing meta-analytic data using forest-plot derivatives. We consider the orchard plot as a variant on the classic forest plot, cultivated to the needs of meta-analysts in ecology and evolution. Hopefully, the orchard plot will prove fruitful for visualizing large collections of heterogeneous effect sizes regardless of the field of study.

Identification of and Correction for Publication Bias: Comment

2019 ◽  
Author(s):  
Amanda Kvarven ◽  
Eirik Strømland ◽  
Magnus Johannesson
Keyword(s):  
Publication Bias ◽  
False Positive ◽  
Large Scale ◽  
Meta Analysis ◽  
False Positive Rate ◽  
Effect Sizes ◽  
Replication Studies ◽  
Moderate Reduction ◽  
Positive Rate ◽  
Meta Analyses

Andrews & Kasy (2019) propose an approach for adjusting effect sizes in meta-analysis for publication bias. We use the Andrews-Kasy estimator to adjust the result of 15 meta-analyses and compare the adjusted results to 15 large-scale multiple labs replication studies estimating the same effects. The pre-registered replications provide precisely estimated effect sizes, which do not suffer from publication bias. The Andrews-Kasy approach leads to a moderate reduction of the inflated effect sizes in the meta-analyses. However, the approach still overestimates effect sizes by a factor of about two or more and has an estimated false positive rate of between 57% and 100%.

Effectiveness of psychological interventions to improve emotion regulation in youth - a meta-analysis

2019 ◽  
Author(s):  
Bettina Moltrecht ◽  
Jessica Deighton ◽  
Praveetha Patalay ◽  
Julian Childs
Keyword(s):  
Emotion Regulation ◽  
Emotion Dysregulation ◽  
Psychological Intervention ◽  
Meta Analysis ◽  
Age Groups ◽  
Effect Sizes ◽  
What Works ◽  
Large Heterogeneity ◽  
Larger Sample

Background: Research investigating the role of emotion regulation (ER) in the development and treatment of psychopathology has increased in recent years. Evidence suggests that an increased focus on ER in treatment can improve existing interventions. Most ER research has neglected young people, therefore the present meta-analysis summarizes the evidence for existing psychosocial intervention and their effectiveness to improve ER in youth. Methods: A systematic review and meta-analysis was conducted according to the PRISMA guidelines. Twenty-one randomized-control-trials (RCTs) assessed changes in ER following a psychological intervention in youth exhibiting various psychopathological symptoms.Results: We found moderate effect sizes for current interventions to decrease emotion dysregulation in youth (g=-.46) and small effect sizes to improve emotion regulation (g=0.36). Significant differences between studies including intervention components, ER measures and populations studied resulted in large heterogeneity. Conclusion: This is the first meta-analysis that summarizes the effectiveness for existing interventions to improve ER in youth. The results suggest that interventions can enhance ER in youth, and that these improvements correlate with improvements in psychopathology. More RCTs including larger sample sizes, different age groups and psychopathologies are needed to increase our understanding of what works for who and when.

scholarly journals A protocol for the systematic review and meta-analysis of thigmotactic behaviour in the open field test in rodent models associated with persistent pain

2021 ◽  
Vol 5 (1) ◽  
pp. e100135
Author(s):  
Xue Ying Zhang ◽  
Jan Vollert ◽  
Emily S Sena ◽  
Andrew SC Rice ◽  
Nadia Soliman
Keyword(s):  
Systematic Review ◽  
Outcome Measure ◽  
Meta Analysis ◽  
Persistent Pain ◽  
Effect Sizes ◽  
Bias Assessment ◽  
Animal Pain ◽  
Trim And Fill ◽  
Meta Analyses ◽  
The Impact

ObjectiveThigmotaxis is an innate predator avoidance behaviour of rodents and is enhanced when animals are under stress. It is characterised by the preference of a rodent to seek shelter, rather than expose itself to the aversive open area. The behaviour has been proposed to be a measurable construct that can address the impact of pain on rodent behaviour. This systematic review will assess whether thigmotaxis can be influenced by experimental persistent pain and attenuated by pharmacological interventions in rodents.Search strategyWe will conduct search on three electronic databases to identify studies in which thigmotaxis was used as an outcome measure contextualised to a rodent model associated with persistent pain. All studies published until the date of the search will be considered.Screening and annotationTwo independent reviewers will screen studies based on the order of (1) titles and abstracts, and (2) full texts.Data management and reportingFor meta-analysis, we will extract thigmotactic behavioural data and calculate effect sizes. Effect sizes will be combined using a random-effects model. We will assess heterogeneity and identify sources of heterogeneity. A risk-of-bias assessment will be conducted to evaluate study quality. Publication bias will be assessed using funnel plots, Egger’s regression and trim-and-fill analysis. We will also extract stimulus-evoked limb withdrawal data to assess its correlation with thigmotaxis in the same animals. The evidence obtained will provide a comprehensive understanding of the strengths and limitations of using thigmotactic outcome measure in animal pain research so that future experimental designs can be optimised. We will follow the Preferred Reporting Items for Systematic Reviews and Meta-Analyses reporting guidelines and disseminate the review findings through publication and conference presentation.

scholarly journals Study protocol of a systematic review and qualitative evidence synthesis using two different approaches: Healthcare related needs and desires of older people with post-stroke aphasia

BMJ Open ◽  
2021 ◽  
Vol 11 (4) ◽  
pp. e039348
Author(s):  
Nadine Janis Pohontsch ◽  
Thorsten Meyer ◽  
Yvonne Eisenmann ◽  
Maria-Inti Metzendorf ◽  
Verena Leve ◽  
...  
Keyword(s):  
Systematic Review ◽  
Older People ◽  
Evidence Synthesis ◽  
Meta Analysis ◽  
Research Question ◽  
Treatment Success ◽  
Qualitative Studies ◽  
Qualitative Evidence Synthesis ◽  
Healthcare Needs ◽  
Qualitative Evidence

IntroductionStroke is a frequent disease in the older population of Western Europe with aphasia as a common consequence. Aphasia is known to impede targeting treatment to individual patients’ needs and therefore may reduce treatment success. In Germany, the postacute care of patients who had stroke is provided by different healthcare institutions of different sectors (rehabilitation, nursing and primary care) with substantial difficulties to coordinate services. We will conduct two qualitative evidence syntheses (QESs) aiming at exploring distinct healthcare needs and desires of older people living with poststroke aphasia. We thereby hope to support the development of integrated care models based on needs of patients who are very restricted to communicate them. Since various methods of QESs exist, the aim of the study embedding the two QESs was to determine if findings differ according to the approach used.Methods and analysisWe will conduct two QESs by using metaethnography (ME) and thematic synthesis (ThS) independently to synthesise the findings of primary qualitative studies. The main differences between these two methods are the underlying epistemologies (idealism (ME) vs realism (ThS)) and the type of research question (emerging (ME) vs fixed (ThS)).We will search seven bibliographical databases. Inclusion criteria comprise: patients with poststroke aphasia, aged 65 years and older, studies in German/English, all types of qualitative studies concerning needs and desires related to healthcare or the healthcare system. The protocol was registered in the International Prospective Register of Systematic Reviews, follows Preferred Reporting Items for Systematic Review and Meta-Analysis Protocols guidelines and includes three items from the Enhancing Transparency in Reporting the synthesis of Qualitative Research checklist.Ethics and disseminationEthical approval is not required. Findings will be published in a peer-reviewed journal and presented on national conferences.

Sign in / Sign up

Export Citation Format

Share Document