scholarly journals Examining Psychological Science through Systematic Meta-Method Analysis: A Call for Research

2019 ◽  
Author(s):  
Malte Elson
Keyword(s):  
Data Collection ◽  
Meta Analysis ◽  
Research Synthesis ◽  
Effect Sizes ◽  
Psychological Science ◽  
Meta Analyses ◽  
The Relationship ◽  
Method Analysis

Research synthesis is based on the assumption that when the same association between constructs is observed repeatedly in a field, the relationship is probably real, even if its exact magnitude can be debated. Yet this probability is not only a function of recurring results, but also of the quality and consistency in the empirical procedures that produced those results and that any meta-analysis necessarily inherits. Standardized protocols in data collection, analysis, and interpretation are important empirical properties, and a healthy sign of a discipline's maturity.This manuscript proposes that meta-analyses as typically applied in psychology benefit from complementing their aggregates of observed effect sizes by systematically examining the standardization of methodology that deterministically produced them. Potential units of analyses are described and two examples are offered to illustrate the benefits of such efforts. Ideally, this synergetic approach emphasizes the role of methods in advancing theory by improving the quality of meta-analytic inferences.

scholarly journals Examining Psychological Science through Systematic Meta-Method Analysis: A Call for Research

2019 ◽  
Author(s):  
Malte Elson
Keyword(s):  
Data Collection ◽  
Meta Analysis ◽  
Research Synthesis ◽  
Effect Sizes ◽  
Psychological Science ◽  
Meta Analyses ◽  
The Relationship ◽  
Method Analysis

Research synthesis is based on the assumption that when the same association between constructs is observed repeatedly in a field, the relationship is probably real, even if its exact magnitude can be debated. Yet this probability is not only a function of recurring results, but also of the quality and consistency in the empirical procedures that produced those results and that any meta-analysis necessarily inherits. Standardized protocols in data collection, analysis, and interpretation are important empirical properties, and a healthy sign of a discipline's maturity.This manuscript proposes that meta-analyses as typically applied in psychology benefit from complementing their aggregates of observed effect sizes by systematically examining the standardization of methodology that deterministically produced them. Potential units of analyses are described and two examples are offered to illustrate the benefits of such efforts. Ideally, this synergetic approach emphasizes the role of methods in advancing theory by improving the quality of meta-analytic inferences.

scholarly journals Examining Psychological Science Through Systematic Meta-Method Analysis: A Call for Research

2019 ◽  
Vol 2 (4) ◽  
pp. 350-363 ◽  
Author(s):  
Malte Elson
Keyword(s):  
Data Collection ◽  
Meta Analysis ◽  
Research Synthesis ◽  
Effect Sizes ◽  
Psychological Science ◽  
Units Of Analysis ◽  
Advance Theory ◽  
The Relationship ◽  
Method Analysis

Research synthesis is based on the assumption that when the same association between constructs is observed repeatedly in a field, the relationship is probably real, even if its exact magnitude can be debated. Yet the probability that the relationship is real is a function not only of recurring results, but also of the quality and consistency of the empirical procedures that produced those results and that any meta-analysis necessarily inherits. Standardized protocols in data collection, analysis, and interpretation are foundations of empiricism and a healthy sign of a discipline’s maturity. I propose that meta-analysis as typically applied in psychology will benefit from complementing aggregation of observed effect sizes with systematic examination of the standardization of the methodology that deterministically produced them. I describe potential units of analysis and offer two examples illustrating the benefits of such efforts. Ideally, this synergetic approach will advance theory by improving the quality of meta-analytic inferences.

scholarly journals An individual participant data (IPD) meta-analysis on the attachment network to multiple caregivers

2020 ◽  
Author(s):  
Or Dagan ◽  
Pasco Fearon ◽  
Carlo Schuengel ◽  
Marije Verhage ◽  
Glenn I. Roisman ◽  
...  
Keyword(s):  
Meta Analysis ◽  
Developmental Outcomes ◽  
Research Synthesis ◽  
Individual Participant Data ◽  
Attachment Relationship ◽  
Attachment Research ◽  
Individual Participant ◽  
The Subject ◽  
Meta Analyses

Since the seminal 1992 paper by van IJzendoorn, Sagi, and Lambermon, putting forward the “The multiple caretaker paradox”, relatively little attention has been given to the potential joint effects of the role early attachment network to mother and father play in development. Recently, Dagan and Sagi-Schwartz (2018) have published a paper that attempts to revive this unsettled issue, calling for research on the subject and offering a framework for posing attachment network hypotheses. This Collaboration for Attachment Research Synthesis project attempts to use an Individual Participant Data meta-analyses to test the hypotheses put forward in Dagan and Sagi-Schwartz (2018). Specifically, we test (a) whether the number of secure attachments (0,1, or 2) matter in predicting a range of developmental outcomes, and (b) whether the quality of attachment relationship with one parent contributes more than the other to these outcomes.

Linking Moral Identity With Moral Emotions: A Meta-Analysis

2019 ◽  
Vol 23 (4) ◽  
pp. 444-457 ◽  
Author(s):  
Jean Paul Lefebvre ◽  
Tobias Krettenauer
Keyword(s):  
Positive Emotions ◽  
Moral Identity ◽  
Meta Analysis ◽  
Moral Emotions ◽  
Effect Sizes ◽  
Moral Functioning ◽  
Moral Anger ◽  
Other Regarding ◽  
The Relationship

This meta-analysis examined the relationship between moral identity and moral emotions drawing on 57 independent studies. Moral identity was significantly associated with moral emotions, r = .32, p < .01, 95% confidence interval [CI: .27, .36]. Effect sizes were moderated by the type of moral emotion. Studies reporting other-regarding emotions (sympathy, empathy, and compassion) had the largest effect sizes ( r = .41), while negative other-evaluative emotions (moral anger, contempt, and disgust) had the smallest ( r = .16). Self-evaluative and other-evaluative positive emotions had intermediate effect sizes ( r values between .29 and .32). The type of emotion measure also was a significant moderator, with trait measures of emotion ( r = .38) correlating more strongly with moral identity than state measures ( r = .24). Effect sizes did not differ for the type of moral identity measure being used, publication status, or cultural origin of the study sample. The results of this meta-analysis demonstrate a robust empirical connection between moral identity and moral emotions, which confirms the multifaceted role of moral identity in moral functioning.

scholarly journals Comparative clinical trials in psychotherapy: Have large effects been replicated?

2020 ◽  
Vol 29 ◽  
Author(s):  
Nickolas D. Frost ◽  
Thomas W. Baskin ◽  
Bruce E. Wampold
Keyword(s):  
Clinical Trials ◽  
Treatment Efficacy ◽  
Literature Search ◽  
Meta Analysis ◽  
Effect Sizes ◽  
Depression And Anxiety ◽  
Inclusion Criteria ◽  
Average Quality ◽  
Meta Analyses

Abstract Aims The purpose of this review is to examine the replication attempts of psychotherapy clinical trials for depression and anxiety. We focus specifically on replications of trials that exhibit large differences between psychotherapies. The replicability of these trials is especially important for meta-analysis, where the inclusion of false-positive trials can lead to erroneous conclusions about treatment efficacy. Methods Standard replication criteria were developed to distinguish direct from conceptual replication methodologies. Next, an exhaustive literature search was conducted for published meta-analyses of psychotherapy comparisons. Trials that exhibited large effects (d > 0.8) were culled from these meta-analyses. For each trial, a cited replication was conducted to determine if the trial had been subsequently replicated by either ‘direct’ or ‘conceptual’ methods. Finally, a broader search was conducted to examine the extent of replication efforts in the psychotherapy literature overall. Results In the meta-analytic search, a total of N = 10 meta-analyses met the inclusion criteria. From these meta-analyses, N = 12 distinct trials exhibited large effect sizes. The meta-analyses containing more than two large effect trials reported evidence for treatment superiority. A cited replication search yielded no direct replication attempts (N = 0) for the trials with large effects, and N = 4 conceptual replication attempts of average or above average quality. However, of these four attempts, only two partially corroborated the results from their original trial. Conclusion Meta-analytic reviews are influenced by trials with large effects, and it is not uncommon for these reviews to contain several such trials. Since we find no evidence that trials with such large effects are directly replicable, treatment superiority conclusions from these reviews are highly questionable. To enhance the quality of clinical science, the development of authoritative replication criteria for clinical trials is needed. Moreover, quality benchmarks should be considered before trials are included in a meta-analysis, or replications are attempted.

scholarly journals The comparative effectiveness of physical exercise interventions in individuals with chronic non-specific neck pain: protocol for a network meta-analysis

BMJ Open ◽  
2020 ◽  
Vol 10 (5) ◽  
pp. e034846 ◽  
Author(s):  
Rutger MJ de Zoete ◽  
James H McAuley ◽  
Nigel R Armfield ◽  
Michele Sterling
Keyword(s):  
Physical Exercise ◽  
Neck Pain ◽  
Meta Analysis ◽  
Network Effect ◽  
Effect Sizes ◽  
Controlled Trials ◽  
Cochrane Central Register ◽  
Exercise Interventions ◽  
Meta Analyses

IntroductionNeck pain is a global burdensome problem, with a large proportion of neck pain cases becoming chronic. Although physical exercise is a commonly prescribed treatment, the evidence on the effectiveness of isolated exercise interventions remains limited. Traditional pairwise randomised controlled trials (RCTs) and meta-analyses are limited in only comparing two interventions. This protocol describes the design of a network meta-analysis, which enables a comparative investigation of all physical exercise interventions for which RCTs are available. We aim to systematically compare the effectiveness of different types of physical exercise in people with chronic non-specific neck pain.Methods and analysisNine electronic databases (AMED, CINAHL, Cochrane Central Register of Controlled Trials, Embase, MEDLINE, Physiotherapy Evidence Database, PsycINFO, Scopus and SPORTDiscus) were searched for RCTs from inception to 12 March 2019. Titles and abstract firstly, and full-text papers secondly, will be screened by two reviewers. Data will be extracted by two reviewers. The primary outcome measure is effectiveness of the intervention. Methodological quality of included studies will be assessed by two reviewers using the PEDro scale. The overall quality of evidence will be assessed with the Grading of Recommendations Assessment, Development and Evaluation (GRADE) framework, which has been adapted for network meta-analyses. The available evidence will be summarised using a network diagram. A contribution matrix will be presented to allow assessment of direct and indirect evidence. Forest plots will be constructed to visualise effects of all included exercise interventions. Pairwise effect sizes will be calculated by including all evidence available in the network. Effect measures for treatments that have not been compared in a pairwise RCT can be compared indirectly by contrasting effect sizes of comparisons with a common comparator.Ethics and disseminationThis work synthesises evidence from previously published studies and does not require ethics review or approval. A manuscript describing the findings will be submitted for publication in a peer-reviewed scientific journal.PROSPERO registration numberCRD42019126523.

scholarly journals 10 easy steps to conducting transparent, reproducible meta-analyses for infant researchers

2021 ◽  
Author(s):  
Loretta Gasparini ◽  
Sho Tsuji ◽  
Christina Bergmann
Keyword(s):  
Developmental Psychology ◽  
Evidence Synthesis ◽  
Meta Analysis ◽  
Research Question ◽  
The Body ◽  
Effect Sizes ◽  
Robust Methods ◽  
Meta Analyses ◽  
Analytic Models

Meta-analyses provide researchers with an overview of the body of evidence in a topic, with quantified estimates of effect sizes and the role of moderators, and weighting studies according to their precision. We provide a guide for conducting a transparent and reproducible meta-analysis in the field of developmental psychology within the framework of the MetaLab platform, in 10 steps: 1) Choose a topic for your meta-analysis, 2) Formulate your research question and specify inclusion criteria, 3) Preregister and carefully document all stages of your meta-analysis, 4) Conduct the literature search, 5) Collect and screen records, 6) Extract data from eligible studies, 7) Read the data into analysis software and compute effect sizes, 8) Create meta-analytic models to assess the strength of the effect and investigate possible moderators, 9) Visualize your data, 10) Write up and promote your meta-analysis. Meta-analyses can inform future studies, through power calculations, by identifying robust methods and exposing research gaps. By adding a new meta-analysis to MetaLab, datasets across multiple topics of developmental psychology can be synthesized, and the dataset can be maintained as a living, community-augmented meta-analysis to which researchers add new data, allowing for a cumulative approach to evidence synthesis.

scholarly journals Adolescent Sport Participation and Symptoms of Anxiety and Depression: A Systematic Review and Meta-Analysis

2020 ◽  
Vol 42 (3) ◽  
pp. 201-218
Author(s):  
Michael J. Panza ◽  
Scott Graupensperger ◽  
Jennifer P. Agans ◽  
Isabelle Doré ◽  
Stewart A. Vella ◽  
...  
Keyword(s):  
Systematic Review ◽  
Sport Participation ◽  
Causal Effect ◽  
Meta Analysis ◽  
Effect Sizes ◽  
Anxiety And Depression ◽  
Organized Sport ◽  
Protective Environment ◽  
Meta Analyses ◽  
The Relationship

Sport may protect against symptoms of mental disorders that are increasingly prevalent among adolescents. This systematic review explores the relationship between adolescent organized sport participation and self-reported symptoms of anxiety and depression. From 9,955 records screened, 29 unique articles were selected that included 61 effect sizes and 122,056 participants. Effects were clustered into four categories based on the operationalization of sport involvement: absence or presence of involvement, frequency of involvement, volume of involvement, and duration of participation. Results from the random-effects meta-analyses indicated that symptoms of anxiety and depression were significantly lower among sport-involved adolescents than in those not involved in sport, although this effect size was small in magnitude. Meta-regression was used to identify how age and sex explained heterogeneity in effects. Although these results do not signify a causal effect, they do support theorizing that sport participation during adolescence may be a protective environment against anxiety and depressive symptoms.

Assessing Study Quality in Meta-Analysis

2020 ◽  
Vol 46 (2-3) ◽  
pp. 334-342
Author(s):  
Thomas Hugh Feeley
Keyword(s):  
Meta Analysis ◽  
Effect Sizes ◽  
Study Quality ◽  
Acceptable Standard ◽  
Meta Analyses ◽  
Relationship Of

Abstract An assumption of meta-analyses is made with regard to the quality of the primary studies included for analysis. Specifically, the process assumes each study is a valid estimation of a hypothesized relationship of interest. In instances when a primary study's quality is below an acceptable standard, one option is for the study be excluded from further analyses. Alternatively, studies of acceptable merit could be further investigated through moderator analyses in an attempt to explain heterogeneity among effect sizes due to quality elements. The current essay discusses methods for evaluating study quality before proposing guidelines for their assessment. It is recommended that future meta-analyses in communication include a dedicated section detailing how study quality is addressed when reviewing studies for inclusion.

Abstract P155: Phenotyping The Association Between Nocturia And Hypertension: A Systematic Review And Meta-analysis

Hypertension ◽  
2020 ◽  
Vol 76 (Suppl_1) ◽  
Author(s):  
Syed N Rahman ◽  
Daniel J Cao ◽  
Thomas F Monaghan ◽  
Viktor X Flores ◽  
Matthew W Moy ◽  
...  
Keyword(s):  
Assessment Tool ◽  
Meta Analysis ◽  
Current Evidence ◽  
Quality Assessment Tool ◽  
Observable Effect ◽  
Cross Sectional ◽  
Observational Cohort ◽  
Meta Analyses ◽  
The Relationship

Introduction: The association between nocturia and hypertension has been widely reported, yet remains poorly characterized, precluding a more refined understanding of blood pressure as it relates to the clinical urology setting. We synthesized current evidence on the relationship between nocturia and hypertension as a function of nocturia severity, age, sex, race, and diuretic use. Methods: We searched PubMed, EMBASE, and Cochrane databases for studies published up to May 2020. Random effects meta-analyses were performed to identify the pooled odds ratio (OR) for nocturia given the presence of hypertension. Meta-regression and subgroup analyses were performed to identify differences across demographic factors. We applied the Quality Assessment Tool for Observational Cohort and Cross-Sectional Studies to evaluate the quality of evidence. Results: Of 1530 identified studies, 23 met the criteria for inclusion. The overall pooled OR for the association of hypertension with nocturia was 1.26 (95% CI 1.22-1.29, p<0.001). Pooled estimates were 1.21 (1.16-1.25, p<0.001) and 1.31 (1.26-1.36, p<0.001) using a 1- and 2-void cutoff for nocturia, respectively (p<0.001 between cutoffs). The association was stronger in females compared to males (1.41 [1.29-1.53] vs. 1.26 [1.20-1.32], p<0.001), and in African-American (1.56 [1.25-1.94]) and Asian (1.28 [1.24-1.32]) versus Caucasian populations (1.16 [1.09-1.24]) (p<0.05for both). Age and diuretic use had no observable effect on the association between nocturia and hypertension. Conclusions: There exists an increased odds of nocturia in hypertensive individuals. This association is stronger following a higher nocturia cutoff, in females, and in some racial subgroups, but unrelated to age and diuretic utilization.

Sign in / Sign up

Export Citation Format

Share Document