Coordinated Data Analysis: Knowledge Accumulation in Lifespan Developmental Psychology

Coordinated analysis is a powerful form of integrative analysis, and is well suited in its capacity to promote cumulative scientific knowledge, particularly in subfields of psychology that focus on the processes of lifespan development and aging. Coordinated analysis uses raw data from individual studies to create similar hypothesis tests for a given research question across multiple datasets, thereby making it less vulnerable to common criticisms of meta-analysis such as file drawer effects or publication bias. Coordinated analysis can sometimes use random effects meta-analysis to summarize results, which does not assume a single true effect size for a given statistical test. By fitting parallel models in separate datasets, coordinated analysis preserves the heterogeneity among studies, and provides a window into the generalizability and external validity of a set of results. The current paper achieves three goals: First, it describes the phases of a coordinated analysis so that interested researchers can more easily adopt these methods in their labs. Second, it discusses the importance of coordinated analysis within the context of the credibility revolution in psychology. Third, it encourages the use of existing data networks and repositories for conducting coordinated analysis, in order to enhance accessibility and inclusivity. Subfields of research that require time- or resource- intensive data collection, such as longitudinal aging research, would benefit by adopting these methods.

Download Full-text

10 easy steps to conducting transparent, reproducible meta-analyses for infant researchers

10.31219/osf.io/wp3fs ◽

2021 ◽

Author(s):

Loretta Gasparini ◽

Sho Tsuji ◽

Christina Bergmann

Keyword(s):

Developmental Psychology ◽

Evidence Synthesis ◽

Meta Analysis ◽

Research Question ◽

The Body ◽

Effect Sizes ◽

Robust Methods ◽

Meta Analyses ◽

Analytic Models

Meta-analyses provide researchers with an overview of the body of evidence in a topic, with quantified estimates of effect sizes and the role of moderators, and weighting studies according to their precision. We provide a guide for conducting a transparent and reproducible meta-analysis in the field of developmental psychology within the framework of the MetaLab platform, in 10 steps: 1) Choose a topic for your meta-analysis, 2) Formulate your research question and specify inclusion criteria, 3) Preregister and carefully document all stages of your meta-analysis, 4) Conduct the literature search, 5) Collect and screen records, 6) Extract data from eligible studies, 7) Read the data into analysis software and compute effect sizes, 8) Create meta-analytic models to assess the strength of the effect and investigate possible moderators, 9) Visualize your data, 10) Write up and promote your meta-analysis. Meta-analyses can inform future studies, through power calculations, by identifying robust methods and exposing research gaps. By adding a new meta-analysis to MetaLab, datasets across multiple topics of developmental psychology can be synthesized, and the dataset can be maintained as a living, community-augmented meta-analysis to which researchers add new data, allowing for a cumulative approach to evidence synthesis.

Download Full-text

Estimating the Effect of the File Drawer Problem in Meta-Analysis

Perceptual and Motor Skills ◽

10.2466/pms.1997.85.2.719 ◽

1997 ◽

Vol 85 (2) ◽

pp. 719-722 ◽

Cited By ~ 13

Author(s):

M. T. Bradley ◽

R. D. Gupta

Keyword(s):

Effect Size ◽

Meta Analysis ◽

True Effect Size ◽

True Effect ◽

File Drawer ◽

File Drawer Problem

Although meta-analysis appears to be a useful technique to verify the existence of an effect and to summarize large bodies of literature, there are problems associated with its use and interpretation. Amongst difficulties is the “file drawer problem.” With this problem it is assumed that a certain percentage of studies are not published or are not available to be included in any given meta-analysis. We present a cautionary table to quantify the magnitude of this problem. The table shows that distortions exaggerating the effect size are substantial and that the exaggerations of effects are strongest when the true effect size approaches zero. A meta-analysis could be very misleading were the true effect size close to zero.

Download Full-text

Genetic Determinants of Cortical Structure (Thickness, Surface Area and Volumes) among Disease Free Adults in the CHARGE Consortium

10.1101/409649 ◽

2018 ◽

Cited By ~ 4

Author(s):

Edith Hofer ◽

Gennady V. Roshchupkin ◽

Hieab H. H. Adams ◽

Maria J. Knol ◽

Honghuang Lin ◽

...

Keyword(s):

Surface Area ◽

Meta Analysis ◽

Genetic Correlations ◽

Brain Regions ◽

Aging Research ◽

Genomic Epidemiology ◽

Discovery Sample ◽

Genome Wide ◽

Development And Aging ◽

Pathway Analyses

AbstractCortical thickness, surface area and volumes (MRI cortical measures) vary with age and cognitive function, and in neurological and psychiatric diseases. We examined heritability, genetic correlations and genome-wide associations of cortical measures across the whole cortex, and in 34 anatomically predefined regions. Our discovery sample comprised 22,822 individuals from 20 cohorts within the Cohorts for Heart and Aging Research in Genomic Epidemiology (CHARGE) consortium and the United Kingdom Biobank. Significant associations were replicated in the Enhancing Neuroimaging Genetics through Meta-analysis (ENIGMA) consortium, and their biological implications explored using bioinformatic annotation and pathway analyses. We identified genetic heterogeneity between cortical measures and brain regions, and 161 genome-wide significant associations pointing to wnt/β-catenin, TGF-β and sonic hedgehog pathways. There was enrichment for genes involved in anthropometric traits, hindbrain development, vascular and neurodegenerative disease and psychiatric conditions. These data are a rich resource for studies of the biological mechanisms behind cortical development and aging.

Download Full-text

An Epistemological Criticism of Social Entrepreneurship: Is Social Entrepreneurship a Sound Scientific Field of Research

10.31124/advance.7091927.v1 ◽

2018 ◽

Author(s):

Amir Forouharfar

Keyword(s):

Social Entrepreneurship ◽

Social Constructivism ◽

Social Epistemology ◽

American English ◽

Meta Analysis ◽

Research Question ◽

Knowledge Structure ◽

Scientific Field ◽

The Other ◽

First Time

The paper was shaped around the pivotal question: Is SE a sound and scientific field of research? The question has given a critical tone to the paper and has also helped to bring out some of the controversial debates in the realm of SE. The paper was organized under five main discussions to be able to provide a scientific answer to the research question: (1) is “social entrepreneurship” an oxymoron?, (2) the characteristics of SE knowledge, (3) sources of social entrepreneurship knowledge, (4) SE knowledge: structure and limitations and (5) contributing epistemology-making concepts for SE. Based on the sections, the study relied on the relevant philosophical schools of thought in Epistemology (e.g. Empiricism, Rationalism, Skepticism, Internalism vs. Externalism, Essentialism, Social Constructivism, Social Epistemology, etc.) to discuss these controversies around SE and proposes some solutions by reviewing SE literature. Also, to determine the governing linguistic discourse in the realm of SE, which was necessary for our discussion, Corpus of Contemporary American English (COCA) for the first time in SE studies was used. Further, through the study, SE buzzwords which constitute SE terminology were derived and introduced to help us narrowing down and converging the thoughts in this field and demarking the epistemological boundaries of SE. The originality of the paper on one hand lies in its pioneering discussions on SE epistemology and on the other hand in paving the way for a construction of sound epistemology for SE; therefore in many cases after preparing the philosophical ground for the discussions, it went beyond the prevalent SE literature through meta-analysis to discuss the cases which were raised. The results of the study verified previously claimed embryonic pre-paradigmatic phase in SE which was far from a sound and scientific knowledge, although the scholarly endeavors are the harbingers of such a possibility in the future which calls for further mature academic discussion and development of SE knowledge by the SE academia.

Download Full-text

Study protocol of a systematic review and qualitative evidence synthesis using two different approaches: Healthcare related needs and desires of older people with post-stroke aphasia

BMJ Open ◽

10.1136/bmjopen-2020-039348 ◽

2021 ◽

Vol 11 (4) ◽

pp. e039348

Author(s):

Nadine Janis Pohontsch ◽

Thorsten Meyer ◽

Yvonne Eisenmann ◽

Maria-Inti Metzendorf ◽

Verena Leve ◽

...

Keyword(s):

Systematic Review ◽

Older People ◽

Evidence Synthesis ◽

Meta Analysis ◽

Research Question ◽

Treatment Success ◽

Qualitative Studies ◽

Qualitative Evidence Synthesis ◽

Healthcare Needs ◽

Qualitative Evidence

IntroductionStroke is a frequent disease in the older population of Western Europe with aphasia as a common consequence. Aphasia is known to impede targeting treatment to individual patients’ needs and therefore may reduce treatment success. In Germany, the postacute care of patients who had stroke is provided by different healthcare institutions of different sectors (rehabilitation, nursing and primary care) with substantial difficulties to coordinate services. We will conduct two qualitative evidence syntheses (QESs) aiming at exploring distinct healthcare needs and desires of older people living with poststroke aphasia. We thereby hope to support the development of integrated care models based on needs of patients who are very restricted to communicate them. Since various methods of QESs exist, the aim of the study embedding the two QESs was to determine if findings differ according to the approach used.Methods and analysisWe will conduct two QESs by using metaethnography (ME) and thematic synthesis (ThS) independently to synthesise the findings of primary qualitative studies. The main differences between these two methods are the underlying epistemologies (idealism (ME) vs realism (ThS)) and the type of research question (emerging (ME) vs fixed (ThS)).We will search seven bibliographical databases. Inclusion criteria comprise: patients with poststroke aphasia, aged 65 years and older, studies in German/English, all types of qualitative studies concerning needs and desires related to healthcare or the healthcare system. The protocol was registered in the International Prospective Register of Systematic Reviews, follows Preferred Reporting Items for Systematic Review and Meta-Analysis Protocols guidelines and includes three items from the Enhancing Transparency in Reporting the synthesis of Qualitative Research checklist.Ethics and disseminationEthical approval is not required. Findings will be published in a peer-reviewed journal and presented on national conferences.

Download Full-text

Abstract 3789: Nessun Dorma : Have the Risks of Rosiglitazone been Exaggerated?

Circulation ◽

10.1161/circ.116.suppl_16.ii_861-c ◽

2007 ◽

Vol 116 (suppl_16) ◽

Author(s):

George A Diamond ◽

Sanjay Kaul

Keyword(s):

Confidence Intervals ◽

Effect Size ◽

Fixed Effects ◽

Meta Analysis ◽

Cardiovascular Death ◽

Sensitivity Analyses ◽

Odds Ratios ◽

True Effect Size ◽

Index Study ◽

The Stability

Background A highly publicized meta-analysis of 42 clinical trials comprising 27,844 diabetics ignited a firestorm of controversy by charging that treatment with rosiglitazone was associated with a “…worrisome…” 43% greater risk of myocardial infarction ( p =0.03) and a 64% greater risk of cardiovascular death ( p =0.06). Objective The investigators excluded 4 trials from the infarction analysis and 19 trials from the mortality analysis in which no events were observed. We sought to determine if these exclusions biased the results. Methods We compared the index study to a Bayesian meta-analysis of the entire 42 trials (using odds ratio as the measure of effect size) and to fixed-effects and random-effects analyses with and without a continuity correction that adjusts for values of zero. Results The odds ratios and confidence intervals for the analyses are summarized in the Table . Odds ratios for infarction ranged from 1.43 to 1.22 and for death from 1.64 to 1.13. Corrected models resulted in substantially smaller odds ratios and narrower confidence intervals than did uncorrected models. Although corrected risks remain elevated, none are statistically significant (*p<0.05). Conclusions Given the fragility of the effect sizes and confidence intervals, the charge that roziglitazone increases the risk of adverse events is not supported by these additional analyses. The exaggerated values observed in the index study are likely the result of excluding the zero-event trials from analysis. Continuity adjustments mitigate this error and provide more consistent and reliable assessments of true effect size. Transparent sensitivity analyses should therefore be performed over a realistic range of the operative assumptions to verify the stability of such assessments especially when outcome events are rare. Given the relatively wide confidence intervals, additional data will be required to adjudicate these inconclusive results.

Download Full-text

Comparative Effectiveness Research: Using Systematic Reviews and Meta-Analyses to Synthesize Empirical Evidence

Research and Theory for Nursing Practice ◽

10.1891/1541-6577.25.3.191 ◽

2011 ◽

Vol 25 (3) ◽

pp. 191-209 ◽

Cited By ~ 2

Author(s):

Maria C. Katapodi ◽

Laurel L. Northouse

Keyword(s):

Systematic Reviews ◽

Comparative Effectiveness Research ◽

Comparative Effectiveness ◽

Meta Analysis ◽

Research Question ◽

Alternative Methods ◽

Evidence Based ◽

Effectiveness Research ◽

Meta Analyses ◽

Research Studies

The increased demand for evidence-based health care practices calls for comparative effectiveness research (CER), namely the generation and synthesis of research evidence to compare the benefits and harms of alternative methods of care. A significant contribution of CER is the systematic identification and synthesis of available research studies on a specific topic. The purpose of this article is to provide an overview of methodological issues pertaining to systematic reviews and meta-analyses to be used by investigators with the purpose of conducting CER. A systematic review or meta-analysis is guided by a research protocol, which includes (a) the research question, (b) inclusion and exclusion criteria with respect to the target population and studies, © guidelines for obtaining relevant studies, (d) methods for data extraction and coding, (e) methods for data synthesis, and (f ) guidelines for reporting results and assessing for bias. This article presents an algorithm for generating evidence-based knowledge by systematically identifying, retrieving, and synthesizing large bodies of research studies. Recommendations for evaluating the strength of evidence, interpreting findings, and discussing clinical applicability are offered.

Download Full-text

Relationship between hormonal contraceptives and sleep among women of reproductive age: a systematic review protocol

BMJ Open ◽

10.1136/bmjopen-2020-045819 ◽

2021 ◽

Vol 11 (10) ◽

pp. e045819

Author(s):

Jinhui Ma ◽

Megan Cheng ◽

Lehana Thabane ◽

Caihong Ma ◽

Ning Zhang ◽

...

Keyword(s):

Systematic Review ◽

Meta Analysis ◽

Research Question ◽

Risk Of Bias ◽

Reproductive Age ◽

Hormonal Contraceptives ◽

Controlled Trials ◽

Cochrane Central Register ◽

Women Of Reproductive Age

IntroductionThe aetiology of sleep disruptions is unknown, but hormonal fluctuations during the menstrual cycle, pregnancy and menopause have been shown to potentially affect how well a woman sleeps. The aim of this systematic review was to investigate whether hormonal contraceptives are associated with a decreased quality of sleep and increased sleep duration in women of reproductive age.MethodsThis review will analyse data from randomised controlled trials or non-randomised comparative studies investigating the association between hormonal contraceptives and sleep outcomes among women of reproductive age. Reviews addressing the same research question with similar eligibility criteria will be included. A literature search will be performed using the MEDLINE, Embase and Cochrane Central Register of Controlled Trials databases from inception to 7 March 2021. The Cochrane Collaboration’s Risk of Bias for Randomised Trials V.2.0 and The Risk of Bias for Non-randomised Studies of Interventions tool will be used to assess risk of bias for each outcome in eligible studies. Two reviewers will independently assess eligibility of studies and risk of bias and extract the data. All extracted data will be presented in tables and narrative form. For sleep measures investigated by two or more studies with low heterogeneity, we will conduct random-effects meta-analysis to estimate the magnitude of the overall effect of hormonal contraceptives. If studies included in this systematic review form a connected network, a network meta-analysis will be conducted to estimate the comparative effect of different contraceptives. The Grading of Recommendations, Assessment, Development, and Evaluation approach will be used to summarise the quality of evidence. Our protocol follows the Preferred Reporting Items for Systematic Reviews and Meta-Analyses for Protocols 2015 guidelines.Ethics and disseminationEthics approval is not required as data were sourced from previously reported studies. The findings of this review will be published in a peer-reviewed journal and presented at relevant conferences.PROSPERO registration numberCRD42020199958.

Download Full-text

Meta-analysis of Results from Multisite Studies

Handbook of Meta-analysis in Ecology and Evolution ◽

10.23943/princeton/9780691137285.003.0019 ◽

2013 ◽

Author(s):

Jessica Gurevitch

Keyword(s):

Experimental Work ◽

Research Group ◽

Quantitative Research ◽

Meta Analysis ◽

Research Question ◽

Research Synthesis ◽

Large Group ◽

Literature Reviews ◽

Analysis Of Results

Research synthesis in ecology has typically been based on literature reviews, as is also common in other fields. That is, a search is conducted for relevant data addressing a particular research question, the utility of published and unpublished data is assessed, and the results are synthesized to address questions based on all of the available evidence. This chapter discusses the case of individual researchers who wish to combine the results of distinct experiments that they have conducted themselves or within a single research group, sometimes over the course of many years. Such efforts have many similarities with literature-based quantitative research synthesis, but differ in some important ways. It begins with several examples of such work, and investigates the challenges, potential pitfalls, advantages, and issues involved in using meta-analysis for the synthesis of such large-group collaborative experimental work.

Download Full-text

Extraction and Critical Appraisal of Data

Handbook of Meta-analysis in Ecology and Evolution ◽

10.23943/princeton/9780691137285.003.0005 ◽

2013 ◽

Author(s):

Peter S. Curtis ◽

Kerrie Mengersen ◽

Marc J. Lajeunesse ◽

Hannah R. Rothstein ◽

Gavin B. Stewart

Keyword(s):

Relational Databases ◽

Critical Appraisal ◽

Data Extraction ◽

Meta Analysis ◽

Research Question ◽

Extraction Process ◽

Research Review ◽

Flat File ◽

Microsoft Access ◽

Definition Of

This chapter discusses the data extraction process, meta-analysis database, and critical appraisal of data. The efficient and accurate extraction of data from primary studies is an important component of successful research reviews. It is one of the most time-consuming parts of a research review and should be approached with the goal of repeatability and transparency of results. Careful definition of the research question and identification of the effect size metric(s) to be used are prerequisites to efficient data extraction. The extraction spreadsheet may simply be appended to a growing database stored in a single spreadsheet (also known as “flat file database”) (e.g., Microsoft Excel, Lotus, Quattro Pro), but it may be advantageous to develop relational databases (e.g., by using Microsoft Access, Paradox or dBase software), particularly for large or complex data. During the process of data extraction the investigator also has an opportunity for critical appraisal of data quality. One approach to quantitative assessment of study quality has been the use of numerical scales in which points are assigned to specific elements of the study and summed to produce an overall quality score.

Download Full-text