scholarly journals Pooling resources to enhance rigour in psychophysiological research: Insights from open science approaches to meta-analysis

2021 ◽  
Vol 162 ◽  
pp. 112-120
Author(s):  
Blair Saunders ◽  
Michael Inzlicht
Keyword(s):  
Meta Analysis ◽  
Open Science ◽  
Psychophysiological Research

Race, Ethnicity, and the Scarr-Rowe Hypothesis: A Cautionary Example of Fringe Science Entering the Mainstream

2021 ◽  
Author(s):  
Evan Giangrande ◽  
Eric Turkheimer
Keyword(s):  
Systematic Review ◽  
Ethnic Groups ◽  
Meta Analysis ◽  
Scientific Journal ◽  
The United States ◽  
Open Science ◽  
Group Differences ◽  
Ethnic Group Differences ◽  
Race Ethnicity ◽  
Mean Differences

In 2020, Pesta et al. published “Racial and ethnic group differences in the heritability of intelligence: A systematic review and meta-analysis” in Intelligence. The authors frame their analysis as an examination of the Scarr-Rowe hypothesis, which holds that the heritability of intelligence varies as a function of socioeconomic status. Pesta et al. (2020) conclude that the heritability of intelligence does not differ across racial and ethnic groups in the United States. They claim their results challenge the Scarr-Rowe hypothesis and support the hereditarian position that mean differences in IQ among racial and ethnic groups are attributable to genetic differences rather than environmental disparities. In this reply, we outline severe theoretical, methodological, and rhetorical flaws in every step of Pesta et al.’s meta-analysis. The most reliable finding Pesta et al. report is consistent with the Scarr-Rowe hypothesis and directly contradicts a hereditarian understanding of group differences in intelligence. Finally, we suggest that Pesta et al. (2020) serves as an example of how racially motivated and poorly executed work can find its way into a mainstream scientific journal, underscoring the importance of robust peer review and rigorous editorial judgement in the open science era.

scholarly journals Comparison of COVID-19 outcomes in organ transplant recipients (OTr) and non-transplant patients: a study protocol for rapid review

2021 ◽  
Vol 10 (1) ◽  
Author(s):  
Alexis H. Lerner ◽  
Elizabeth J. Klein ◽  
Anna Hardesty ◽  
Orestis A. Panagiotou ◽  
Chelsea Misquith ◽  
...  
Keyword(s):  
Meta Analysis ◽  
Organ Transplant ◽  
Open Science ◽  
Rapid Review ◽  
Secondary Outcome ◽  
Severe Illness ◽  
Link Type ◽  
Increased Risk ◽  
Potential Sources ◽  
Measure Of Association

Abstract Background The COVID-19 pandemic has devastated the global community with nearly 4.9 million deaths as of October 2021. While organ transplant (OT) recipients (OTr) may be at increased risk for severe COVID-19 due to their chronic immunocompromised state, outcomes for OTr with COVID-19 remain disputed in the literature. This review will examine whether OTr with COVID-19 are at higher risk for severe illness and death than non-immunocompromised individuals. Methods MEDLINE (via Ovid and PubMed) and EMBASE (via Embase.com) will be searched from December 2019 to October 2021 for observational studies (including cohort and case-control) that compare COVID-19 clinical outcomes in OTr to those in individuals without history of OT. The primary outcome of interest will be mortality as defined in each study, with possible further analyses of in-hospital mortality, 28 or 30-day mortality, and all-cause mortality versus mortality attributable to COVID-19. The secondary outcome of interest will be the severity of COVID-19 disease, most frequently defined as requiring intensive care unit admission or mechanical ventilation. Two reviewers will independently screen all abstracts and full-text articles. Potential conflicts will be resolved by a third reviewer and potentially discussion among all investigators. Methodological quality will be appraised using the Newcastle-Ottawa Scale. If data permit, we will perform random-effects meta-analysis with the Sidik-Jonkman estimator and the Hartung-Knapp adjustment for confidence intervals to estimate a summary measure of association between histories of transplant with each outcome. Potential sources of heterogeneity will be explored using meta-regression. Additional analyses will be conducted to explore the potential sources of heterogeneity (e.g., subgroup analysis) considering least minimal adjustment for confounders. Discussion This rapid review will assess the available evidence on whether OTr diagnosed with COVID-19 are at higher risk for severe illness and death compared to non-immunocompromised individuals. Such knowledge is clinically relevant and may impact risk stratification, allocation of organs and healthcare resources, and organ transplantation protocols during this, and future, pandemics. Systematic review registration Open Science Framework (OSF) registration DOI: 10.17605/osf.io/4n9d7.

scholarly journals Investigation of Risk Of Bias due to Unreported and SelecTively included results in meta-analyses of nutrition research: the ROBUST study protocol

F1000Research ◽  
2019 ◽  
Vol 8 ◽  
pp. 1760 ◽  
Author(s):  
Matthew J. Page ◽  
Lisa Bero ◽  
Cynthia M. Kroeger ◽  
Zhaoli Dai ◽  
Sally McDonald ◽  
...  
Keyword(s):  
Meta Analysis ◽  
Scientific Evidence ◽  
Open Science ◽  
Included Study ◽  
Risk Of Bias ◽  
Dietary Guidelines ◽  
Healthy Population ◽  
Nutrition Research ◽  
Health Related ◽  
Meta Analyses

Background: Dietary guidelines should be informed by systematic reviews (SRs) of the available scientific evidence. However, if the SRs that underpin dietary guidelines are flawed in their design, conduct or reporting, the recommendations contained therein may be misleading or harmful. To date there has been little empirical investigation of bias due to selective inclusion of results, and bias due to missing results, in SRs of food/diet-outcome relationships. Objectives: To explore in SRs with meta-analyses of the association between food/diet and health-related outcomes: (i) whether systematic reviewers selectively included study effect estimates in meta-analyses when multiple effect estimates were available; (ii) what impact selective inclusion of study effect estimates may have on meta-analytic effects, and; (iii) the risk of bias due to missing results (publication bias and selective non-reporting bias) in meta-analyses. Methods: We will systematically search for SRs with meta-analysis of the association between food/diet and health-related outcomes in a generally healthy population, published between January 2018 and June 2019. We will randomly sort titles and abstracts and screen them until we identify 50 eligible SRs. The first reported meta-analysis of a binary or continuous outcome in each SR (the ‘index meta-analysis’) will be evaluated. We will extract from study reports all study effect estimates that were eligible for inclusion in the index meta-analyses (e.g. from multiple instruments and time points) and will quantify and test for evidence of selective inclusion of results. We will also assess the risk of bias due to missing results in the index meta-analyses using a new tool (ROB-ME). Ethics and dissemination: Ethics approval is not required because information will only be extracted from published studies. Dissemination of the results will be through peer-reviewed publications and presentations at conferences. We will make all data collected from this study publicly available via the Open Science Framework.

scholarly journals Investigation of Risk Of Bias due to Unreported and SelecTively included results in meta-analyses of nutrition research: the ROBUST study protocol

F1000Research ◽  
2020 ◽  
Vol 8 ◽  
pp. 1760
Author(s):  
Matthew J. Page ◽  
Lisa Bero ◽  
Cynthia M. Kroeger ◽  
Zhaoli Dai ◽  
Sally McDonald ◽  
...  
Keyword(s):  
Meta Analysis ◽  
Scientific Evidence ◽  
Open Science ◽  
Included Study ◽  
Risk Of Bias ◽  
Dietary Guidelines ◽  
Healthy Population ◽  
Nutrition Research ◽  
Health Related ◽  
Meta Analyses

Background: Dietary guidelines should be informed by systematic reviews (SRs) of the available scientific evidence. However, if the SRs that underpin dietary guidelines are flawed in their design, conduct or reporting, the recommendations contained therein may be misleading or harmful. To date there has been little empirical investigation of bias due to selective inclusion of results, and bias due to missing results, in SRs of food/diet-outcome relationships. Objectives: To explore in SRs with meta-analyses of the association between food/diet and health-related outcomes: (i) whether systematic reviewers selectively included study effect estimates in meta-analyses when multiple effect estimates were available; (ii) what impact selective inclusion of study effect estimates may have on meta-analytic effects, and; (iii) the risk of bias due to missing results (publication bias and selective non-reporting bias) in meta-analyses. Methods: We will systematically search for SRs with meta-analysis of the association between food/diet and health-related outcomes in a generally healthy population, published between January 2018 and June 2019. We will randomly sort titles and abstracts and screen them until we identify 50 eligible SRs. The first reported meta-analysis of a binary or continuous outcome in each SR (the ‘index meta-analysis’) will be evaluated. We will extract from study reports all study effect estimates that were eligible for inclusion in the index meta-analyses (e.g. from multiple instruments and time points) and will quantify and test for evidence of selective inclusion of results. We will also assess the risk of bias due to missing results in the index meta-analyses using a new tool (ROB-ME). Ethics and dissemination: Ethics approval is not required because information will only be extracted from published studies. Dissemination of the results will be through peer-reviewed publications and presentations at conferences. We will make all data collected from this study publicly available via the Open Science Framework.

scholarly journals Abordagens de reúso e a questão da reusabilidade dos dados científicos | Approaches for data reuse and the issue of scientific data reusability

2019 ◽  
Vol 15 (2) ◽  
Author(s):  
Renata Curty
Keyword(s):  
Data Sharing ◽  
Data Science ◽  
Meta Analysis ◽  
Science Research ◽  
Open Science ◽  
Scientific Data ◽  
Data Reuse ◽  
Data Repositories ◽  
Documentation Quality ◽  
Data Documentation

RESUMO As diretivas governamentais e institucionais em torno do compartilhamento de dados de pesquisas financiadas com dinheiro público têm impulsionado a rápida expansão de repositórios digitais de dados afim de disponibilizar esses ativos científicos para reutilização, com propósitos nem sempre antecipados, pelos pesquisadores que os produziram/coletaram. De modo contraditório, embora o argumento em torno do compartilhamento de dados seja fortemente sustentado no potencial de reúso e em suas consequentes contribuições para o avanço científico, esse tema permanece acessório às discussões em torno da ciência de dados e da ciência aberta. O presente artigo de revisão narrativa tem por objetivo lançar um olhar mais atento ao reúso de dados e explorar mais diretamente esse conceito, ao passo que propõe uma classificação inicial de cinco abordagens distintas para o reúso de dados de pesquisa (reaproveitamento, agregação, integração, metanálise e reanálise), com base em situações hipotéticas acompanhadas de casos de reúso de dados publicados na literatura científica. Também explora questões determinantes para a condição de reúso, relacionando a reusabilidade à qualidade da documentação que acompanha os dados. Oferece discussão sobre os desafios da documentação de dados, bem como algumas iniciativas e recomendações para que essas dificuldades sejam contornadas. Espera-se que os argumentos apresentados contribuam não somente para o avanço conceitual em torno do reúso e da reusabilidade de dados, mas também reverberem em ações relacionadas à documentação dos dados de modo a incrementar o potencial de reúso desses ativos científicos.Palavras-chave: Reúso de Dados; Reprodutibilidade Científica; Reusabilidade; Ciência Aberta; Dados de Pesquisa. ABSTRACT The availability of scientific assets through data repositories has been greatly increased as a result of government and institutional data sharing policies and mandates for publicly funded research, allowing data to be reused for purposes not always anticipated by primary researchers. Despite the fact that the argument favoring data sharing is strongly grounded in the possibilities of data reuse and its contributions to scientific advancement, this subject remains unobserved in discussions about data science and open science. This paper follows a narrative review method to take a closer look at data reuse in order to better conceptualize this term, while proposing an early classification of five distinct data reuse approaches (repurposing, aggregation, integration, meta-analysis and reanalysis) based on hypothetical cases and literature examples. It also explores the determinants of what constitutes reusable data, and the relationship between data reusability and documentation quality. It presents some challenges associated with data documentation and points out some initiatives and recommendations to overcome such problems. It expects to contribute not only for the conceptual advancement around the reusability and effective reuse of the data, but also to result in initiatives related to data documentation in order to increase the reuse potential of these scientific assets.Keywords:Data Reuse; Scientific Reproducibility; Reusability; Open Science; Research Data.  

scholarly journals Accelerating open science: The collaborative replications and education project (CREP)

2018 ◽  
Author(s):  
Gerit Pfuhl ◽  
Jon Grahe
Keyword(s):  
Behavioral Science ◽  
Research Training ◽  
Meta Analysis ◽  
Academic Research ◽  
Open Data ◽  
Open Science ◽  
Open Access Publishing ◽  
Quality Data ◽  
Education Project ◽  
Science Practices

Watch the VIDEO.Recent years have seen a revolution in publishing, and large support for open access publishing. There has been a slower acceptance and transition to other open science principles such as open data, open materials, and preregistration. To accelerate the transition and make open science the new standard, the collaborative replications and education project (CREP; http://osf.io/wfc6u/)) was launched in 2013, hosted on the Open Science Framework (osf.io). OSF is like a preprint, collecting partial data with each individual contributors project. CREP introduces open science at the start of academic research, facilitating student research training in open science and solidifying behavioral science results. The CREP team attempts to achieve this by inviting contributors to replicate one of several replication studies selected for scientific impact and suitability for undergraduates to complete during one academic term. Contributors follow clear protocols with students interacting with a CREP team that reviews the materials and video of the procedure to ensure quality data collection while students are learning science practices and methods. By combining multiple replications from undergraduates across the globe, the findings can be pooled to conduct meta-analysis and so contribute to generalizable and replicable research findings. CREP is careful to not interpret any single result. CREP has recently joined forces with the psychological science accelerator (PsySciAcc), a globally distributed network of psychological laboratories accelerating the accumulation of reliable and generalizable results in the behavioral sciences. The Department of Psychology at UiT is part of the network and has two ongoing CREP studies, maintaining open science practices early on. In this talk, we will present our experiences of conducting transparent replicable research, and experience with preprints from a supervisor and researcher perspective.

scholarly journals Choice of choice models: Theory of signal detectability outperforms Bradley-Terry-Luce choice model

2018 ◽  
Author(s):  
Diana E Kornbrot ◽  
George J Georgiou ◽  
Mike Page
Keyword(s):  
Decision Making ◽  
Goodness Of Fit ◽  
Choice Model ◽  
Meta Analysis ◽  
Open Science ◽  
Logistic Distribution ◽  
Data Sets ◽  
Logit Function ◽  
Signal Detectability ◽  
Science Collaboration

Identifying the best framework for two-choice decision-making has been a goal of psychology theory for many decades (Bohil, Szalma, & Hancock, 2015; Macmillan & Creelman, 1991). There are two main candidates: the theory of signal detectability (TSD) (Swets, Tanner Jr, & Birdsall, 1961; Thurstone, 1927) based on a normal distribution/probit function, and the choice-model theory (Link, 1975; Luce, 1959) that uses the logistic distribution/logit function. A probit link function, and hence TSD, was shown to have a better Bayesian Goodness of Fit than the logit function for every one of eighteen diverse psychology data sets (Open-Science-Collaboration, 2015a), conclusions having been obtained using Generalized Linear Mixed Models (Lindstrom & Bates, 1990; Nelder & Wedderburn, 1972) . These findings are important, not only for the psychology of perceptual, cognitive and social decision-making, but for any science that use binary proportions to measure effectiveness, as well as the meta-analysis of such studies.

scholarly journals Maternal and neonatal outcomes of preeclamptic and normotensive women who underwent cesarean section under spinal Anesthesia: Systematic review and Meta-Analysis

2020 ◽  
Author(s):  
semagn Abate ◽  
Getachew Mergia ◽  
Bivash Basu
Keyword(s):  
Systematic Review ◽  
Meta Analysis ◽  
Experimental Studies ◽  
Open Science ◽  
Neonatal Outcomes ◽  
Forest Plot ◽  
Language Restriction ◽  
Maternal And Neonatal Outcomes ◽  
Risk Of Aspiration

Abstract Background: preeclampsia is very challenging for anesthetists due to the heterogeneous clinical spectrum of the disease characterized by hypertension, risk of hypotension, high risk of aspiration, and difficult airway. Therefore, the Meta-Analysis is intended to provide evidence on maternal and neonatal outcomes of preeclamptic parturient. Methods: A comprehensive strategy was conducted in PubMed/Medline, Science Direct, and Cochrane from January 2000 to May 2020 without language restriction. The Heterogeneity among the included studies was checked with forest plot and I2 test. Observational and experimental studies reporting maternal and neonatal outcomes among preeclamptic and normotensive women were included. Results: The Meta-Analysis revealed that pooled incidence of hypotension was reduced by thirty-eight percent in preeclamptic as compared to normotensive parturient, RR=0.62(95% confidence interval (CI): 0.52 to 0.75)Conclusion: The Meta-Analysis revealed that the incidence of hypotension was lower in preeclamptic women when compared to normotensive women. The included studies were low to a very low quality of evidence which entails further randomized controlled trials.Registration: This systematic review and meta-analysis was registered in Open science Network on June 6, 2020, and the registration is available at https://osf.io/jcedt/.

scholarly journals Coronavirus Disease (COVID-19) Pandemic: An Overview of Systematic Reviews

2020 ◽  
Author(s):  
Israel Júnior Borges do Nascimento ◽  
Dónal P. O’Mathúna ◽  
Thilo Caspar von Groote ◽  
Hebatullah Mohamed Abdulazeem ◽  
Ishanka Weerasekara ◽  
...  
Keyword(s):  
Systematic Reviews ◽  
Troponin I ◽  
Clinical Symptoms ◽  
Meta Analysis ◽  
Severe Disease ◽  
Ground Glass Opacity ◽  
Open Science ◽  
Frequent Pattern ◽  
Cochrane Library ◽  
The Impact

AbstractBackgroundNavigating the rapidly growing body of scientific literature on the SARS-CoV-2 pandemic is challenging and ongoing critical appraisal of this output is essential. We aimed to collate and summarize all published systematic reviews on the coronavirus disease (COVID-19).MethodsNine databases (Medline, EMBASE, Cochrane Library, CINAHL, Web of Sciences, PDQ-Evidence, WHO’s Global Research, LILACS and Epistemonikos) were searched from December 1, 2019 to March 24, 2020. Systematic reviews analyzing primary studies of COVID-19 were included. Two authors independently undertook screening, selection, extraction (data on clinical symptoms, prevalence, pharmacological and non-pharmacological interventions, diagnostic test assessment, laboratory and radiological findings) and quality assessment (AMSTAR 2). Meta-analysis on prevalence of clinical outcomes was performed.ResultsEighteen systematic reviews were included; one was empty. Using AMSTAR 2, confidence in the results of 13 reviews was rated as “critically low”, one as “low”, one as “moderate” and two as “high”. Symptoms of COVID-19 were (range values of point estimates): fever (82–95%), cough with or without sputum (58–72%), dyspnea (26–59%), myalgia or muscle fatigue (29–51%), sore throat (10–13%), headache (8– 12%) and gastrointestinal complaints (5–9%). Severe symptoms were more common in men. Elevated C-reactive protein (associated with lymphocytopenia) and lactate dehydrogenase, and slightly elevated aspartate and alanine aminotransferase, were commonly described. Thrombocytopenia and elevated levels of procalcitonin and cardiac troponin I were associated with severe disease. Chest imaging described a frequent pattern of uni- or bilateral multilobar ground-glass opacity. Only one review investigated the impact of medication (chloroquine) but found no verifiable clinical data. All-cause mortality ranged from 0.3% to 14%.ConclusionsConfidence in the results of most reviews was “critically low”. Future studies and systematic reviews should adhere to established methodologies. The majority of included systematic reviews were hampered by imprecise search strategy and no previous protocol submission.Protocol registrationThis is an extension of a PROSPERO protocol (CRD42020170623); protocol available on Open Science Framework (https://osf.io/6xtyw).

Sign in / Sign up

Export Citation Format

Share Document