Torturing Data

2019 ◽  
pp. 111-126
Author(s):  
Gary Smith ◽  
Jay Cordes
Keyword(s):  
Scientific Research ◽  
Popular Media ◽  
Replication Crisis ◽  
File Drawer ◽  
File Drawer Effect

Researchers seeking fame and funding may be tempted to go on fishing expeditions (p-hacking) or to torture the data to find novel, provocative results that will be picked up by the popular media. Provocative findings are provocative because they are novel and unexpected, and they are often novel and unexpected because they are simply not true. The publication effect (or the file drawer effect) keeps the failures hidden and have created a replication crisis. Research that gets reported in the popular media is often wrong—which fools people and undermines the credibility of scientific research.

Dolphins in Popular Literature and Media

2006 ◽  
Vol 14 (4) ◽  
pp. 321-349 ◽  
Author(s):  
Diana Reiss ◽  
Jessica Sickler ◽  
Sarah Gruber ◽  
Paul Boyle ◽  
Elizabeth Elliott ◽  
...  
Keyword(s):  
Scientific Research ◽  
Popular Literature ◽  
Popular Media ◽  
Public Acceptance ◽  
Research Results ◽  
Higher Power

AbstractThis review of how dolphins are portrayed in popular media (including literature, film, television, and music) reveals four themes that may influence public acceptance of current scientific research into dolphin cognition. These themes are: (a) dolphin as peer to humans, of equal intelligence or at least capable of communicating with or helping humans; (b) the dolphin as the representation of a romantic notion of ideal freedom in nature, embodying principles of peace, harmony or love; (c) the dolphin as a naïve, innocent being that is subordinate and in need of human protection; and (d) the dolphin as superior to humans, potentially affiliating with a higher power or intelligence. This review revealed that the use of dolphins in humor reinforced or lampooned the four identified themes, indicating a common acceptance of these themes. The paper concludes with a discussion of the importance of considering popular narratives in the presentation of scientific research results.

Some Late Nineteenth Century Perspectives on Sex and Emotional Expression

2000 ◽  
Vol 86 (2) ◽  
pp. 575-585 ◽  
Author(s):  
Ellen B. Braaten ◽  
Wayne Viney
Keyword(s):  
Nineteenth Century ◽  
Sex Differences ◽  
Emotional Expression ◽  
Public Interest ◽  
Scientific Research ◽  
Popular Literature ◽  
Popular Media ◽  
Late Nineteenth Century ◽  
The Public ◽  
Late Nineteenth

A review of nineteenth century popular literature indicates a deep and sustained public interest in sex differences in emotional expression. The conclusions advanced by popular writers included a catalog of perceived sex differences, reinforced by an essentialist philosophy that provided justification for the separation of sexual spheres and restrictions on political, educational, and vocational opportunities for women. Current scientific research on sex differences appears in popular media and is often presented in the context of an essentialist philosophy comparable with that which was dominant in the nineteenth century. Unfortunately, the subtleties and complexities of sex differences are not always communicated to the public and there is thus a potential for misinterpretation or even misuse.

The file drawer effect – a long-lasting issue in science

2021 ◽  
Author(s):  
Bastien Lemaire ◽  
◽  
Raphaël Lenoble ◽  
Mirko Zanon ◽  
Thibaud Jacquel ◽  
...  
Keyword(s):  
Current Literature ◽  
Experimental Designs ◽  
Scientific Journals ◽  
Scientific Outputs ◽  
Science Problem ◽  
Scientific Facts ◽  
File Drawer ◽  
File Drawer Effect ◽  
Unpublished Works

Most of the scientific outputs produced by researchers are inaccessible since they are not published in scientific journals: they remain in the researchers' drawers, forming what we call the Dark Science. This is a long-standing issue in research, creating a misleading view of the scientific facts. Contrary to the current literature overfed with positive findings, the Dark Science is nurtured with null findings, replications, flawed experimental designs and other research outputs. Publishers, researchers, institutions and funders all play an important role in the accumulation of those unpublished works, but it is only once we understand the reasons and the benefits of publishing all the scientific findings that we can collectively act to solve the Dark Science problem. In this article, we discuss the causes and consequences of the Dark Science expansion, arguing that science and scientists would benefit from getting all their findings to the light of publication.

Improving the Rigor and Replicability of Applied Psychophysiology Research: Sample Size, Standardization, Transparency, and Preregistration

Biofeedback ◽  
2020 ◽  
Vol 48 (1) ◽  
pp. 2-6
Author(s):  
Michael J. Larson
Keyword(s):  
Sample Size ◽  
Scientific Research ◽  
Applied Psychophysiology ◽  
Sample Sizes ◽  
Research Practices ◽  
Scientific Rigor ◽  
Rigorous Research ◽  
Replication Crisis ◽  
Publication Guidelines ◽  
Psychophysiological Data

Scientific research across a number of areas, including applied psychophysiology, biofeedback, and neurofeedback, is facing considerable scrutiny for poor replication rates, high numbers of false-positive findings, and insufficient scientific rigor. There are many factors underlying this replication crisis in scientific research; yet incentives for more rigorous research practices at the institutional and editorial levels lag behind the need for improvement. The author provides examples of replication and rigor difficulties in scientific research with an eye toward psychophysiological research, including researcher flexibility in data analysis, “p-hacking,” insufficient sample sizes, and lack of availability and implementation of rigorous methodological and publication guidelines. Subsequently, the author highlights examples and opportunities for improvement, including decreasing researcher flexibility, reporting sample size information, increasing sample sizes through collaboration, improving reporting standards/following established guidelines for reporting psychophysiological data, and increasing adoption of preregistration and registered reports. The author concludes that the Association for Applied Psychophysiology and Biofeedback (AAPB) can improve clinical practice and perception of public and scientific credibility by implementing rigorous and transparent research practices with a focus on replicability and clear methodological and reporting techniques and standards.

Psychedelic medicalization, public discourse, and the morality of ego dissolution

2021 ◽  
pp. 136787792110194
Author(s):  
Alex K Gearin ◽  
Neşe Devenot
Keyword(s):  
Public Discourse ◽  
Cultural Practices ◽  
Scientific Research ◽  
The United States ◽  
Popular Media ◽  
Research Groups ◽  
The United Kingdom ◽  
Media Institutions ◽  
Political Orientations ◽  
History Of

Emerging from a diverse and long history of shamanic and religious cultural practices, psychedelic substances are increasingly being foregrounded as medicines by an assemblage of scientific research groups, media institutions, government drug authorities, and patient and consumer populations. Considering scientific studies and recent popular media associated with the medicalization of psychedelic substances, this article responds to scholarly debates over the imbrication of scientific knowledge and moral discourse. It contends that, while scientific research into psychedelic medicine presents itself as amoral and objective, it often reverts to moral and political claims in public discourse. We illustrate how psychedelic medicine discourse in recent popular media in the United States and the United Kingdom is naturalizing specific moral and political orientations as pharmacological and healthy. The article traces how psychedelic substances have become ego-dissolving medicines invested with neoliberal and anti-authoritarian agency.

scholarly journals Questionable Research Practices in Single-Case Experimental Designs: Examples and Possible Solutions

2021 ◽  
Author(s):  
Matt Tincani ◽  
Jason C Travers
Keyword(s):  
Single Case ◽  
The Body ◽  
Research Practices ◽  
Questionable Research Practices ◽  
Participant Selection ◽  
Independent Variable ◽  
Meta Analyses ◽  
Research Choices ◽  
File Drawer ◽  
File Drawer Effect

Questionable research practices (QRPs) are a variety of research choices that introduce bias into the body of scientific literature. Researchers have documented widespread presence of QRPs across disciplines and promoted practices aimed at preventing them. More recently, Single-Case Experimental Design (SCED) researchers have explored how QRPs could manifest in SCED research. In the chapter, we describe QRPs in participant selection, independent variable selection, procedural fidelity documentation, graphical depictions of behavior, and effect size measures and statistics. We also discuss QRPs in relation to the file drawer effect, publication bias, and meta-analyses of SCED research. We provide recommendations for researchers and the research community to promote practices for preventing QRPs in SCED.

THE FILE DRAWER EFFECT

PEDIATRICS ◽  
1996 ◽  
Vol 97 (1) ◽  
pp. 70-70
Keyword(s):  
File Drawer ◽  
Positive Results ◽  
File Drawer Effect

Statistics can tell us when published numbers truly point to the probability of a negative result, even though we, in our hopes, have mistakenly conferred a positive interpretation. But statistics cannot rescue us . . . when we publish positive results and consign our probable negativities to nonscrutiny in our file drawers.

scholarly journals A case study exploring associations between popular media attention of scientific research and scientific citations

PLoS ONE ◽  
2020 ◽  
Vol 15 (7) ◽  
pp. e0234912 ◽  
Author(s):  
P. Sage Anderson ◽  
Aubrey R. Odom ◽  
Hunter M. Gray ◽  
Jordan B. Jones ◽  
William F. Christensen ◽  
...  
Keyword(s):  
Scientific Research ◽  
Popular Media ◽  
Media Attention

scholarly journals Assessing robustness against potential publication bias in coordinate based fMRI meta-analyses using the Fail-Safe N

2017 ◽  
Author(s):  
Freya Acar ◽  
Ruth Seurinck ◽  
Simon B. Eickhoff ◽  
Beatrijs Moerkerke
Keyword(s):  
Publication Bias ◽  
Meta Analysis ◽  
Popular Approach ◽  
Analytical Results ◽  
Research Findings ◽  
Meta Analyses ◽  
Fail Safe ◽  
File Drawer ◽  
Cluster Level ◽  
File Drawer Effect

AbstractThe importance of integrating research findings is incontrovertible and coordinate based meta-analyses have become a popular approach to combine results of fMRI studies when only peaks of activation are reported. Similar to classical meta-analyses, coordinate based meta-analyses may be subject to different forms of publication bias which impacts results and possibly invalidates findings. We develop a tool that assesses the robustness to potential publication bias on cluster level. We investigate the possible influence of the file-drawer effect, where studies that do not report certain results fail to get published, by determining the number of noise studies that can be added to an existing fMRI meta-analysis before the results are no longer statistically significant. In this paper we illustrate this tool through an example and test the effect of several parameters through extensive simulations. We provide an algorithm for which code is freely available to generate noise studies and enables users to determine the robustness of meta-analytical results.

Sign in / Sign up

Export Citation Format

Share Document