Supplemental Material for Toward the Data-Driven Dissemination of Findings From Psychological Science

The Scientific Work We Love: A Duplex Theory of Scientific Impact and Its Application to the Top-Cited Articles in the First 30 Years of APS Journals

Perspectives on Psychological Science ◽

10.1177/1745691617752690 ◽

2018 ◽

Vol 13 (2) ◽

pp. 260-267 ◽

Cited By ~ 3

Author(s):

Robert J. Sternberg

Keyword(s):

Scientific Work ◽

High Impact ◽

Data Driven ◽

Scientific Impact ◽

Psychological Science ◽

Review Articles ◽

Triangular Elements ◽

Duplex Theory

This article proposes a duplex theory for understanding the scientific impact of contributions to psychological science. I argue that articles that we “love” can be understood in terms of (a) triangular elements of intimacy, passion, and commitment and (b) types of stories that characterize high-impact articles. Certain kinds of stories (e.g., review articles) are more likely to have lasting impact, on average, than other kinds of stories (e.g., data-driven empirical articles).

Download Full-text

Toward the data-driven dissemination of findings from psychological science.

American Psychologist ◽

10.1037/amp0000721 ◽

2020 ◽

Vol 75 (8) ◽

pp. 1052-1066

Author(s):

Jonathan Purtle ◽

Jacob S. Marzalik ◽

Raquel W. Halfond ◽

Lynn F. Bufka ◽

Bethany A. Teachman ◽

...

Keyword(s):

Data Driven ◽

Psychological Science

Download Full-text

Uncovering mental structure through data-driven ontology discovery

10.31234/osf.io/fvqej ◽

2018 ◽

Cited By ~ 3

Author(s):

Ian W Eisenberg ◽

Patrick Bissett ◽

Ayse Zeynep Enkavi ◽

Jamie Li ◽

David MacKinnon ◽

...

Keyword(s):

Cognitive Processes ◽

Real World ◽

Predictive Power ◽

Empirical Relationship ◽

Self Regulation ◽

Self Report ◽

Data Driven ◽

Psychological Science ◽

Individual Traits ◽

Cumulative Knowledge

Psychological sciences have identified a wealth of cognitive processes and behavioral phenomena, yet struggle to produce cumulative knowledge. Progress is hamstrung by siloed scientific traditions and a focus on explanation over prediction, two issues we address by examining individual differences across an unprecedented range of behavioral tasks, self-report surveys, and real-world outcomes. We derive a cognitive ontology and evaluate the predictive power of many psychological measurements related to self-regulation. Though both tasks and surveys putatively measure self-regulation, they show little empirical relationship. Within tasks and surveys, however, the ontology reveals opportunities for theoretic synthesis and identifies stable individual traits. Additionally, surveys predict self-reported real-world outcomes while tasks largely do not. We conclude that data-driven ontologies lay the groundwork for a cumulative psychological science.

Download Full-text

A pragmatist philosophy of psychological science and its implications for replication

Behavioral and Brain Sciences ◽

10.1017/s0140525x18000626 ◽

2018 ◽

Vol 41 ◽

Cited By ~ 3

Author(s):

Ana Gantman ◽

Robin Gomila ◽

Joel E. Martinez ◽

J. Nathan Matias ◽

Elizabeth Levy Paluck ◽

...

Keyword(s):

William James ◽

Mental Processes ◽

Psychological Science ◽

Behavioral Measurement ◽

The World ◽

Pragmatist Philosophy

AbstractA pragmatist philosophy of psychological science offers to the direct replication debate concrete recommendations and novel benefits that are not discussed in Zwaan et al. This philosophy guides our work as field experimentalists interested in behavioral measurement. Furthermore, all psychologists can relate to its ultimate aim set out by William James: to study mental processes that provide explanations for why people behave as they do in the world.

Download Full-text

Replications can cause distorted belief in scientific progress

Behavioral and Brain Sciences ◽

10.1017/s0140525x18000584 ◽

2018 ◽

Vol 41 ◽

Cited By ~ 2

Author(s):

Michał Białek

Keyword(s):

Real World ◽

Scientific Progress ◽

The Public ◽

Psychological Science

AbstractIf we want psychological science to have a meaningful real-world impact, it has to be trusted by the public. Scientific progress is noisy; accordingly, replications sometimes fail even for true findings. We need to communicate the acceptability of uncertainty to the public and our peers, to prevent psychology from being perceived as having nothing to say about reality.

Download Full-text

Values in Psychological Science

10.1017/9781316471302 ◽

2018 ◽

Cited By ~ 1

Author(s):

Lisa Osbeck

Keyword(s):

Psychological Science

Download Full-text

Translocation Relative to Range

Methodology ◽

10.1027/1614-2241.4.3.132 ◽

2008 ◽

Vol 4 (3) ◽

pp. 132-138 ◽

Cited By ~ 3

Author(s):

Michael Höfler

Keyword(s):

Risk Difference ◽

Binary Outcomes ◽

Number Needed To Treat ◽

Likert Scales ◽

Psychological Science ◽

The Absolute ◽

The Difference ◽

Variance Parameters ◽

Standardized Index ◽

Maximum Possible Magnitude

A standardized index for effect intensity, the translocation relative to range (TRR), is discussed. TRR is defined as the difference between the expectations of an outcome under two conditions (the absolute increment) divided by the maximum possible amount for that difference. TRR measures the shift caused by a factor relative to the maximum possible magnitude of that shift. For binary outcomes, TRR simply equals the risk difference, also known as the inverse number needed to treat. TRR ranges from –1 to 1 but is – unlike a correlation coefficient – a measure for effect intensity, because it does not rely on variance parameters in a certain population as do effect size measures (e.g., correlations, Cohens d). However, the use of TRR is restricted on outcomes with fixed and meaningful endpoints given, for instance, for meaningful psychological questionnaires or Likert scales. The use of TRR vs. Cohens d is illustrated with three examples from Psychological Science 2006 (issues 5 through 8). It is argued that, whenever TRR applies, it should complement Cohens d to avoid the problems related to the latter. In any case, the absolute increment should complement d.

Download Full-text