From Outcome to Process Focus: Fostering a More Robust Psychological Science Through Registered Reports and Results-Blind Reviewing

2018 ◽  
Vol 13 (4) ◽  
pp. 448-456 ◽  
Author(s):  
James A. Grand ◽  
Steven G. Rogelberg ◽  
George C. Banks ◽  
Ronald S. Landis ◽  
Scott Tonidandel
Keyword(s):  
Research Process ◽  
Psychological Research ◽  
Scientific Practices ◽  
Psychological Science ◽  
Research Outcomes ◽  
Published Research ◽  
Distinguishing Features

A variety of alternative mechanisms, strategies, and “ways of doing” have been proposed for improving the rigor and robustness of published research in the psychological sciences in recent years. In this article, we describe two existing but underused publication models—registered reporting (RR) and results-blind reviewing (RBR)—that we believe would contribute in important ways to improving both the conduct and evaluation of psychological research. We first outline the procedures and distinguishing features of both publication pathways and note their value for promoting positive changes to current scientific practices. We posit that a significant value of RR and RBR is their potential to promote a greater focus on the research process (i.e., how and why research is conducted) relative to research outcomes (i.e., what was observed or concluded from research). We conclude by discussing what we perceive to be five common beliefs about RR and RBR practices and attempt to provide a balanced perspective of the realities likely to be experienced with these systems.

scholarly journals Comment: A Perspective on the Relevance of Psychological Science and Some Implications for the Reform Movement

2018 ◽  
Author(s):  
Jonathon McPhetres
Keyword(s):  
Psychological Research ◽  
Reform Movement ◽  
Social Psychological ◽  
Impact Factor Journal ◽  
Psychological Science ◽  
Publication Record ◽  
Short Commentary ◽  
Replication Crisis ◽  
Social Psychological Research ◽  
Published Research

Concerns about the generalizability, veracity, and relevance of social psychological research often resurface within psychology. While many changes are being implemented to improve the integrity of published research and to clarify the publication record, less attention has been given to the questions of relevance. In this short commentary, I offer my perspective on questions of relevance and present some data from the website Reddit. The data show that people care greatly about psychological research—social psychology studies being among the highest upvoted on the subreddit r/science. However, upvotes on Reddit are unrelated to metrics used by researchers to gauge importance (e.g., impact factor, journal rankings and citations), suggesting a disconnect between what psychologists and lay-audiences may see as relevant. I interpret these data in light of the replication crisis and suggest that the spotlight on our field puts greater importance on the need for reform. Whether we like it or not, people care about, share, and use psychological research in their lives, which means we should ensure that our findings are reported accurately and transparently.

scholarly journals Lay people are unimpressed by the effect sizes typically reported in psychological science

2020 ◽  
Author(s):  
Jonathon McPhetres ◽  
Gordon Pennycook
Keyword(s):  
Psychological Research ◽  
Psychological Effects ◽  
Effect Sizes ◽  
Research Results ◽  
Psychological Science ◽  
Cohen’S D ◽  
Lay People ◽  
Cohen's D ◽  
Published Research ◽  
Statistical Results

It is recommended that researchers report effect sizes along with statistical results to aid in interpreting the magnitude of results. According to recent surveys of published research, psychologists typically find effect sizes ranging from r = .11 to r = .30. While these numbers may be informative for scientists, no research has examined how lay people perceive the range of effect sizes typically reported in psychological research. In two studies, we showed online participants (N = 1,204) graphs depicting a range of effect sizes in different formats. We demonstrate that lay people perceive psychological effects to be small, rather meaningless, and unconvincing. Even the largest effects we examined (corresponding to a Cohen’s d = .90), which are exceedingly uncommon in reality, were considered small-to-moderate in size by lay people. Science communicators and policymakers should consider this obstacle when attempting to communicate the effectiveness of research results.

Psychological Research Literature Review Canvas

2020 ◽  
Author(s):  
Quang Ngoc Nguyen
Keyword(s):  
Literature Review ◽  
Research Process ◽  
Psychological Research ◽  
Research Literature ◽  
Status Quo ◽  
Chaotic Process ◽  
Psychological Construct ◽  
The Status ◽  
Guiding Questions

Without a guideline or structure, conducting a literature review on a psychological construct might become a chaotic process . This canvas was built based on the author's experience in order to help psychological researchers classify, organize, and summarize the information relating to the psychological construct of interest into several essential aspects including definition, classification, measurement, sample, predictors and outcomes, mediators and moderators, interventions, and theories. For each aspect, there are some guiding questions which are expected to help researcher decice which information should be focused while examining scientific documents. The completely filled canvas should depict the status quo of the research on the psychological construct of interest, facilitating the research process.

Preregistration: Comparing Dream to Reality

2019 ◽  
Author(s):  
Aline Claesen ◽  
Sara Lucia Brazuna Tavares Gomes ◽  
francis tuerlinckx ◽  
wolf vanpaemel
Keyword(s):  
Research Finding ◽  
Third Party ◽  
Good Practices ◽  
Psychological Science ◽  
Research Outcomes

Doing research inevitably involves making numerous decisions that can influence research outcomes in such a way that it leads to overconfidence in statistical conclusions. One proposed method to increase the interpretability of a research finding is preregistration, which involves documenting analytic choices on a public, third-party repository prior to any influence by data. To investigate whether, in psychology, preregistration lives up to that potential, we focused on all articles published in Psychological Science with a preregistered badge between February 2015 and November 2017, and assessed the adherence to their corresponding preregistration plans. We observed deviations from the plan in all studies, and, more importantly, in all but one study, at least one of these deviations was not fully disclosed. We discuss examples and possible explanations, and highlight good practices for preregistering research.

The Goldwater Rule: Perspectives From, and Implications for, Psychological Science

2017 ◽  
Author(s):  
scott lilienfeld ◽  
Josh Miller ◽  
Donald Lynam
Keyword(s):  
Mental Health ◽  
Mental Health Professionals ◽  
Clinical Judgment ◽  
Cognitive Biases ◽  
Psychological Research ◽  
Public Attention ◽  
Psychological Science ◽  
Psychiatric Status ◽  
Public Figures ◽  
Duty To Inform

When, if ever, should psychological scientists be permitted to offer professional opinions concerning the mental health of public figures they have never directly examined? This contentious question, which attracted widespread public attention during the 1964 U.S. presidential election involving Barry Goldwater, received renewed scrutiny during and after the 2016 U.S. presidential campaign, when many mental health professionals raised pointed questions concerning the psychiatric status of Donald Trump. Although the Goldwater Rule prohibits psychiatrists from offering diagnostic opinions on individuals they have never examined, no comparable rule exists for psychologists. We contend that, owing largely to the Goldwater Rule’s origins in psychiatry, a substantial body of psychological research on assessment and clinical judgment, including work on the questionable validity of unstructured interviews, the psychology of cognitive biases, and the validity of informant reports and of L (lifetime) data, has been overlooked in discussions of its merits. We conclude that although the Goldwater Rule may have been defensible several decades ago, it is outdated and premised on dubious scientific assumptions. We further contend that there are select cases in which psychological scientists with suitable expertise may harbor a “duty to inform,” allowing them to offer informed opinions concerning public figures’ mental health with appropriate caveats.

Intersectionality as a Framework for Theory and Research in Feminist Psychology

2018 ◽  
Author(s):  
Leah R. Warner ◽  
Stephanie A. Shields
Keyword(s):  
Social Activism ◽  
Data Gathering ◽  
Psychological Research ◽  
Social Identities ◽  
Future Directions ◽  
Science Practices ◽  
Intersectionality Theory ◽  
Psychological Science ◽  
Psychological Theories ◽  
History Of

Intersectionality theory concerns the interdependence of systems of inequality and implications for psychological research. Social identities cannot be studied independently of one another nor separately from the societal processes that maintain inequality. In this chapter we provide a brief overview of the history of intersectionality theory and then address how intersectionality theory challenges the way psychological theories typically conceive of the person, as well as the methods of data gathering and analysis customarily used by many psychologists. We specifically address two concerns often expressed by feminist researchers. First, how to reconcile the use of an intersectionality framework with currently-valued psychological science practices. Second, how intersectionality transforms psychology’s concern with individual experience by shifting the focus to the individual’s position within sociostructural frameworks and their social and political underpinnings. In a concluding section we identify two future directions for intersectionality theory: how psychological research on intersectionality can facilitate social activism, and current developments in intersectionality theory.

History of Sexual Orientation and Mental Health

2020 ◽  
pp. 9-24
Author(s):  
Kristin A. Hancock ◽  
Douglas C. Haldeman
Keyword(s):  
Mental Health ◽  
Psychological Research ◽  
The Political ◽  
Political Movements ◽  
International Policy ◽  
Psychological Science ◽  
Aids Epidemic ◽  
History Of ◽  
And Bisexual ◽  
The Impact

Psychology’s understanding of lesbian, gay, and bisexual (LGB) people has evolved, become more refined, and impacted the lives of LGB people in profound ways. This chapter traces the history of LGB psychology from the nineteenth century to the present and focuses on major events and the intersections of theory, psychological science, politics, and activism in the history of this field. It explores various facets of cultural and psychological history that include the pathologizing of homosexuality, the rise of psychological science and the political movements in the mid-twentieth century, and the major shifts in policy that ensued. The toll of the AIDS epidemic on the field is discussed as is the impact of psychological research on national and international policy and legislation.

scholarly journals METHODOLOGY OF COMPLEX AND INTERDISCIPLINARY RESEARCHES IN MODERN PSYCHOLOGY

2020 ◽  
Vol 17 (1) ◽  
pp. 29-45
Author(s):  
Vladimir A. MAZILOV
Keyword(s):  
Psychological Research ◽  
Scientific Concept ◽  
Psychological Science ◽  
Modern Psychology ◽  
Adequate Understanding ◽  
The Subject ◽  
The Moment

The article is devoted to the evaluation of perspectives of interdisciplinary researches using in psychological science in modern conditions. There are formulated approaches that let to improve the efficiency of interdisciplinary researches. Although this issue has received considerable attention of researchers, the problem is not solved at the moment. The theory of complex psychological researches as a scientific concept, reflecting the specificity of psychological research has not been developed yet. The reason for this is that researchers try to develop the principles of organization and procedure of this kind of researches. It is possible to develop the methodology and theory of complex psychological researches, based on the understanding of the subject of psychology. It is alleged that at present time the most adequate understanding of the subject of psychology is his interpretation like the subjective world of human.

scholarly journals Illusory essences: A bias holding back theorizing in psychological science

2020 ◽  
Author(s):  
Cameron Brick ◽  
Bruce Hood ◽  
Vebjørn Ekroll ◽  
Lee de-Wit
Keyword(s):  
Biological Psychology ◽  
Theory Development ◽  
Psychological Research ◽  
Sufficient Evidence ◽  
Complex Phenomenon ◽  
Psychological Science ◽  
Wide Range ◽  
Category Labels ◽  
Vital Forces ◽  
Holding Back

The reliance in psychology on verbal definitions means that psychological research is unusually moored to how humans think and communicate about categories. Psychological concepts (e.g., intelligence; attention) are easily assumed to represent objective, definable categories with an underlying essence. Like the 'vital forces' previously thought to animate life, these assumed essences can create an illusion of understanding. We describe a pervasive tendency across psychological science to assume that essences explain phenomena by synthesizing a wide range of research lines from cognitive, clinical, and biological psychology and neuroscience. Labeling a complex phenomenon can appear as theoretical progress before sufficient evidence that the described category has a definable essence or known boundary conditions. Category labels can further undermine progress by masking contingent and contextual relationships and obscuring the need to specify mechanisms. Finally, we highlight examples of promising methods that circumvent the lure of essences and we suggest four concrete strategies to identify and avoid essentialist intuitions in theory development.

scholarly journals A Model Hierarchy for Psychological Science

2019 ◽  
Author(s):  
David Kellen
Keyword(s):  
Scientific Practices ◽  
Comprehensive List ◽  
Psychological Science ◽  
Model Hierarchy

Lee et al. (2019) provided a comprehensive list of recommendations for modelers thataims at improving the robustness of their results. Drawing from the literature onphilosophy of science, the present commentary argues for a broader view of modeling that considers the different roles that they play in our scientific practices. Following Suppes (1966), I propose a model hierarchy and discuss the distinct issues that arise at each of its levels. The benefit of a hierarchy of this kind is that it can aid researchers in better understanding the different challenges that they face.

Sign in / Sign up

Export Citation Format

Share Document