Big Data, Exploratory Data Analyses and Questionable Research Practices: Suggestion for a Foundational Principle

2021 ◽  
Author(s):  
J. A. Bissonette

2019 ◽  
Author(s):  
Dustin Fife ◽  
Joseph Lee Rodgers

In light of the “replication crisis,” some (e.g., Nelson, Simmons, & Simonsohn, 2018) advocate for greater policing and transparency in research methods. Others (Baumeister, 2016; Finkel, Eastwick, & Reis, 2017; Goldin-meadow, 2016; Levenson, 2017) argue against rigid requirements that may inadvertently restrict discovery. We embrace both positions and argue that proper understanding and implementation of the well-established paradigm of Exploratory Data Analysis (EDA; Tukey, 1977) is necessary to push beyond the replication crisis. Unfortunately, many don’t realize EDA exists (Goldin-Meadow, 2016), fail to understand the philosophy and proper tools for exploration (Baumeister, 2016), or reject EDA as unscientific (Lindsay, 2015). EDA’s mistreatment is unfortunate, and is usually based on misunderstanding the nature and goal of EDA. We develop an expanded typology that situates EDA, CDA, and rough CDA in the same framework with fishing, p-hacking, and HARKing, and argue that most, if not all, questionable research practices (QRPs) would be resolved by understanding and implementing the EDA/CDA gradient. We argue most psychological research is “rough CDA,” which has often and inadvertently used the wrong tools. We conclude with guidelines about how these typologies can be integrated into a cumulative research program that is necessary to move beyond the replication crisis.







2020 ◽  
Vol 51 (1) ◽  
pp. 151-174
Author(s):  
Chung Joo Chung ◽  
Yunna Rhee ◽  
Heewon Cha


2018 ◽  
Author(s):  
Dick Bierman ◽  
Jacob Jolij

We have tested the feasibility of a method to prevent the occurrence of so-called Questionable Research Practices (QRP). A part from embedded pre-registration the major aspect of the system is real-time uploading of data on a secure server. We outline the method, discuss the drop-out treatment and compare it to the Born-open data method, and report on our preliminary experiences. We also discuss the extension of the data-integrity system from secure server to use of blockchain technology.



2019 ◽  
Author(s):  
Rens van de Schoot ◽  
Elian Griffioen ◽  
Sonja Désirée Winter

The trial-and-roulette method is a popular method to extract experts’ beliefs about a statistical parameter. However, most studies examining the validity of this method only use ‘perfect’ elicitation results. In practice, it is sometimes hard to obtain such neat elicitation results. In our project about predicting fraud and questionable research practices among PhD candidates, we ran into issues with imperfect elicitation results. The goal of the current chapter is to provide an over-view of the solutions we used for dealing with these imperfect results, so that others can benefit from our experience. We present information about the nature of our project, the reasons for the imperfect results, and how we resolved these sup-ported by annotated R-syntax.



2017 ◽  

As machine-readable data comes to play an increasingly important role in everyday life, researchers find themselves with rich resources for studying society. The novel methods and tools needed to work with such data require not only new knowledge and skills, but also a new way of thinking about best research practices. This book critically reflects on the role and usefulness of big data, challenging overly optimistic expectations about what such information can reveal, introducing practices and methods for its analysis and visualisation, and raising important political and ethical questions regarding its collection, handling, and presentation.



2021 ◽  
Author(s):  
Siyang Lu ◽  
Yihong Chen ◽  
Xiaolin Zhu ◽  
Ziyi Wang ◽  
Yangjun Ou ◽  
...  


Sign in / Sign up

Export Citation Format

Share Document