Machine Learning and Psychological Research: The Unexplored Effect of Measurement
Machine learning (i.e., data mining, artificial intelligence, big data) has seen an increase in application in psychological science. Although some areas of research have benefited tremendously from a new set of statistical tools, most often in the use of biological or genetic variables, the hype has not been substantiated in more traditional areas of research. We offer an explanation for this phenomena: namely that poor measurement prevents machine learning algorithms from accurately modeling nonlinear relationships, if they exist. This is showcased across a set of simulated examples, demonstrating that model selection between a machine learning algorithm and regression depends on the measurement quality, regardless of sample size. We conclude with a set of recommendations and a discussion of ways to better integrate machine learning with statistics as traditionally practiced in psychological science.