May I have your attention, please? Methodological and Analytical Flexibility in the Addiction Stroop
Theoretical models of addiction predict that an attentional bias towards drug/alcohol related cues plays a key role in development and maintenance of addictive behaviours. However, data from the empirical research testing these predictions is equivocal. This ambiguity may in part be a consequence of substantial variability in the methods used to operationalise attentional bias. The aim of this research was to examine the variability in key design and analysis decisions of the addiction Stroop. Using a pre-registered design, we systematically searched the literature and identified 46 studies describing an alcohol Stroop task and 25 studies describing a smoking Stroop task. We extracted information about the design of the Stroop task, including but not limited to; administration (paper-and-pencil vs. computerised), response (key press vs. voice), number of drug-related stimuli used, number of stimulus repetitions, design (block vs. mixed). For analysis decisions we extracted information on upper- and lower-bound reaction time cut-offs, removal of individual reaction times based on standard error cut-offs, removal of participants based on overall performance, type of outcome used, and removal of errors. Based on variability from previous research there are at least 1,451,520 different possible designs of the computerised alcohol Stroop and 77,760 designs of the computerised smoking Stroop. Many key design decisions were unreported. Similarly, variability in analyses decisions would allow for 9,000 different methods for analysing the alcohol Stroop and 5,376 for the smoking Stroop. This flexibility in the design and analysis of a widely used measure of attentional bias may contribute to equivocal findings, unreliable tasks and inability to replicate research. Future research should work towards a gold standard addiction Stroop with reliable psychometric properties. This will allow us to confidently test predictions and lead to progress within the field.