The Making and Evaluation of Digital Games Used for the Assessment of Attention: A Systematic Review (Preprint)

BACKGROUND Serious games are now widely used in many contexts, including for psychological research and clinical use. One area of growing interest is that of cognitive assessment, which seeks to measure different cognitive functions like memory, attention and perception. Measuring these functions on both a population and individual level can inform research and indicate health issues. Attention is an important function to assess, as an accurate measure of attention can help diagnose many common disorders, such as attention-deficit/hyperactivity disorder and dementia. Using games to assess attention poses unique problems, as games inherently manipulate attention through elements like sound effects, graphics and rewards, and research on adding game elements to assessments (ie, gamification) has shown mixed results. OBJECTIVE The process for developing cognitive tasks is robust, with high psychometric standards that must be met before these tasks are used for assessment. Games offer more diverse approaches for assessment, but there is no standard for how they should be developed. In order to better understand the field, our objective was to answer the question: How are digital games used for the cognitive assessment of attention made and measured? METHODS After an initial database search that returned 44,172 papers, we conducted a systematic review of 62 papers that use a digital game to measure cognitive functions relating to attention. RESULTS Across the studies in our review, we found three approaches to making assessment games: gamifying cognitive tasks, creating custom games based on theories of cognition, and exploring potential assessment properties of commercial games. In regard to measuring the assessment properties of these games (eg, how accurately they assess attention), we found three approaches: comparison to a traditional cognitive task, comparison to a clinical diagnosis, and comparison to knowledge of cognition; however, the majority of studies in our review did not evaluate the game’s properties (eg, if participants enjoyed the game). CONCLUSIONS Our review provides an overview of how games used for the assessment of attention are developed and evaluated. We further identified three barriers to advancing the field and recommend best practices to address these barriers. Our review can act as a resource to help guide the field towards the more standardized approaches and rigorous evaluation required for the widespread adoption of assessment games.