Can Computers Outperform Humans in Detecting User Zone-Outs? Implications for Intelligent Interfaces

The ability to identify whether a user is “zoning out” (mind wandering) from video has many HCI (e.g., distance learning, high-stakes vigilance tasks). However, it remains unknown how well humans can perform this task, how they compare to automatic computerized approaches, and how a fusion of the two might improve accuracy. We analyzed videos of users’ faces and upper bodies recorded 10s prior to self-reported mind wandering (i.e., ground truth) while they engaged in a computerized reading task. We found that a state-of-the-art machine learning model had comparable accuracy to aggregated judgments of nine untrained human observers (area under receiver operating characteristic curve [AUC] = .598 versus .589). A fusion of the two (AUC = .644) outperformed each, presumably because each focused on complementary cues. Furthermore, adding more humans beyond 3–4 observers yielded diminishing returns. We discuss implications of human–computer fusion as a means to improve accuracy in complex tasks.

Targeted Memory Reactivation during Mind-Wandering in Younger and Older Adults

Mind-wandering (MW) is a universal cognitive process that is estimated to comprise ~30% of our everyday thoughts. Despite its prevalence, the functional utility of MW remains a scientific blind spot. The present study sought to investigate whether MW serves a functional role in cognition. Specifically, we investigated whether MW contributes to memory consolidation processes, and if age differences in the ability to reactivate episodic memories during MW may contribute to age-related declines in episodic memory. Younger and older adults encoded paired associates, received targeted reactivation cues during an interval filled with a task which promotes MW, and were tested on their memory for the cued and uncued stimuli from the initial encoding task. Thought probes were presented during the retention (MW) interval to assess participants’ thought contents. Across three experiments, we compared the effect of different cue modalities (i.e., auditory, visual) on cued recall performance, and examined both correct retrieval response times as well as accuracy. Across experiments, there was evidence that stimuli that were cued during the MW task were correctly retrieved more quickly than uncued stimuli and that this effect was more robust for younger adults than older adults. Additionally, the more MW a participant reported during the retention interval, the stronger the cueing effect they produced during retrieval. The results from these experiments are interpreted within a retrieval facilitation framework wherein cues serve to reactivate the earlier traces during MW, and this reactivation benefits retrieval speed for cued items as compared to uncued items.