Tracking stress via the computer mouse? Promises and challenges of a potential behavioral stress marker

Computer mouse tracking offers a simple and cost-efficient way to gather continuous behavioral data and has mostly been utilized in psychological science to study cognitive processes. The present study extends the potential applicability of computer mouse tracking and investigates the feasibility of using computer mouse tracking for stress measurement. Drawing on first empirical results and theoretical considerations, we hypothesized that stress affects sensorimotor processes involved in mouse usage. To explore the relationship between stress and computer mouse usage, we conducted a between-participant field experiment in which N = 994 participants worked on four mouse tasks in a high-stress or low-stress condition. In the manipulation check, participants reported different stress levels between the two conditions. However, frequentist and machine learning data analysis approaches did not reveal a clear and systematic relationship between mouse usage and stress. These findings challenge the feasibility of using straightforward computer mouse tracking for generalized stress measurement.

Cognitive Processing of Miscommunication in Interactive Listening: An Evaluation of Listener Indecision and Cognitive Effort

Purpose The purpose of the current study was to evaluate the social and cognitive underpinnings of miscommunication during an interactive listening task. Method An eye and computer mouse–tracking visual-world paradigm was used to investigate how a listener's cognitive effort (local and global) and decision-making processes were affected by a speaker's use of ambiguity that led to a miscommunication. Results Experiments 1 and 2 found that an environmental cue that made a miscommunication more or less salient impacted listener language processing effort (eye-tracking). Experiment 2 also indicated that listeners may develop different processing heuristics dependent upon the speaker's use of ambiguity that led to a miscommunication, exerting a significant impact on cognition and decision making. We also found that perspective-taking effort and decision-making complexity metrics (computer mouse tracking) predict language processing effort, indicating that instances of miscommunication produced cognitive consequences of indecision, thinking, and cognitive pull. Conclusion Together, these results indicate that listeners behave both reciprocally and adaptively when miscommunications occur, but the way they respond is largely dependent upon the type of ambiguity and how often it is produced by the speaker.

Computer mouse tracking reveals motor signatures in a cognitive task of spatial language grounding

In a novel computer mouse tracking paradigm, participants read a spatial phrase such as “The blue item to the left of the red one” and then see a scene composed of 12 visual items. The task is to move the mouse cursor to the target item (here, blue), which requires perceptually grounding the spatial phrase. This entails visually identifying the reference item (here, red) and other relevant items through attentional selection. Response trajectories are attracted toward distractors that share the target color but match the spatial relation less well. Trajectories are also attracted toward items that share the reference color. A competing pair of items that match the specified colors but are in the inverse spatial relation increases attraction over-additively compared to individual items. Trajectories are also influenced by the spatial term itself. While the distractor effect resembles deviation toward potential targets in previous studies, the reference effect suggests that the relevance of the reference item for the relational task, not its role as a potential target, was critical. This account is supported by the strengthened effect of a competing pair. We conclude, therefore, that the attraction effects in the mouse trajectories reflect the neural processes that operate on sensorimotor representations to solve the relational task. The paradigm thus provides an experimental window through motor behavior into higher cognitive function and the evolution of activation in modal substrates, a longstanding topic in the area of embodied cognition.

Tracking the Multitasking Mind

Task switching paradigms examining executive control in multitask environments typically measure reaction time and accuracy from key press responses. The discrete nature of such responses may limit the ability to capture the dynamics of cognitive control processes that unfold over time in complex environments. The current study used computer mouse tracking methodology to measure the processes that occur during task switching. In two experiments mouse trajectory data were collected as participants used onscreen category labels to respond to two simple tasks. The application of mouse tracking methodology to cued task switching provided both a replication of previous findings using key press responses and a more sensitive measure of the cognitive processes and activated representations underlying those effects. Computer mouse tracking offers a novel methodology for uncovering the mental representations and processes unfolding during multitasking.

Computer-Mouse Tracking Reveals TMS Disruptions of Prefrontal Function During Semantic Retrieval

Converging evidence from neuroimaging and neuropsychological studies is essential for understanding human frontal cortical function. We introduce a new method for studying the effects of transient disruptions of frontal activity during transcranial magnetic stimulation (TMS). Using a novel combination of TMS and computer-mouse tracking, through two experiments we tested process models of semantic competition in left ventrolateral prefrontal cortex (VLPFC). On TMS stimulation of left mid-VLPFC just after presentation of an ambiguous stimulus, participants' mouse-movement trajectories deviated more toward the incorrect target for weak associate trials than for any other trial type. This effect was extinguished when participants were simultaneously shown both target and cue stimuli. Results suggest that left mid-VLPFC is necessary to resolve semantic competition when a response is underdetermined by the stimulus and the interpretive context of the stimulus is ambiguous. Computer-mouse movements reveal the dynamics of competitive interactions as they resolve, making this technique ideally suited for studying cognitive control processes and a more sensitive index of TMS disruption than reaction time and accuracy alone.