Map Evaluation under COVID-19 restrictions: A new visual approach based on think aloud interviews

With the beginning of the COVID-19 pandemic, the execution of eye-tracking user studies in indoor environments was no longer possible, and remote and contactless substitutes are needed. With this paper, we want to introduce an alternative method to eye tracking, completely feasible under COVID-19 restrictions. Our main technique are think aloud interviews, where participants constantly verbalize their thoughts as they move through a test. We record the screen and the mouse movements during the interviews, and analyse both the statements and the mouse positions afterwards. With this information, we can encode the approximate map position of the user’s attention for each second of the interview. This allows us to use the same visual methods as for eye-tracking studies, like attention maps or trajectory maps. We implement our method conducting a user study with 21 participants to identify user behaviour while solving high-level interpretation tasks, and with the results of this study, we can show that or new method provides a useful substitute for eye-tracking user studies.

Quiet Eye and Computerized Precision Tasks in First-Person Shooter Perspective Esport Games

The gaze behavior in sports and other applied settings has been studied for more than 20 years. A common finding is related to the “quiet eye” (QE), predicting that the duration of the last fixation before a critical event is associated with higher performance. Unlike previous studies conducted in applied settings with mobile eye trackers, we investigate the QE in a context similar to esport, in which participants click the mouse to hit targets presented on a computer screen under different levels of cognitive load. Simultaneously, eye and mouse movements were tracked using a high-end remote eye tracker at 300 Hz. Consistent with previous studies, we found that longer QE fixations were associated with higher performance. Increasing the cognitive load delayed the onset of the QE fixation, but had no significant influence on the QE duration. We discuss the implications of our results in the context of how the QE is defined, the quality of the eye-tracker data, and the type of analysis applied to QE data.

Intermittent Control as a Model of Mouse Movements

We present Intermittent Control (IC) models as a candidate framework for modelling human input movements in Human–Computer Interaction (HCI). IC differs from continuous control in that users are not assumed to use feedback to adjust their movements continuously, but only when the difference between the observed pointer position and predicted pointer positions becomes large. We use a parameter optimisation approach to identify the parameters of an intermittent controller from experimental data, where users performed one-dimensional mouse movements in a reciprocal pointing task. Compared to previous published work with continuous control models, based on the Kullback–Leibler divergence from the experimental observations, IC is better able to generatively reproduce the distinctive dynamical features and variability of the pointing task across participants and over repeated tasks. IC is compatible with current physiological and psychological theory and provides insight into the source of variability in HCI tasks.

Mouse movements reflect personality traits and task attentiveness in online experiments

In this rapidly digitizing world, it is becoming ever more important to understand people’s online behaviors in both scientific and consumer research settings. A cost-effective way to gain a deeper understanding of these behaviors is to examine mouse movement patterns. This research explores the feasibility of inferring personality traits from these mouse movement features (i.e., pauses, fixations, cursor speed, clicks) on a simple image choice task. We compare the results of standard univariate (OLS regression, bivariate correlations) and three forms of multivariate partial least squares (PLS) analyses. This work also examines whether mouse movements can predict task attentiveness, and how these might be related to personality traits. Results of the PLS analyses showed significant associations between a linear combination of personality traits (high Conscientiousness, Agreeableness, and Openness, and low Neuroticism) and several mouse movements associated with slower, more deliberate responding (less unnecessary clicks, more fixations). Additionally, several click-related mouse features were associated with attentiveness to the task. Importantly, as the image choice task itself is not intended to assess personality in any way, our results validate the feasibility of using mouse movements to infer internal traits across experimental contexts, particularly when examined using multivariate analyses and a multiverse approach.

Predicting Question Difficulty in Web Surveys: A Machine Learning Approach Based on Mouse Movement Features

Survey research aims to collect robust and reliable data from respondents. However, despite researchers’ efforts in designing questionnaires, survey instruments may be imperfect, and question structure not as clear as could be, thus creating a burden for respondents. If it were possible to detect such problems, this knowledge could be used to predict problems in a questionnaire during pretesting, inform real-time interventions through responsive questionnaire design, or to indicate and correct measurement error after the fact. Previous research has used paradata, specifically response times, to detect difficulties and help improve user experience and data quality. Today, richer data sources are available, for example, movements respondents make with their mouse, as an additional detailed indicator for the respondent–survey interaction. This article uses machine learning techniques to explore the predictive value of mouse-tracking data regarding a question’s difficulty. We use data from a survey on respondents’ employment history and demographic information, in which we experimentally manipulate the difficulty of several questions. Using measures derived from mouse movements, we predict whether respondents have answered the easy or difficult version of a question, using and comparing several state-of-the-art supervised learning methods. We have also developed a personalization method that adjusts for respondents’ baseline mouse behavior and evaluate its performance. For all three manipulated survey questions, we find that including the full set of mouse movement measures and accounting for individual differences in these measures improve prediction performance over response-time-only models.

Bodily Information and Top-Down Affective Priming Jointly Affect the Processing of Fearful Faces

According to embodied theories, the processing of emotions such as happiness or fear is grounded in emotion-specific perceptual, bodily, and physiological processes. Under these views, perceiving an emotional stimulus (e.g., a fearful face) re-enacts interoceptive and bodily states congruent with that emotion (e.g., increases heart rate); and in turn, interoceptive and bodily changes (e.g., increases of heart rate) influence the processing of congruent emotional content. A previous study by Pezzulo et al. (2018) provided evidence for this embodied congruence, reporting that experimentally increasing heart rate with physical exercise facilitated the processing of facial expressions congruent with that interoception (fear), but not those conveying incongruent states (disgust or neutrality). Here, we investigated whether the above (bottom-up) interoceptive manipulation and the (top-down) priming of affective content may jointly influence the processing of happy and fearful faces. The fact that happiness and fear are both associated with high heart rate but have different (positive and negative) valence permits testing the hypothesis that their processing might be facilitated by the same interoceptive manipulation (the increase of heart rate) but two opposite (positive and negative) affective primes. To test this hypothesis, we asked participants to perform a gender-categorization task of happy, fearful, and neutral faces, which were preceded by positive, negative, and neutral primes. Participants performed the same task in two sessions (after rest, with normal heart rate, or exercise, with faster heart rate) and we recorded their response times and mouse movements during the choices. We replicated the finding that when participants were in the exercise condition, they processed fearful faces faster than when they were in the rest condition. However, we did not find the same reduction in response time for happy (or neutral) faces. Furthermore, we found that when participants were in the exercise condition, they processed fearful faces faster in the presence of negative compared to positive or neutral primes; but we found no equivalent facilitation of positive (or neutral) primes during the processing of happy (or neutral) faces. While the asymmetries between the processing of fearful and happy faces require further investigation, our findings promisingly indicate that the processing of fearful faces is jointly influenced by both bottom-up interoceptive states and top-down affective primes that are congruent with the emotion.

An Innovative Approach for the Evaluation of the Web Page Impact Combining User Experience and Neural Network Score

The proposed paper introduces an innovative methodology useful to assign intelligent scores to web pages. The approach is based on the simultaneous use of User eXperience (UX), Artificial Neural Network (ANN), and Long Short-Term Memory (LSTM) algorithms, providing the web page scoring and taking into account outlier conditions to construct the training dataset. Specifically, the UX tool analyses different parameters addressing the score, such as navigation time, number of clicks, and mouse movements for page, finding possible outliers, the ANN are able to predict outliers, and the LSTM processes the web pages tags together with UX and user scores. The final web page score is assigned by the LSTM model corrected by the UX output and improved by the navigation user score. This final score is useful for the designer by suggesting the tags typologies structuring a new web page layout of a specific topic. By using the proposed methodology, the web designer is addressed to allocate contents in the web page layout. The work has been developed within a framework of an industry project oriented on the formulation of an innovative AI interface for web designers.

Detection of advanced web bots by combining web logs with mouse behavioural biometrics

Web bots vary in sophistication based on their purpose, ranging from simple automated scripts to advanced web bots that have a browser fingerprint, support the main browser functionalities, and exhibit a humanlike behaviour. Advanced web bots are especially appealing to malicious web bot creators, due to their browser-like fingerprint and humanlike behaviour which reduce their detectability. This work proposes a web bot detection framework that comprises two detection modules: (i) a detection module that utilises web logs, and (ii) a detection module that leverages mouse movements. The framework combines the results of each module in a novel way to capture the different temporal characteristics of the web logs and the mouse movements, as well as the spatial characteristics of the mouse movements. We assess its effectiveness on web bots of two levels of evasiveness, (a) moderate web bots that have a browser fingerprint and (b) advanced web bots that have a browser fingerprint and also exhibit a humanlike behaviour. We show that combining web logs with visitors? mouse movements is more effective and robust towards detecting advanced web bots that try to evade detection, as opposed to using only one of those approaches.