Fixation classification: how to merge and select fixation candidates

Eye trackers are applied in many research fields (e.g., cognitive science, medicine, marketing research). To give meaning to the eye-tracking data, researchers have a broad choice of classification methods to extract various behaviors (e.g., saccade, blink, fixation) from the gaze signal. There is extensive literature about the different classification algorithms. Surprisingly, not much is known about the effect of fixation and saccade selection rules that are usually (implicitly) applied. We want to answer the following question: What is the impact of the selection-rule parameters (minimal saccade amplitude and minimal fixation duration) on the distribution of fixation durations? To answer this question, we used eye-tracking data with high and low quality and seven different classification algorithms. We conclude that selection rules play an important role in merging and selecting fixation candidates. For eye-tracking data with good-to-moderate precision (RMSD < 0.5∘), the classification algorithm of choice does not matter too much as long as it is sensitive enough and is followed by a rule that selects saccades with amplitudes larger than 1.0∘ and a rule that selects fixations with duration longer than 60 ms. Because of the importance of selection, researchers should always report whether they performed selection and the values of their parameters.

Distilling vector space model scores for the assessment of constructed responses with bifactor Inbuilt Rubric method and latent variables

In this paper, we highlight the importance of distilling the computational assessments of constructed responses to validate the indicators/proxies of constructs/trins using an empirical illustration in automated summary evaluation. We present the validation of the Inbuilt Rubric (IR) method that maps rubrics into vector spaces for concepts’ assessment. Specifically, we improved and validated its scores’ performance using latent variables, a common approach in psychometrics. We also validated a new hierarchical vector space, namely a bifactor IR. 205 Spanish undergraduate students produced 615 summaries of three different texts that were evaluated by human raters and different versions of the IR method using latent semantic analysis (LSA). The computational scores were validated using multiple linear regressions and different latent variable models like CFAs or SEMs. Convergent and discriminant validity was found for the IR scores using human rater scores as validity criteria. While this study was conducted in the Spanish language, the proposed scheme is language-independent and applicable to any language. We highlight four main conclusions: (1) Accurate performance can be observed in topic-detection tasks without hundreds/thousands of pre-scored samples required in supervised models. (2) Convergent/discriminant validity can be improved using measurement models for computational scores as they adjust for measurement errors. (3) Nouns embedded in fragments of instructional text can be an affordable alternative to use the IR method. (4) Hierarchical models, like the bifactor IR, can increase the validity of computational assessments evaluating general and specific knowledge in vector space models. R code is provided to apply the classic and bifactor IR method.

Test–retest reliability and practice effect of the Leuven Perceptual Organisation Screening Test

Visual diagnostic tests must have a high degree of consistency in their measurements (high reliability) to ensure accurate assessment of perceptual abilities. The current study assessed test–retest reliability and practice effects in the Leuven Perceptual Organisation Screening Test (L-POST) in 144 healthy volunteers, with time intervals between 0 and 756 days. We used Pearson's and intraclass correlation analysis, Bland–Altman analysis and multilevel modelling. Results from our analyses converged and supported an adequate reliability of the L-POST. Multilevel modelling demonstrated an absence of practice effect, suggesting that the L-POST is suitable for repeat administration. This study suggests that the L-POST has adequate reliability and is suitable for repeat administration even at short intervals. This study provides the basis for a more systematic evaluation for neuropsychological assessments, which can lead to the development of more reliable assessment batteries.

The use of a tablet-based app for investigating the influence of autistic and ADHD traits on performance in a complex drawing task

This paper describes a smart tablet-based drawing app to digitally record participants’ engagement with the Rey-Osterrieth complex figure (ROCF) task, a well-characterised perceptual memory task that assesses local and global memory. Digitisation of the tasks allows for improved ecological validity, especially in children attracted to tablet devices. Further, digital translation of the tasks affords new measures, including accuracy and computation of the fine motor control kinematics employed to carry out the drawing Here, we report a feasibility study to test the relationship between two neurodevelopmental conditions: autism spectrum disorder (ASD) and attention-deficit/hyperactivity disorder (ADHD). The smart tablet app was employed with 39 adult participants (18-35) characterised for autistic and ADHD traits, and scored using the ROCF perceptual and organisational scoring systems. Trait scores and conditions were predictor variables in linear regression models. Positive correlations were found between the attention-to-detail, attention-switching and communication subscales of the autistic trait questionnaire and organisational scores on the ROCF task. These findings suggest that autistic traits might be linked to differential performance on the ROCF task. Novelty and future applications of the app are discussed.

A web-based interface to calculate phonological neighborhood density for words and nonwords in Modern Standard Arabic

The availability of online databases (e.g., Balota et al., 2007) and calculators (e.g., Storkel & Hoover, 2010) has contributed to an increase in psycholinguistic-related research, to the development of evidence-based treatments in clinical settings, and to scientifically supported training programs in the language classroom. The benefit of online language resources is limited by the fact that the majority of such resources provide information only for the English language (Vitevitch, Chan & Goldstein, 2014). To address the lack of diversity in these resources for languages that differ phonologically and morphologically from English, the present article describes an online database to compute phonological neighborhood density (i.e., the number of words that sound similar to a given word) for words and nonwords in Modern Standard Arabic (MSA). A full description of how the calculator can be used is provided. It can be freely accessed at https://calculator.ku.edu/density/about.

The effect of latent and error non-normality on corrections to the test statistic in structural equation modeling

In structural equation modeling, several corrections to the likelihood-ratio model test statistic have been developed to counter the effects of non-normal data. Previous robustness studies investigating the performance of these corrections typically induced non-normality in the indicator variables. However, non-normality in the indicators can originate from non-normal errors or non-normal latent factors. We conducted a Monte Carlo simulation to analyze the effect of non-normality in factors and errors on six different test statistics based on maximum likelihood estimation by evaluating the effect on empirical rejection rates and derived indices (RMSEA and CFI) for different degrees of non-normality and sample sizes. We considered the uncorrected likelihood-ratio model test statistic and the Satorra–Bentler scaled test statistic with Bartlett correction, as well as the mean and variance adjusted test statistic, a scale-shifted approach, a third moment-adjusted test statistic, and an approach drawing inferences from the relevant asymptotic chi-square mixture distribution. The results indicate that the values of the uncorrected test statistic—compared to values under normality—are associated with a severely inflated type I error rate when latent variables are non-normal, but virtually no differences occur when errors are non-normal. Although no general pattern regarding the source of non-normality for all analyzed measures of fit can be derived, the Satorra–Bentler scaled test statistic with Bartlett correction performed satisfactorily across conditions.

Introducing the Prototypical Stimulus Characteristics Toolbox: Protosc

Many studies use different categories of images to define their conditions. Since any difference between these categories is a valid candidate to explain category-related behavioral differences, knowledge about the objective image differences between categories is crucial for the interpretation of the behaviors. However, natural images vary in many image features and not every feature is equally important in describing the differences between the categories. Here, we provide a methodological approach to find as many of the image features as possible, using machine learning performance as a tool, that have predictive value over the category the images belong to. In other words, we describe a means to find the features of a group of images by which the categories can be objectively and quantitatively defined. Note that we are not aiming to provide a means for the best possible decoding performance; instead, our aim is to uncover prototypical characteristics of the categories. To facilitate the use of this method, we offer an open-source, MATLAB-based toolbox that performs such an analysis and aids the user in visualizing the features of relevance. We first applied the toolbox to a mock data set with a ground truth to show the sensitivity of the approach. Next, we applied the toolbox to a set of natural images as a more practical example.