Why subtitle speed matters: Evidence from word skipping and rereading

High subtitle speed undoubtedly impacts the viewer experience. However, little is known about how fast subtitles might impact the reading of individual words. This article presents new findings on the effect of subtitle speed on viewers’ reading behavior using word-based eye-tracking measures with specific attention to word skipping and rereading. In multimodal reading situations such as reading subtitles in video, rereading allows people to correct for oculomotor error or comprehension failure during linguistic processing or integrate words with elements of the image to build a situation model of the video. However, the opportunity to reread words, to read the majority of the words in the subtitle and to read subtitles to completion, is likely to be compromised when subtitles are too fast. Participants watched videos with subtitles at 12, 20, and 28 characters per second (cps) while their eye movements were recorded. It was found that comprehension declined as speed increased. Eye movement records also showed that faster subtitles resulted in more incomplete reading of subtitles. Furthermore, increased speed also caused fewer words to be reread following both horizontal eye movements (likely resulting in reduced lexical processing) and vertical eye movements (which would likely reduce higher-level comprehension and integration).

Predictability effects and parafoveal processing in older readers

Normative aging is accompanied by visual and cognitive changes that impact the systems that are critical for fluent reading. The patterns of eye movements during reading displayed by older adults have been characterized as demonstrating a trade-off between longer forward saccades and more word skipping versus higher rates of regressions back to previously read text. This pattern is assumed to reflect older readers’ reliance on top-down contextual information to compensate for reduced uptake of parafoveal information from yet-to-be fixated words. However, the empirical evidence for these assumptions is equivocal. This study investigated the depth of older readers’ parafoveal processing as indexed by sensitivity to the contextual plausibility of parafoveal words in both neutral and highly constraining sentence contexts. The eye movements of 65 cognitively intact older adults (61-87 years) were compared with data previously collected from young adults in two sentence reading experiments in which critical target words were replaced by valid, plausible, related, or implausible previews until the reader fixated on the target word location. Older and younger adults showed equivalent plausibility preview benefits on first-pass reading measures of both predictable and unpredictable words. However, older readers did not show the benefit of preview orthographic relatedness that was observed in young adults, and showed significantly attenuated preview validity effects. Taken together, the data suggest that older readers are specifically impaired in the integration of parafoveal and foveal information but do not show deficits in the depth of parafoveal processing. The implications for understanding the effects of aging on reading are discussed.

Promoting L2 acquisition of multiword units through textually enhanced audiovisual input: An eye-tracking study

This study examines the effect of textual enhancement on learners’ attention to and learning of multiword units from captioned audiovisual input. We adopted a within-participants design in which 28 learners of English as a foreign language (EFL) watched a captioned video containing enhanced (underlined) and unenhanced multiword units. Using eye-tracking, we measured learners’ online processing of the multiword units as they appeared in the captions. Form recall pre- and posttests measured learners’ acquisition of the target items. The results of mixed effects models indicate that enhanced items received greater visual attention, with longer reading times, less single word skipping and more rereading. Further, a positive relationship was found between amount of visual attention and learning odds: items fixated longer, particularly during the first pass, were more likely to be recalled in an immediate posttest. Our findings provide empirical support for the positive effect of visual attention on form recall of multiword units encountered in captioned television. The results also suggest that item difficulty and amount of attention were more important than textual enhancement in predicting learning gains.

Effects of word predictability on eye movements during Arabic reading

Contextual predictability influences both the probability and duration of eye fixations on words when reading Latinate alphabetic scripts like English and German. However, it is unknown whether word predictability influences eye movements in reading similarly for Semitic languages like Arabic, which are alphabetic languages with very different visual and linguistic characteristics. Such knowledge is nevertheless important for establishing the generality of mechanisms of eye-movement control across different alphabetic writing systems. Accordingly, we investigated word predictability effects in Arabic in two eye-movement experiments. Both produced shorter fixation times for words with high compared to low predictability, consistent with previous findings. Predictability did not influence skipping probabilities for (four- to eight-letter) words of varying length and morphological complexity (Experiment 1). However, it did for short (three- to four-letter) words with simpler structures (Experiment 2). We suggest that word-skipping is reduced, and affected less by contextual predictability, in Arabic compared to Latinate alphabetic reading, because of specific orthographic and morphological characteristics of the Arabic script.

Prediction and preview strongly affect reading times but not skipping during natural reading

In a typical text, readers look much longer at some words than at others and fixate some words multiple times, while skipping others altogether. Historically, researchers explained this variation via low-level visual or oculomotor factors, but today it is primarily explained via cognitive factors, such as how well words can be predicted from context or discerned from parafoveal preview. While the existence of these effects has been well established in experiments, the relative importance of prediction, preview and low-level factors for eye movement variation in natural reading is unclear. Here, we address this question using a deep neural network and Bayesian ideal observer to model linguistic prediction and parafoveal preview from moment to moment in natural reading (n=104, 1.5 million words). Strikingly, neither prediction nor preview was important for explaining word skipping - the vast majority of skipping was explained by a simple oculomotor model. For reading times, by contrast, we found clear but independent contributions of both prediction and preview, and effect sizes matching those from controlled experiments. Together, these results challenge dominant models of eye movements in reading by showing that linguistic prediction and parafoveal preview are not important determinants of word skipping.

A dynamic adjustment account of word skipping in reading: Evidence from simulations and invisible boundary experiments

Prevalent models of eye movement control in reading of word-spaced orthographies assume a discrete control account to explain word skipping. Accordingly, word skipping results from advanced parafoveal word activation during preview, leading to the selection of word n+2 as a saccade target. Consequently, visual and linguistic content of the parafoveal word affect which word is selected as a saccade target. In contrast, according to dynamic adjustment view, saccade lengths in reading are affected by current and next word properties in a continuous manner. First, these predictions were confirmed via simulations for representative models of discrete and dynamic control. Then results of three gaze-contingent invisible boundary experiments studying the visuo-attentional mechanism of skipping are being reported. The results support the dynamic adjustment view by showing a unimodal landing position distribution across a candidate word for skipping and its subsequent word, and that the content of parafoveal word modulates the saccade length only slightly and in a continuous manner. In addition, the location of a visual anomaly within a word had no effect on saccade length, suggesting that parafoveal vision of next word is subject of attention enhancement early enough to affect saccade length computation.

Word skipping during the lexical acquisition process

During reading, most words are identified in the fovea through a direct fixation; however, readers also identify some words in the parafovea without directly fixating them. This word skipping process is influenced by many lexical and visual factors including word length, launch position, frequency, and predictability. Although these factors are well understood, there is some disagreement about the process that leads to word skipping and the degree to which skipped words are processed. The purpose of this study was to investigate the word skipping process when readers are exposed to novel words in an incidental lexical acquisition paradigm. Participants read 18 three-letter novel words (i.e., pru, cho) in three different informative contexts each while their eye movements were monitored. They then completed a surprise test of their orthographic and semantic acquisition and a spelling skill assessment. Mixed-effects models indicated that participants learned spellings and meanings of words at the same rate regardless of the number of times that they were skipped. However, word skipping rates increased across the three exposures and reading times decreased. Results indicate that readers appear to process skipped words to the same degree as fixated words. However, this may be due to a more cautious skipping process used during lexical acquisition of unfamiliar words compared to processing of already known words.

A further look at ageing and word predictability effects in Chinese reading: Evidence from one-character words

Older adults are thought to compensate for slower lexical processing by making greater use of contextual knowledge, relative to young adults, to predict words in sentences. Accordingly, compared to young adults, older adults should produce larger contextual predictability effects in reading times and skipping rates for words. Empirical support for this account is nevertheless scarce. Perhaps the clearest evidence to date comes from a recent Chinese study showing larger word predictability effects for older adults in reading times but not skipping rates for two-character words. However, one possibility is that the absence of a word-skipping effect in this experiment was due to the older readers skipping words infrequently because of difficulty processing two-character words parafoveally. We therefore took a further look at this issue, using one-character target words to boost word-skipping. Young (18–30 years) and older (65+ years) adults read sentences containing a target word that was either highly predictable or less predictable from the prior sentence context. Our results replicate the finding that older adults produce larger word predictability effects in reading times but not word-skipping, despite high skipping rates. We discuss these findings in relation to ageing effects on reading in different writing systems.