The impact of audio on the reading of intralingual versus interlingual subtitles: Evidence from eye movements

2021 ◽  
pp. 1-33
Author(s):  
Sixin Liao ◽  
Lili Yu ◽  
Jan-Louis Kruger ◽  
Erik D. Reichle
Keyword(s):  
Second Language ◽  
Eye Movements ◽  
Eye Movement ◽  
Information Sources ◽  
Lexical Processing ◽  
Auditory Information ◽  
Video Content ◽  
Video Presentation ◽  
Chinese Speakers ◽  
The Impact

Abstract This study investigated how semantically relevant auditory information might affect the reading of subtitles, and if such effects might be modulated by the concurrent video content. Thirty-four native Chinese speakers with English as their second language watched video with English subtitles in six conditions defined by manipulating the nature of the audio (Chinese/L1 audio vs. English/L2 audio vs. no audio) and the presence versus absence of video content. Global eye-movement analyses showed that participants tended to rely less on subtitles with Chinese or English audio than without audio, and the effects of audio were more pronounced in the presence of video presentation. Lexical processing of subtitles was not modulated by the audio. However, Chinese audio, which presumably obviated the need to read the subtitles, resulted in more superficial post-lexical processing of the subtitles relative to either the English or no audio. On the contrary, English audio accentuated post-lexical processing of the subtitles compared with Chinese audio or no audio, indicating that participants might use English audio to support subtitle reading (or vice versa) and thus engaged in deeper processing of the subtitles. These findings suggest that, in multimodal reading situations, eye movements are not only controlled by processing difficulties associated with properties of words (e.g., their frequency and length) but also guided by metacognitive strategies involved in monitoring comprehension and its online modulation by different information sources.

Why subtitle speed matters: Evidence from word skipping and rereading

2021 ◽  
pp. 1-26
Author(s):  
Jan-Louis Kruger ◽  
Natalia Wisniewska ◽  
Sixin Liao
Keyword(s):  
Eye Movements ◽  
Eye Tracking ◽  
Eye Movement ◽  
Lexical Processing ◽  
Situation Model ◽  
Linguistic Processing ◽  
Reading Behavior ◽  
New Findings ◽  
Word Skipping

Abstract High subtitle speed undoubtedly impacts the viewer experience. However, little is known about how fast subtitles might impact the reading of individual words. This article presents new findings on the effect of subtitle speed on viewers’ reading behavior using word-based eye-tracking measures with specific attention to word skipping and rereading. In multimodal reading situations such as reading subtitles in video, rereading allows people to correct for oculomotor error or comprehension failure during linguistic processing or integrate words with elements of the image to build a situation model of the video. However, the opportunity to reread words, to read the majority of the words in the subtitle and to read subtitles to completion, is likely to be compromised when subtitles are too fast. Participants watched videos with subtitles at 12, 20, and 28 characters per second (cps) while their eye movements were recorded. It was found that comprehension declined as speed increased. Eye movement records also showed that faster subtitles resulted in more incomplete reading of subtitles. Furthermore, increased speed also caused fewer words to be reread following both horizontal eye movements (likely resulting in reduced lexical processing) and vertical eye movements (which would likely reduce higher-level comprehension and integration).

scholarly journals Lexical and post-lexical complexity effects on eye movements in reading

2011 ◽  
Vol 4 (1) ◽  
Author(s):  
Tessa Warren ◽  
Erik D. Reichle ◽  
Nikole D. Patson
Keyword(s):  
Eye Movements ◽  
Eye Movement ◽  
Lexical Processing ◽  
Movement Control ◽  
Eye Movement Control ◽  
Lexical Complexity ◽  
Eye Movements During Reading ◽  
Processing Difficulty

The current study investigated how a post-lexical complexity manipulation followed by a lexical complexity manipulation affects eye movements during reading. Both manipulations caused disruption in all measures on the manipulated words, but the patterns of spillover differed. Critically, the effects of the two kinds of manipulations did not interact, and there was no evidence that post-lexical processing difficulty delayed lexical processing on the next word (c.f. Henderson & Ferreira, 1990). This suggests that post-lexical processing of one word and lexical processing of the next can proceed independently and likely in parallel. This finding is consistent with the assumptions of the E-Z Reader model of eye movement control in reading (Reichle, Warren, & McConnell, 2009).

scholarly journals Hearing in a world of light: why, where, and how visual and auditory information are connected by the brain

2019 ◽  
Vol 12 (7) ◽  
Author(s):  
Jennifer M. Groh
Keyword(s):  
Eye Movements ◽  
Eye Movement ◽  
Auditory Processing ◽  
Activity Patterns ◽  
Visual Space ◽  
Critical Issue ◽  
Video Stream ◽  
Auditory Information ◽  
Auditory Space ◽  
Keynote Presentation

Keynote by Jenny Groh (Duke University) at the 20th European Conference on Eye Movement Research (ECEM) in Alicante, 19.8.2019 Video stream: https://vimeo.com/356576513 Abstract: Information about eye movements with respect to the head is required for reconciling visual and auditory space. This keynote presentation describes recent findings concerning how eye movements affect early auditory processing via motor processes in the ear (eye movement-related eardrum oscillations, or EMREOs). Computational efforts to understand how eye movements are factored in to auditory processing to produce a reference frame aligned with visual space uncovered a second critical issue: sound location is not mapped but is instead rate (meter) coded in the primate brain, unlike visual space. Meter coding would appear to limit the representation of multiple simultaneous sounds. The second part of this presentation concerns how such a meter code could use fluctuating activity patterns to circumvent this limitation

scholarly journals Word Skipping in Reading English as a Foreign Language: Evidence from Eye Tracking

2016 ◽  
Vol 3 (2) ◽  
pp. 22-31
Author(s):  
Емраг Долґунсоз ◽  
Аріф Сарісобан
Keyword(s):  
Second Language ◽  
Cognitive Psychology ◽  
Eye Movements ◽  
Eye Tracking ◽  
Eye Movement ◽  
Movement Control ◽  
Vocabulary Knowledge ◽  
Psychological Review ◽  
Eye Movement Control ◽  
Word Skipping

During reading, readers never fixate on all words in the text; shorter words sometimes gain zero fixation and skipped by the reader. Relying on E-Z Reader Model, this research hypothesized that a similar skipping effect also exists for a second language. The current study examined word skipping rates in EFL (English as a Foreign Language) with 75 EFL learners by using eye tracking methodology. The results showed that word skipping was affected by EFL reading proficiency significantly and articles (a, an, the) were skipped more than content words. Furthermore, more skilled learners were observed to have less fixation count and skipped more words during reading while less skilled learners employed more fixations and skipped less words. Eye tracking as a novel method to observe learner development and progress in EFL reading was also discussed.  References Altarriba, J., Kroll, J. F., Sholl, A.. & Rayner, K. (1996). The influence of lexical andconceptual constraints on reading mixed-language sentences: Evidence from eye fixations andnaming times. Memory & Cognition, 24, 477–492. Balota, D. A., Pollatsek, A., & Rayner, K. (1985). The interaction of contextual constraints andparafoveal visual information in reading. Cognitive Psychology, 17, 364–388. Binder, K. S., Pollatsek, A., & Rayner, K. (1999). Extraction of information to the left of thefixated word in reading. Journal of Experimental Psychology: Human Perception andPerformance, 25, 1162–1172. Brysbaert, M., & Vitu, F. (1998). Word Skipping: Implications for Theories of Eye MovementControl in Reading. In: Eye Guidance in Reading and Scene Perception. (pp. 125–147).G. Underwood, (Ed.). Oxford: Elsevier. Carpenter, P. A., & Just, M. A. (1983). What your eyes do while your mind is reading. In: EyeMovements in Reading: Perceptual and Language processes , (pp. 275–307), K. Rayner (ed.).New York: Academic Press. Djamasbi, S., Siegel, M., Skorinko, J., & Tullis, T. (2011). Online viewing and aestheticpreferences of generation y and the baby boom generation: Testing user web site experiencethrough eye tracking. International Journal of Electronic Commerce, 15(4), 121–158. Dolgunsöz, E. (2015). Measuring Attention in Second Language Reading Using Eye-tracking:The Case of the Noticing Hypothesis. Journal of Eye Movement Research, 8(5). Drieghe, D., Brysbaert, M., Desmet, T., & De Baecke, C. (2004). Word skipping in reading: Onthe interplay of linguistic and visual factors. European Journal of Cognitive Psychology,16(1–2), 79–103. Godfroid, A., Boers, F., & Housen, A. (2013). An eye for words: Gauging the role of attentionin incidental L2 vocabulary acquisition by means of eye-tracking. Studies in Second languageAcquisition, 35(3), 483–517. Henderson, J. M., & Ferreira, F. (1993). Eye movement control during reading: Fixationmeasures reflect foveal but not parafoveal processing difficulty. Canadian Journal ofExperimental Psychology, 47, 201–221. Joe, A. (1995). Text based tasks and incidental vocabulary learning. Foreign languageResearch, 11(2), 95–111. Just, M. A., & Carpenter, P. (1980). A theory of reading: From eye fixations tocomprehension. Psychological Review, 85, 109–130. Liu, P. L. (2014). Using eye tracking to understand the responses of learners to vocabularylearning strategy instruction and use. Computer Assisted Language Learning, 27(4), 330–343. McNeill, A. (1996). Vocabulary Knowledge profiles: Evidence from Chinese speaking ESLspeakers. Hong Kong Journal of Applied Linguistics 1(1), 39–63. Pollatsek, A., Reichle, E., & Rayner, K. (2003). Modeling eye movements in reading. In: TheMind’s Eyes: Cognitive and Applied Aspects of Eye Movement Research. (pp. 361–390).J. Hyona, R. Radach, & H. Deubel, (Eds.). Amsterdam: Elsevier. Radach, R., & Kempe, V. (1993). An individual analysis of initial fixation positions inreading. In: Perception and cognition: Advances in eye movement research (pp. 213–226). G.d’Ydewalle & J. Van Rensbergen (Eds.). Amsterdam: North Holland. Rayner, K. (1998). Eye Movements in Reading and Information Processing: 20 Years ofResearch, Psychological Bulletin, 124 (3), 372–422 Rayner, K., & Fischer, M. H. (1996). Mindless reading revisited: eye movements duringreading and scanning are different. Perception & Psychophysics, 58(5), 734–747. Rayner, K., & Well, A. D. (1996). Effects of contextual constraint on eye movements duringreading: a further examination. Psychonomic Bulletin & Review, 3, 504–509. Rayner, K., Binder, K. S., Ashby, J., & Pollatsek, A. (2001). Eye movement control inreading: word predictability has little influence on initial landing positions in words. VisionResearch, 41(7), 943–954. Rayner, K., Reichle, E. D., & Pollatsek, A. (2005). Eye movement control in reading and theE-Z Reader model. In: Cognitive Processes in Eye Guidance (pp. 131-162). G. Underwood(Ed.),. Oxford: Oxford University Press. Rayner, K., Sereno, S. C., & Raney, G. E. (1996). Eye movement control in reading: acomparison of two types of models. Journal of Experimental Psychology: Human Perceptionand Performance, 22, 1188–1200. Reichle, E., Pollatsek, A., Fisher, D. L., & Rayner, K. (1998). Toward a model of eyemovement control in reading. Psychological Review, 105, 125–157. Scarcella, R. & C. Zimmerman (1998). ESL student performance on a text of academiclexicon. Studies in Second language Acquisition, 20(1), 27–49. Schilling, H. E., Rayner, K., & Chumbley, J. I. (1998). Comparing naming, lexical decision,and eye fixation times: Word frequency effects and individual differences. Memory &Cognition, 26(6), 1270–1281. Schroeder, S., Hyönä, J., & Liversedge, S. P. (2015). Developmental eye-tracking research inreading: Introduction to the special issue. Journal of Cognitive Psychology, 27(5), 500–510. Smith, B. (2012). Eye tracking as a measure of noticing: A study of explicit recasts in SCMC.Language Learning & Technology, 16(3), 53–81. Wesche, M. & T. Paribakht (1996). Assessing vocabulary knowledge: depth vs. breadth.Canadian Modern Language Review, 53(1), 13–40. Winke, P., Gass, S., & Sydorenko, T. (2013). Factors Influencing the Use of Captions byForeign Language Learners: An Eye‐Tracking Study. The Modern Language Journal, 97(1),254–275.

scholarly journals Fixational eye movement waveforms in amblyopia: Characteristics of fast and slow eye movements

2019 ◽  
Vol 12 (6) ◽  
Author(s):  
Sarah Linda Kang ◽  
Sinem B. Beylergil ◽  
Jorge Otero-Millan ◽  
Aasef Shaikh ◽  
Fatema Ghasia
Keyword(s):  
Visual Acuity ◽  
Eye Movements ◽  
Eye Movement ◽  
Treatment Effectiveness ◽  
Eye Diseases ◽  
Fixational Eye Movements ◽  
Fixation Stability ◽  
Slow Eye Movements ◽  
Fixational Saccades ◽  
The Impact

Fixational eye movements comprise of fast microsaccades alternating with slow inter-saccadic drifts. These physiologic eye movements play an important role in visual perception.  Amblyopic patients are known to have fixation instability, particularly of the amblyopic eye. We examined eye movement abnormalities that contribute to this instability. We found that fixation stability is affected by the presence of fusion maldevelopment nystagmus (FMN). However, some amblyopes can have nystagmus without nasally directed slow phases and reversal in direction of the quick phase on ocular occlusion, features seen in FMN. In patients without nystagmus, we found increased amplitude of fixational saccades and inter-saccadic drifts. We categorized amblyopia patients by type (anisometropic, strabismic, or mixed) and eye movement waveform (no nystagmus, nystagmus without FMN, and FMN). We found specific fast and slow eye movement abnormalities of the fellow and amblyopic eye during fellow, amblyopic and both eyes viewing conditions across eye movement waveforms and types of amblyopia. These eye movement abnormalities can serve as biomarkers that can predict the impact of amblyopia as measured by visual acuity and stereopsis. Evaluation of fixational eye movements in amblyopia could be important to diagnose these common eye diseases and predict treatment effectiveness. 

scholarly journals Eye tracker as an implied social presence: awareness of being eye-tracked induces social-norm-based looking behaviour

2019 ◽  
Vol 12 (2) ◽  
Author(s):  
Hoo Keat Wong ◽  
Ian D. Stephen
Keyword(s):  
Eye Movements ◽  
Eye Movement ◽  
Social Presence ◽  
Recognition Task ◽  
Movement Pattern ◽  
The Body ◽  
Eye Tracker ◽  
Movement Behaviour ◽  
Area Of Interest ◽  
The Impact

Human behaviour is not only influenced by the physical presence of others, but also implied social presence. According to Risko and Kingstone (2011), an eye tracker can represent an implied social presence which could influence individuals’ gaze behaviour. This study examines the impact of awareness of being eye-tracked on eye movement behaviour in a laboratory setting.  During a classic yes/no face recognition task, participants were made to believe that their eye movements were recorded (or not recorded) by eye trackers. Their looking patterns with and without the awareness of being eye-tracked were compared while perceiving social (faces, faces-and-bodies) and non-social (inanimate objects) video stimuli. Area-of-interest (AOI) analysis revealed that misinformed participants (who were not aware that their eye movements were being recorded) looked more at the body (chest and waist) compared to informed participants (who believed they were being eye-tracked), whereas informed participants fixated longer on the mouth and shorter on the eyes of female models than misinformed participants did. These findings highlight the potential impact of an awareness of being eye tracked on one’s eye movement pattern when perceiving a social stimulus. We therefore suggest that even within laboratory settings an eye tracker may function as an implied social presence that leads individuals to modify their eye movement behaviour according to socially-derived inhibitory norms.

scholarly journals Character-complexity effects in Chinese reading and visual search: A comparison and theoretical implications

2018 ◽  
Vol 71 (1) ◽  
pp. 140-151 ◽  
Author(s):  
Lili Yu ◽  
Qiaoming Zhang ◽  
Caspian Priest ◽  
Erik D Reichle ◽  
Heather Sheridan
Keyword(s):  
Eye Movement ◽  
Visual Processing ◽  
Lexical Processing ◽  
Search Task ◽  
Visual Complexity ◽  
English Speakers ◽  
Chinese Reading ◽  
Chinese Speakers ◽  
Fixation Durations ◽  
Native Chinese Speakers

Three eye-movement experiments were conducted to examine how the complexity of characters in Chinese words (i.e., number of strokes per character) influences their processing and eye-movement behaviour. In Experiment 1, English speakers with no significant knowledge of Chinese searched for specific low-, medium-, and high-complexity target characters in a multi-page narrative containing characters of varying complexity (3–16 strokes). Fixation durations and skipping rates were influenced by the visual complexity of both the target characters and the characters being searched even though participants had no knowledge of Chinese. In Experiment 2, native Chinese speakers performed the same character-search task, and a similar pattern of results was observed. Finally, in Experiment 3, a second sample of native Chinese speakers read the same text used in Experiments 1 and 2, with text characters again exhibiting complexity effects. These results collectively suggest that character-complexity effects on eye movements may not be due to lexical processing per se but may instead reflect whatever visual processing is required to know whether or not a character corresponds to an episodically represented target. The theoretical implications of this for our understanding of normal reading are discussed.

scholarly journals Impairment of Shooting Performance by Background Complexity and Motion

2015 ◽  
Vol 62 (2) ◽  
pp. 98-109 ◽  
Author(s):  
Loïc Caroux ◽  
Ludovic Le Bigot ◽  
Nicolas Vibert
Keyword(s):  
Eye Movements ◽  
Video Games ◽  
Virtual Environments ◽  
Virtual Environment ◽  
Eye Movement ◽  
Visual Features ◽  
Lateral Motion ◽  
Gaze Control ◽  
Background Motion ◽  
The Impact

In many visual displays such as virtual environments, human tasks involve objects superimposed on both complex and moving backgrounds. However, most studies investigated the influence of background complexity or background motion in isolation. Two experiments were designed to investigate the joint influences of background complexity and lateral motion on a simple shooting task typical of video games. Participants had to perform the task on the moving and static versions of backgrounds of three levels of complexity, while their eye movements were recorded. The backgrounds displayed either an abstract (Experiment 1) or a naturalistic (Experiment 2) virtual environment. The results showed that performance was impaired by background motion in both experiments. The effects of motion and complexity were additive for the abstract background and multiplicative for the naturalistic background. Eye movement recordings showed that performance impairments reflected at least in part the impact of the background visual features on gaze control.

scholarly journals Searching for Chaos Evidence in Eye Movement Signals

Entropy ◽  
2018 ◽  
Vol 20 (1) ◽  
pp. 32 ◽  
Author(s):  
◽  
Keyword(s):  
Time Series ◽  
Fractal Dimension ◽  
Eye Movements ◽  
Noise Reduction ◽  
Eye Movement ◽  
Eye Tracker ◽  
Dynamic Features ◽  
Pass Filter ◽  
Low Pass Filter ◽  
The Impact

Most naturally-occurring physical phenomena are examples of nonlinear dynamic systems, the functioning of which attracts many researchers seeking to unveil their nature. The research presented in this paper is aimed at exploring eye movement dynamic features in terms of the existence of chaotic nature. Nonlinear time series analysis methods were used for this purpose. Two time series features were studied: fractal dimension and entropy, by utilising the embedding theory. The methods were applied to the data collected during the experiment with “jumping point” stimulus. Eye movements were registered by means of the Jazz-novo eye tracker. One thousand three hundred and ninety two (1392) time series were defined, based on the horizontal velocity of eye movements registered during imposed, prolonged fixations. In order to conduct detailed analysis of the signal and identify differences contributing to the observed patterns of behaviour in time scale, fractal dimension and entropy were evaluated in various time series intervals. The influence of the noise contained in the data and the impact of the utilized filter on the obtained results were also studied. The low pass filter was used for the purpose of noise reduction with a 50 Hz cut-off frequency, estimated by means of the Fourier transform and all concerned methods were applied to time series before and after noise reduction. These studies provided some premises, which allow perceiving eye movements as observed chaotic data: characteristic of a space-time separation plot, low and non-integer time series dimension, and the time series entropy characteristic for chaotic systems.

Sign in / Sign up

Export Citation Format

Share Document