scholarly journals Prediction of upcoming speech under fluent and disfluent conditions: eye tracking evidence from immersive virtual reality

2020 ◽  
Author(s):  
Eleanor Huizeling ◽  
David Peeters ◽  
Peter Hagoort
Keyword(s):  
Virtual Reality ◽  
Eye Movements ◽  
Eye Tracking ◽  
Sentence Processing ◽  
Immersive Virtual Reality ◽  
Virtual Agent ◽  
Visual World Paradigm ◽  
Visual Objects ◽  
Auditory Speech ◽  
Efficient Processing

Traditional experiments indicate that prediction is important for the efficient processing of incoming speech. In three virtual reality (VR) visual world paradigm experiments, we here tested whether such findings hold in naturalistic settings (Experiment 1) and provided novel insights into whether disfluencies in speech (repairs/hesitations) inform one’s predictions in rich environments (Experiments 2-3). In all three experiments, participants’ eye movements were recorded while they listened to sentences spoken by a virtual agent during a virtual tour of eight scenes. Experiment 1 showed that listeners predict upcoming speech in naturalistic environments, with a higher proportion of anticipatory target fixations in Restrictive (predictable) compared to Unrestrictive (unpredictable) trials. Experiments 2-3 provided novel findings that disfluencies reduce anticipatory fixations towards a predicted referent in naturalistic environments, compared to Conjunction sentences (Experiment 2) and Fluent sentences (Experiment 3). Unexpectedly, Experiment 2 provided no evidence that participants made new predictions from a repaired verb – there was no increase in the proportion of fixations towards objects compatible with the repaired verb – thereby supporting an attention rather than a predictive account of effects of repair disfluencies on sentence processing. Experiment 3 provided novel evidence that the proportion of fixations to the speaker increased upon hearing a hesitation, supporting current theories of the effects of hesitations on sentence processing. Together, these findings contribute to a better understanding of how listeners make use of visual (objects, speaker) and auditory (speech, including disfluencies) information to predict upcoming words.

Eye movements in sports research and practice: Immersive technologies as optimal environments for the study of gaze behaviour

2020 ◽  
Author(s):  
David Harris ◽  
Mark Wilson ◽  
Tim Holmes ◽  
Toby de Burgh ◽  
Samuel James Vine
Keyword(s):  
Virtual Reality ◽  
Eye Movements ◽  
Eye Tracking ◽  
Visual Information ◽  
Successful Performance ◽  
Experimental Control ◽  
Research And Practice ◽  
Gaze Behaviour ◽  
Immersive Technologies

Head-mounted eye tracking has been fundamental for developing an understanding of sporting expertise, as the way in which performers sample visual information from the environment is a major determinant of successful performance. There is, however, a long running tension between the desire to study realistic, in-situ gaze behaviour and the difficulties of acquiring accurate ocular measurements in dynamic and fast-moving sporting tasks. Here, we describe how immersive technologies, such as virtual reality, offer an increasingly compelling approach for conducting eye movement research in sport. The possibility of studying gaze behaviour in representative and realistic environments, but with high levels of experimental control, could enable significant strides forward for eye tracking in sport and improve understanding of how eye movements underpin sporting skills. By providing a rationale for virtual reality as an optimal environment for eye tracking research, as well as outlining practical considerations related to hardware, software and data analysis, we hope to guide researchers and practitioners in the use of this approach.

scholarly journals Development and Calibration of an Eye-Tracking Fixation Identification Algorithm for Immersive Virtual Reality

Sensors ◽  
2020 ◽  
Vol 20 (17) ◽  
pp. 4956
Author(s):  
Jose Llanes-Jurado ◽  
Javier Marín-Morales ◽  
Jaime Guixeres ◽  
Mariano Alcañiz
Keyword(s):  
Virtual Reality ◽  
Eye Tracking ◽  
Tracking System ◽  
Time Windows ◽  
Relevant Information ◽  
Identification Algorithm ◽  
Immersive Virtual Reality ◽  
Head Mounted Display ◽  
Gaze Patterns ◽  
Head Mounted Displays

Fixation identification is an essential task in the extraction of relevant information from gaze patterns; various algorithms are used in the identification process. However, the thresholds used in the algorithms greatly affect their sensitivity. Moreover, the application of these algorithm to eye-tracking technologies integrated into head-mounted displays, where the subject’s head position is unrestricted, is still an open issue. Therefore, the adaptation of eye-tracking algorithms and their thresholds to immersive virtual reality frameworks needs to be validated. This study presents the development of a dispersion-threshold identification algorithm applied to data obtained from an eye-tracking system integrated into a head-mounted display. Rules-based criteria are proposed to calibrate the thresholds of the algorithm through different features, such as number of fixations and the percentage of points which belong to a fixation. The results show that distance-dispersion thresholds between 1–1.6° and time windows between 0.25–0.4 s are the acceptable range parameters, with 1° and 0.25 s being the optimum. The work presents a calibrated algorithm to be applied in future experiments with eye-tracking integrated into head-mounted displays and guidelines for calibrating fixation identification algorithms

scholarly journals THE USE OF EYE TRACKING IN NON-IMMERSIVE VIRTUAL REALITY FOR COGNITIVE ASSESSMENT

2016 ◽  
Vol 17 (1) ◽  
pp. 23-31 ◽  
Author(s):  
Pedro Rosa ◽  
Pedro Gamito ◽  
Jorge Oliveira ◽  
Diogo Morais ◽  
Matthew Pavlovic ◽  
...  
Keyword(s):  
Virtual Reality ◽  
Eye Tracking ◽  
Cognitive Assessment ◽  
Immersive Virtual Reality

Uses of Eye-Tracking Data in Second Language Sentence Processing Research

2010 ◽  
Vol 30 ◽  
pp. 149-166 ◽  
Author(s):  
Paola E. Dussias
Keyword(s):  
Second Language ◽  
Eye Movements ◽  
Eye Tracking ◽  
Language Processing ◽  
Sentence Processing ◽  
Future Research ◽  
Critical Question ◽  
The People ◽  
Recent Developments ◽  
Eye Movements During Reading

When hearing or reading words and sentences in a second language (L2), we face many uncertainties about how the people and objects referred to are connected to one another. So what do we do under these conditions of uncertainty? Because relatively proficient L2 speakers have access to the grammar and lexicon of each language when comprehending words and sentences or when planning spoken utterances, and because the recent research suggests that these linguistic systems are not entirely independent, there is a critical question about how the knowledge of two languages affects basic aspects of language processing. In this article, I review how eye-tracking methodology has been used as a tool to address this question. I begin by discussing why eye movements are a useful methodology in language processing research, and I provide a description of one experimental paradigm developed to explore eye movements during reading. Second, I present recent developments in the use of eye tracking to study L2 spoken-language comprehension. I also highlight the importance of using multiple measures of online sentence processing by discussing results obtained using a moving window task and eye-tracking records while L2 speakers read syntactically ambiguous relative clauses. Next, I discuss research investigating syntactic processing when L2 speakers process mixed language. I end with suggestions for future research directions.

Processing focus in native and non-native speakers of English: an eye-tracking study in the visual world paradigm

2021 ◽  
pp. 1-32
Author(s):  
Haoyan Ge ◽  
Iris Mulders ◽  
Xin Kang ◽  
Aoju Chen ◽  
Virginia Yip
Keyword(s):  
Eye Movements ◽  
Eye Tracking ◽  
Real Time ◽  
Native Speakers ◽  
Visual World ◽  
Visual World Paradigm ◽  
L2 Processing ◽  
Multiple Interfaces ◽  
Native Speakers Of English ◽  
The Difference

Abstract This “visual-world” eye-tracking study investigated the processing of focus in English sentences with preverbal only by L2 learners whose L1 was either Cantonese or Dutch, compared to native speakers of English. Participants heard only-sentences with prosodic prominence either on the object or on the verb and viewed pictures containing an object-focus alternative and a verb-focus alternative. We found that both L2 groups showed delayed eye movements to the alternative of focus, which was different from the native speakers of English. Moreover, Dutch learners of English were even slower than Cantonese learners of English in directing fixations to the alternative of focus. We interpreted the delayed fixation patterns in both L2 groups as evidence of difficulties in integrating multiple interfaces in real time. Furthermore, the similarity between English and Dutch in the use of prosody to mark focus hindered Dutch learners’ L2 processing of focus, whereas the difference between English and Cantonese in the realization of focus facilitated Cantonese learners’ processing of focus in English.

scholarly journals Eye-tracking in Immersive Virtual Reality for Education: A Review of the Current Progress and Applications

2021 ◽  
Author(s):  
Maria Mikhailenko ◽  
Mikhail Kurushkin
Keyword(s):  
Virtual Reality ◽  
Augmented Reality ◽  
Eye Tracking ◽  
Immersive Virtual Reality ◽  
The Third ◽  
General Overview ◽  
The Past ◽  
Virtual And Augmented Reality

The concept of using eye-tracking in virtual reality for education has been researched in various fields over the past years. With this review, we aim to discuss the recent advancements and applications in this area, explain the technological aspects, highlight the advantages of this approach and inspire interest in the field. Eye-tracking has already been used in science for many decades and now has been substantially reinforced by the addition of virtual and augmented reality technologies. The first part of the review is a general overview of eye-tracking concepts and its applications. In the second part, the focus shifted towards application of eye-tracking in virtual reality. The third part is the description of the recently emerged concept of eye-tracking in virtual reality when applied to education and studying, which has not been thoroughly described before. We describe the main findings, technological aspects and advantages of this approach.

Sign in / Sign up

Export Citation Format

Share Document