scholarly journals Integration of gaze information during online language comprehension and learning

2019 ◽  
Author(s):  
Kyle Earl MacDonald ◽  
Elizabeth Swanson ◽  
Michael C. Frank
Keyword(s):  
Visual Attention ◽  
Eye Tracking ◽  
Language Processing ◽  
Language Comprehension ◽  
Visual Information ◽  
Eye Gaze ◽  
The Other ◽  
Face To Face ◽  
Social Partners ◽  
Processing Mechanisms

Face-to-face communication provides access to visual information that can support language processing. But do listeners automatically seek social information without regard to the language processing task? Here, we present two eye-tracking studies that ask whether listeners’ knowledge of word-object links changes how they actively gather a social cue to reference (eye gaze) during real-time language processing. First, when processing familiar words, children and adults did not delay their gaze shifts to seek a disambiguating gaze cue. When processing novel words, however, children and adults fixated longer on a speaker who provided a gaze cue, which led to an increase in looking to the named object and less looking to the other object in the scene. These results suggest that listeners use their knowledge of object labels when deciding how to allocate visual attention to social partners, which in turn changes the visual input to language processing mechanisms.

The Opposition of Surprisal and Semantic Similarity in the Prediction of Language Processing: Evidence from Eye-tracking Data

2020 ◽  
Author(s):  
Kun Sun
Keyword(s):  
Eye Tracking ◽  
Semantic Similarity ◽  
Cognitive Processing ◽  
Language Processing ◽  
Language Comprehension ◽  
Word Processing ◽  
Reading Time ◽  
Computational Models ◽  
Tracking Data ◽  
Dynamic Approach

Expectations or predictions about upcoming content play an important role during language comprehension and processing. One important aspect of recent studies of language comprehension and processing concerns the estimation of the upcoming words in a sentence or discourse. Many studies have used eye-tracking data to explore computational and cognitive models for contextual word predictions and word processing. Eye-tracking data has previously been widely explored with a view to investigating the factors that influence word prediction. However, these studies are problematic on several levels, including the stimuli, corpora, statistical tools they applied. Although various computational models have been proposed for simulating contextual word predictions, past studies usually preferred to use a single computational model. The disadvantage of this is that it often cannot give an adequate account of cognitive processing in language comprehension. To avoid these problems, this study draws upon a massive natural and coherent discourse as stimuli in collecting the data on reading time. This study trains two state-of-art computational models (surprisal and semantic (dis)similarity from word vectors by linear discriminative learning (LDL)), measuring knowledge of both the syntagmatic and paradigmatic structure of language. We develop a `dynamic approach' to compute semantic (dis)similarity. It is the first time that these two computational models have been merged. Models are evaluated using advanced statistical methods. Meanwhile, in order to test the efficiency of our approach, one recently developed cosine method of computing semantic (dis)similarity based on word vectors data adopted is used to compare with our `dynamic' approach. The two computational and fixed-effect statistical models can be used to cross-verify the findings, thus ensuring that the result is reliable. All results support that surprisal and semantic similarity are opposed in the prediction of the reading time of words although both can make good predictions. Additionally, our `dynamic' approach performs better than the popular cosine method. The findings of this study are therefore of significance with regard to acquiring a better understanding how humans process words in a real-world context and how they make predictions in language cognition and processing.

Affective language processing and bilingualism: Complementary perspectives

2017 ◽  
Vol 53 (1) ◽  
pp. 1-16
Author(s):  
Ewa Tomczak ◽  
Dorota Jaworska-Pasterska
Keyword(s):  
Language Processing ◽  
Language Comprehension ◽  
The Other ◽  
Academic Interest ◽  
Current Volume

AbstractRecent years have witnessed revival of academic interest in the study of two areas. One is related to processing of emotional input, both linguistic and nonlinguistic; the other is centred on mechanisms underlying bilingual language comprehension and production. The current volume comprises substantial contributions by researchers working within various fields of linguistics and psychology. The Authors elaborate upon cognitively sophisticated frameworks for conceptualising the complexities of attitudes towards and beliefs about language, i.e.

Seeing the Way: the Role of Vision in Conversation Turn Exchange Perception

2017 ◽  
Vol 30 (7-8) ◽  
pp. 653-679 ◽  
Author(s):  
Nida Latif ◽  
Agnès Alsius ◽  
K. G. Munhall
Keyword(s):  
Visual Information ◽  
Visual Cues ◽  
Social Cues ◽  
The Other ◽  
Auditory Information ◽  
Face To Face ◽  
Turn Taking ◽  
Overall Efficiency ◽  
Natural Conversation

During conversations, we engage in turn-taking behaviour that proceeds back and forth effortlessly as we communicate. In any given day, we participate in numerous face-to-face interactions that contain social cues from our partner and we interpret these cues to rapidly identify whether it is appropriate to speak. Although the benefit provided by visual cues has been well established in several areas of communication, the use of visual information to make turn-taking decisions during conversation is unclear. Here we conducted two experiments to investigate the role of visual information in identifying conversational turn exchanges. We presented clips containing single utterances spoken by single individuals engaged in a natural conversation with another. These utterances were from either right before a turn exchange (i.e., when the current talker would finish and the other would begin) or were utterances where the same talker would continue speaking. In Experiment 1, participants were presented audiovisual, auditory-only and visual-only versions of our stimuli and identified whether a turn exchange would occur or not. We demonstrated that although participants could identify turn exchanges with unimodal information alone, they performed best in the audiovisual modality. In Experiment 2, we presented participants audiovisual turn exchanges where the talker, the listener or both were visible. We showed that participants suffered a cost at identifying turns exchanges when visual cues from the listener were not available. Overall, we demonstrate that although auditory information is sufficient for successful conversation, visual information plays an important role in the overall efficiency of communication.

scholarly journals Visual attention during the evaluation of facial attractiveness is influenced by facial angles and smile

2018 ◽  
Vol 88 (3) ◽  
pp. 329-337 ◽  
Author(s):  
Seol Hee Kim ◽  
Soonshin Hwang ◽  
Yeon-Ju Hong ◽  
Jae-Jin Kim ◽  
Kyung-Ho Kim ◽  
...  
Keyword(s):  
Young Adults ◽  
Visual Attention ◽  
Eye Tracking ◽  
Eye Gaze ◽  
Facial Attractiveness ◽  
Fixation Time ◽  
Facial Esthetics ◽  
Attractive Face ◽  
Tracking Device ◽  
Face Task

ABSTRACT Objective: To examine the changes in visual attention influenced by facial angles and smile during the evaluation of facial attractiveness. Materials and Methods: Thirty-three young adults were asked to rate the overall facial attractiveness (task 1 and 3) or to select the most attractive face (task 2) by looking at multiple panel stimuli consisting of 0°, 15°, 30°, 45°, 60°, and 90° rotated facial photos with or without a smile for three model face photos and a self-photo (self-face). Eye gaze and fixation time (FT) were monitored by the eye-tracking device during the performance. Participants were asked to fill out a subjective questionnaire asking, “Which face was primarily looked at when evaluating facial attractiveness?” Results: When rating the overall facial attractiveness (task 1) for model faces, FT was highest for the 0° face and lowest for the 90° face regardless of the smile (P < .01). However, when the most attractive face was to be selected (task 2), the FT of the 0° face decreased, while it significantly increased for the 45° face (P < .001). When facial attractiveness was evaluated with the simplified panels combined with facial angles and smile (task 3), the FT of the 0° smiling face was the highest (P < .01). While most participants reported that they looked mainly at the 0° smiling face when rating facial attractiveness, visual attention was broadly distributed within facial angles. Conclusions: Laterally rotated faces and presence of a smile highly influence visual attention during the evaluation of facial esthetics.

Activating Gender Stereotypes During Online Spoken Language Processing

2010 ◽  
Vol 57 (2) ◽  
pp. 126-133 ◽  
Author(s):  
Pirita Pyykkönen ◽  
Jukka Hyönä ◽  
Roger P. G. van Gompel
Keyword(s):  
Eye Tracking ◽  
Mental Models ◽  
Language Processing ◽  
Gender Stereotypes ◽  
Language Comprehension ◽  
Spoken Language ◽  
World Knowledge ◽  
Tracking Method ◽  
Spoken Language Processing ◽  
Coherence Relation

This study used the visual world eye-tracking method to investigate activation of general world knowledge related to gender-stereotypical role names in online spoken language comprehension in Finnish. The results showed that listeners activated gender stereotypes elaboratively in story contexts where this information was not needed to build coherence. Furthermore, listeners made additional inferences based on gender stereotypes to revise an already established coherence relation. Both results are consistent with mental models theory (e.g., Garnham, 2001 ). They are harder to explain by the minimalist account ( McKoon & Ratcliff, 1992 ), which suggests that people limit inferences to those needed to establish coherence in discourse.

A study of customer perception of visual information in food stands through eye-tracking

2021 ◽  
Vol ahead-of-print (ahead-of-print) ◽  
Author(s):  
Yoowha Jeon ◽  
Mi Sook Cho ◽  
Jieun Oh
Keyword(s):  
Visual Attention ◽  
Eye Tracking ◽  
Visual Processing ◽  
Visual Information ◽  
Food Service ◽  
Small Scale ◽  
Textual Information ◽  
Content Type ◽  
Movement Data ◽  
Significant Difference

PurposeThe study selected five small-scale food operations as visual stimuli and eye-tracking experiment was conducted with 36 female participants in a laboratory setting. Heat maps were used to visualize viewers' visual attention on the storefronts. The eye-movement data were analyzed using one-way repeated ANOVA to identify a significant difference between stimuli in terms of average fixation duration, fixation counts and revisits. An independent t-test was also used to examine statistical difference among text and image in menu board. The significance cut-off of p-value was set to <0.05.Design/methodology/approachThe exteriors of food-service establishments are major business representation. However, few studies have been conducted to examine customers' visual processing toward small-scale restaurants. The present study accordingly aims to discover customers' different levels of attention to the frontage in food stands through eye tracking, which would be practical for future owners to plan their exterior shop design.FindingsThe findings can be summarized as follows: First, upper board shows the highest level of attention, suggesting an optimal location of menu board for grasping customers' attention. Second, customers also gaze the inside of a store along with the food on display, which are related with food hygiene and the perception. Third, textual information on menu boards tends to attract more visual attention than those of images. Overall, the current study indicates various customers' attention toward the location of menu boards as well as the type of visual information on menu board.Originality/valueThe results of this study make a new insight into customers' viewing behavior toward exteriors of food-service establishments. This study is one of the first attempts to explore how customers distribute visual attention to the exterior images of food stand by using eye-tracking technology. The findings of this research thus enrich the food-service literature and offer meaningful discoveries on customers' visual behaviors. For example, this study suggests that customers tend to be attracted to textual information on menu boards rather than graphical ones.

The Use of Eye-Tracking in Spatial Thinking Research

2019 ◽  
pp. 588-609
Author(s):  
Alina Nazareth ◽  
Rosalie Odean ◽  
Shannon M. Pruden
Keyword(s):  
Eye Tracking ◽  
Language Processing ◽  
Cognitive Processes ◽  
Response Times ◽  
Eye Gaze ◽  
Cognitive Strategy ◽  
Spatial Thinking ◽  
Educational Pedagogy ◽  
Spatial Tasks ◽  
Tracking Technology

This chapter highlights the benefits of eye-tracking technology in spatial thinking research, specifically in the study of complex cognitive processes used to solve spatial tasks including cognitive strategy selection, cognitive strategy flexibility and spatial language processing. The consistent sex differences found in spatial thinking research (i.e., mental rotation), with males outperforming females, is concerning given the link between spatial ability and success in the STEM fields. Traditional methods like self-reports, checklists and response times methods may not be sufficient to study complex cognitive processes. Advances in eye-tracking technology make it possible to efficiently record and analyze voluminous eye-gaze data as an indirect measure of underlying cognitive processes involved in solving spatial tasks. A better understanding of the cognitive processes underlying spatial thinking will facilitate the design of effective training and educational pedagogy that encourages spatial thinking across both males and females.

Eye Tracking and Spoken Language Comprehension

2017 ◽  
pp. 88-105
Author(s):  
Elizabeth Kaplan ◽  
Tatyana Levari ◽  
Jesse Snedeker
Keyword(s):  
Young Children ◽  
Eye Tracking ◽  
Language Processing ◽  
Language Comprehension ◽  
Developmental Disorders ◽  
Spoken Language ◽  
Syntactic Analysis ◽  
Primary Focus ◽  
Insight Into ◽  
Spoken Language Comprehension

Constructing a more precise and deeper understanding of how listeners, and particularly young children, comprehend spoken language is a primary focus for both psycholinguists and educators alike. This chapter highlights how, over the course of the past 20 years, eye tracking has become a crucial and widely used methodology to gain insight into online spoken language comprehension. We address how various eye-tracking paradigms have informed current theories of language comprehension across the processing stream, focusing on lexical discrimination, syntactic analysis, and pragmatic inferences. Additionally, this chapter aims to bridge the gap between psycholinguistic research and educational topics, such as how early linguistic experiences influence later educational outcomes and ways in which eye-tracking methods can provide additional insight into the language processing of children with developmental disorders.

scholarly journals Does headedness affect processing? A new look at the VO–OV contrast

2009 ◽  
Vol 45 (3) ◽  
pp. 675-710 ◽  
Author(s):  
MIEKO UENO ◽  
MARIA POLINSKY
Keyword(s):  
Language Processing ◽  
Language Comprehension ◽  
Word Order ◽  
The Other ◽  
Differential Processing ◽  
Corpus Studies ◽  
Other Hand ◽  
Compensatory Strategy ◽  
The Relationship ◽  
Affect Processing

This paper examines the relationship between headedness and language processing and considers two strategies that potentially ease language comprehension and production. Both strategies allow a language to minimize the number of arguments in a given clause, either by reducing the number of overtly expressed arguments or by reducing the number of structurally required arguments. The first strategy consists of minimizing the number of overtly expressed arguments by using more pro-drop for two-place predicates (Pro-drop bias). According to the second strategy, a language gives preference to one-place predicates over two-place predicates, thus minimizing the number of structural arguments (Intransitive bias). In order to investigate these strategies, we conducted a series of comparative corpus studies of SVO and SOV languages. Study 1 examined written texts of various genres and children's utterances in English and Japanese, while Study 2 examined narrative stories in English, Spanish, Japanese, and Turkish. The results for these studies showed that pro-drop was uniformly more common with two-place predicates than with one-place predicates, regardless of the OV/VO distinction. Thus the Pro-drop bias emerges as a universal economy principle for making utterances shorter. On the other hand, SOV languages showed a much stronger Intransitive bias than SVO languages. This finding suggests that SOV word order with all the constituents explicitly expressed is potentially harder to process; the dominance of one-place predicates is therefore a compensatory strategy in order to reduce the number of preverbal arguments. The overall pattern of results suggests that human languages utilize both general (Pro-drop bias) and headedness-order-specific (Intransitive bias) strategies to facilitate processing. The results on headedness-order-specific strategies are consistent with other researchers' findings on differential processing in head-final and non-head-final languages, for example, Yamashita & Chang's (2001) ‘long-before-short’ parameterization.

scholarly journals Avoiding the Other Side?

2020 ◽  
Vol 32 (3) ◽  
pp. 158-164
Author(s):  
Desirée Schmuck ◽  
Miriam Tribastone ◽  
Jörg Matthes ◽  
Franziska Marquart ◽  
Eva Maria Bergel
Keyword(s):  
Eye Movements ◽  
Visual Attention ◽  
Eye Tracking ◽  
Selective Exposure ◽  
The Other ◽  
Research Attention ◽  
Political Views ◽  
Political Ads ◽  
First Time ◽  
Practical Implications

Abstract. Partisan selective exposure, the phenomenon of selectively attending to information that is in line with one’s political views, has received extensive research attention. Researchers have thus far largely neglected, however, to examine the tendency to avoid attitude-discrepant information, that is, selective avoidance. Selective avoidance can be considered a different phenomenon that is not necessarily implied by, nor only occurs simultaneously with, selective exposure. This study investigates these two separate phenomena, for the first time, using eye-tracking methodology. We exposed participants to political ads by liberal and conservative parties placed next to neutral political ads and tracked eye movements unobtrusively. Findings showed that individuals paid more visual attention to political ads that were consistent with their partisan ideology. Additionally, we found that individuals tended to avoid political ads that were inconsistent with their partisan ideology, which provides some evidence for selective avoidance processes. Theoretical and practical implications of these findings are discussed.

Sign in / Sign up

Export Citation Format

Share Document