Chapter 1. Eye-tracking: facts and figures

In chapter 1 we describe the method of eye-tracking and how the interest to studying eye movements developed in time. We describe how modern eye-tracking devices work, including several most commonly used in cognitive research (SR-Research, SMI, Tobii). We also give some general information about eye movement parameters during reading and a brief over- view of main models of eye movement control in reading (SWIFT, E-Z Reader). These models take into account a significant amount of empirical data and simulate the interaction of oculo- motor and cognitive processes involved in reading. Differences between the models, as well as different interpretations allowed within the same model, reflect the complexity of reading and the ongoing discussion about the processes involved in it. The section ends up with the pros and cons of using LCD and CRT displays in eye-tracking studies.

Chapter 2. Eye-tracking research of reading mechanisms: processing of letters, words, sen- tences and texts

In Сhapter 2 we describe how verbal information is processed at different linguistic levels, from recognizing single letters to reading and comprehension of coherent texts. We present the results of several experimental studies on reading in Russian which has specific features like Cyrillic script, rich morphology and flexible word order. First, we show some features of Cyrillic letters recognition of different font types in the experiment with invisible boundary. Our results reveal that the font type affects the recognition of crowed letters (letters in Courier New were harder to identify than the ones in Georgia), while recognition efficiency of isolat- ed letters remains at the same level. Since crowded letters imitate real reading, we claim that Georgia is more readable font than Courier New. Second, we describe the lexical, syntactic and referential ambiguity processing emphasizing the role of semantic context. Thus, we show that the processing of ambiguous words does not depend on the type of their meaning (literal or non-literal) …, and the referential ambiguity advantage effect. Third, we compare the process- ing of literal and non-literal expressions in Russian. We try to tease apart different approaches to idioms as well as to give a better explanation of what units may be stored in the mental lex- icon and how syntactic processing may proceed. Finally, we demonstrate the influence of the text type, functional style and reading skills on text processing. We show that the text type is among the readability categories and it influences the effect of reading perspective: eye-track- ing parameters of reading a static text (descriptive sentences) and a dynamic text (sequence of events following swiftly on one another) differ a lot.

Chapter 3. Where do we look, what do we see, what do we talk about

In Сhapter 3 we compare how verbal and non-verbal visual information is processed. The questions we addresses are: How do the readers integrate text-figure information when reading and understanding verbal and non-verbal patterns, namely one and the same text in verbal for- mat and infographics? How the way humans perceive visual information determines the way they express it in natural language? How the verbalization affects the oculomotor behavior in visual processing? Our results support the assumption of the Cognitive Theory of Multimedia Learning that integration of verbal and pictural information with each other (a polycode text) helps the learners to understand and memorize the text and makes the comprehension easier. We demonstrate the advantages and disadvantages of the infographics (graphical visual repre- sentations of complex information) and verbal text. Also we discuss the relationship between visual processing of images and their verbalization. On one hand, the characteristics of eye movements when looking at the image determine its subsequent verbal description: the more fixations are made and the longer the gaze is directed to the certain area of the image, the more words are dedicated to this area in the following description. On the other hand, verbalization of the previously seen image affects the parameters of eye movements when re-viewing the same image, resulting with the appearance of the ambient processing pattern (short fixations and long saccades), while the re-viewing without verbalization results with the focal processing pattern (longer fixations and shorter saccades). The results obtained open up prospects for fur- ther research on visual perception and can also be used for computer vision models.