Speakers construct reference based on listeners’ expected visual search

2020 ◽  
Author(s):  
Julian Jara-Ettinger ◽  
Paula Rubio-Fernandez
Keyword(s):  
Visual Search ◽  
Computational Model ◽  
Referential Communication ◽  
Human Communication ◽  
Quantitative Accuracy ◽  
Utterance Length ◽  
Acceptability Judgments

A foundational assumption of human communication is that speakers ought to say as much as necessary, but no more. How speakers determine what is necessary in a given context, however, is unclear. In studies of referential communication, this expectation is often formalized as the idea that speakers should construct reference by selecting the shortest, sufficiently informative, description. Here we propose that reference production is, instead, a process whereby speakers adopt listeners’ perspectives to facilitate their visual search, without concern for utterance length. We show that a computational model of our proposal predicts graded acceptability judgments with quantitative accuracy, systematically outperforming brevity models. Our model also explains crosslinguistic differences in speakers’ propensity to over-specify in different visual contexts. Our findings suggest that reference production is best understood as driven by a cooperative goal to help the listener understand the intended message, rather than by an egocentric effort to minimize utterance length.

scholarly journals The social basis of referential communication: Speakers construct reference based on listeners' expected visual search.

2020 ◽  
Author(s):  
Julian Jara-Ettinger ◽  
Paula Rubio-Fernandez
Keyword(s):  
Visual Search ◽  
Language Processing ◽  
Referential Communication ◽  
Production Costs ◽  
Alternative View ◽  
Human Communication ◽  
Social Basis ◽  
Quantitative Accuracy ◽  
Referential Expressions ◽  
Acceptability Judgments

A foundational assumption of human communication is that speakers should say as much as necessary, but no more. In referential communication, the pressure to be efficient is typically formalized as an egocentric bias where speakers aim to minimize production costs. While intuitive, this view has failed to explain why people routinely produce redundant adjectives, particularly color words, or why this phenomenon varies cross-linguistically. Here we propose an alternative view of referential efficiency, whereby speakers create referential expressions designed to facilitate the listener's visual search for the referent as they process words in real time. We present a computational model of our account, the Incremental Communicative Efficiency (ICE) model, which generates referential expressions by considering listeners' expected visual search during online language processing. Our model captures a number of known effects in the literature, including cross-linguistic differences in speakers' propensity to over-specify. Moreover, our model predicts graded acceptability judgments with quantitative accuracy, systematically outperforming an alternative, brevity-based model. Our findings suggest that reference production is best understood as driven by a cooperative goal to help the listener identify the intended referent, rather than by an egocentric effort to minimize utterance length.

scholarly journals A computational model for task inference in visual search

2013 ◽  
Vol 13 (3) ◽  
pp. 29-29 ◽  
Author(s):  
A. Haji-Abolhassani ◽  
J. J. Clark
Keyword(s):  
Visual Search ◽  
Computational Model

scholarly journals Real-time pragmatic inference across cultures: evidence from a non-industrialized society

2021 ◽  
Author(s):  
Rachel Anna Ryskin ◽  
Miguel Angel Salinas ◽  
Steven T. Piantadosi ◽  
Edward Gibson
Keyword(s):  
Real Time ◽  
Formal Education ◽  
Eye Gaze ◽  
Visual Display ◽  
Referential Communication ◽  
Human Communication ◽  
Pragmatic Inference ◽  
Communication Task ◽  
Patterns Of Behavior ◽  
Referential Expressions

Speakers and listeners are thought to routinely make sophisticated inferences, in real time, about their conversation partner’s knowledge state and communicative intentions. However, these inferences have only been studied in industrialized cultures. Communicative expectations may be language-dependent, as are many phonological, syntactic, and semantic aspects of language. We study pragmatic inference in communication in the Tsimane’, an indigenous people of the Bolivian Amazon, who have little contact with industrialization or formal education. Using a referential communication task and eye-tracking, we probe how Tsimane' speakers use and understand referential expressions (e.g., ``Hand me the cup.'') across contexts. We manipulated aspects of the visual display to elicit contrastive inferences, including whether the referent was unique or part of a set as well as whether members of the same set differed in size or color. Strikingly, in all cases, patterns of behavior and eye-gaze of Tsimane' and English speakers were qualitatively identical, suggesting that real-time inference may be a core feature of human communication that is shared across cultures rather than a product of life in an industrialized society.

scholarly journals Accurate Referential Communication and its Relation with Private and Social Speech in a Naturalistic Context

2004 ◽  
Vol 7 (2) ◽  
pp. 81-92 ◽  
Author(s):  
Dolors Girbau ◽  
Humbert Boada
Keyword(s):  
Interpersonal Communication ◽  
Referential Communication ◽  
Private Speech ◽  
Communication Behavior ◽  
Human Communication ◽  
High Skill

Research into human communication has been grouped under two traditions: referential and sociolinguistic. The study of a communication behavior simultaneously from both paradigms appears to be absent. Basically, this paper analyzes the use of private and social speech, through both a referential task (Word Pairs) and a naturalistic dyadic setting (Lego-set) administered to a sample of 64 children from grades 3 and 5. All children, of 8 and 10 years of age, used speech that was not adapted to the decoder, and thus ineffective for interpersonal communication, in both referential and sociolinguistic communication. Pairs of high-skill referential encoders used significantly more task-relevant social speech, that is, cognitively more complex, than did low-skill dyads in the naturalistic context. High-skill referential encoder dyads showed a trend to produce more inaudible private speech than did low-skill ones during spontaneous communication. Gender did not affect the results.

scholarly journals Before L1

Gesture ◽  
2008 ◽  
Vol 8 (2) ◽  
pp. 180-196 ◽  
Author(s):  
Ulf Liszkowski
Keyword(s):  
Cognitive Skills ◽  
Social Cognitive ◽  
Referential Communication ◽  
Human Communication ◽  
Symbolic Communication ◽  
The Social ◽  
Social Acts ◽  
Communicative Act ◽  
Socially Situated ◽  
Representational Gestures

This paper investigates the social-cognitive and motivational complexities underlying prelinguistic infants’ gestural communication. With regard to deictic referential gestures, new and recent experimental evidence shows that infant pointing is a complex communicative act based on social-cognitive skills and cooperative motives. With regard to infant representational gestures, findings suggest the need to re-interpret these gestures as initially non-symbolic gestural social acts. Based on the available empirical evidence, the paper argues that deictic referential communication emerges as a foundation of human communication first in gestures, already before language. Representational symbolic communication, instead, emerges as a transformation of deictic communication first in the vocal modality and, perhaps, in gestures through non-symbolic, socially situated routines.

scholarly journals Laughter and smiling facial expression modelling for the generation of virtual affective behavior

PLoS ONE ◽  
2021 ◽  
Vol 16 (5) ◽  
pp. e0251057
Author(s):  
Miquel Mascaró ◽  
Francisco J. Serón ◽  
Francisco J. Perales ◽  
Javier Varona ◽  
Ramon Mas
Keyword(s):  
Facial Expression ◽  
Computational Model ◽  
Facial Expressions ◽  
User Study ◽  
New Method ◽  
Affective Behavior ◽  
Human Communication ◽  
Virtual Character ◽  
Different Types ◽  
Proprietary Database

Laughter and smiling are significant facial expressions used in human to human communication. We present a computational model for the generation of facial expressions associated with laughter and smiling in order to facilitate the synthesis of such facial expressions in virtual characters. In addition, a new method to reproduce these types of laughter is proposed and validated using databases of generic and specific facial smile expressions. In particular, a proprietary database of laugh and smile expressions is also presented. This database lists the different types of classified and generated laughs presented in this work. The generated expressions are validated through a user study with 71 subjects, which concluded that the virtual character expressions built using the presented model are perceptually acceptable in quality and facial expression fidelity. Finally, for generalization purposes, an additional analysis shows that the results are independent of the type of virtual character’s appearance.

scholarly journals Predicting Search Performance in Heterogeneous Scenes: Quantifying the Impact of Homogeneity Effects in Efficient Search

2019 ◽  
Vol 5 (1) ◽  
Author(s):  
Alejandro Lleras ◽  
Zhiyuan Wang ◽  
Anna Madison ◽  
Simona Buetti
Keyword(s):  
Visual Search ◽  
Computational Model ◽  
Real World ◽  
Predictive Power ◽  
Facilitatory Effect ◽  
Stimulus Complexity ◽  
Search Performance ◽  
Efficient Search ◽  
The Impact

Recently, Wang, Buetti and Lleras (2017) developed an equation to predict search performance in heterogeneous visual search scenes (i.e., multiple types of non-target objects simultaneously present) based on parameters observed when participants perform search in homogeneous scenes (i.e., when all non-target objects are identical to one another). The equation was based on a computational model where every item in the display is processed with unlimited capacity and independently of one another, with the goal of determining whether the item is likely to be a target or not. The model was tested in two experiments using real-world objects. Here, we extend those findings by testing the predictive power of the equation to simpler objects. Further, we compare the model’s performance under two stimulus arrangements: spatially-intermixed (items randomly placed around the scene) and spatially-segregated displays (identical items presented near each other). This comparison allowed us to isolate and quantify the facilitatory effect of processing displays that contain identical items (homogeneity facilitation), a factor that improves performance in visual search above-and-beyond target-distractor dissimilarity. The results suggest that homogeneity facilitation effects in search arise from local item-to-item interaction (rather than by rejecting items as “groups”) and that the strength of those interactions might be determined by stimulus complexity (with simpler stimuli producing stronger interactions and thus, stronger homogeneity facilitation effects).

Sign in / Sign up

Export Citation Format

Share Document