From improvisation to learning: how naturalness and systematicity shape language evolution

Silent gesture studies, in which hearing participants from different linguistic backgrounds produce gestures to communicate events, have been used to test hypotheses about the cognitive biases that govern cross-linguistic word order preferences. In particular, the differential use of SOV and SVO order to communicate, respectively, extensional events (where the direct object exists independently of the event; e.g., girl throws ball) and intensional events (where the meaning of the direct object is potentially dependent on the verb; e.g., girl thinks of ball), has been suggested to represent a natural preference, demonstrated in improvisation contexts. However, natural languages tend to prefer systematic word orders, where a single order is used regardless of the event being communicated. We present a series of studies that investigate ordering preferences for SOV and SVO orders using an online forced-choice experiment, where participants select orders for different events i) in the absence of conventions and ii) after learning event-order mappings in different frequencies in a regularisation experiment. Our results show that natural ordering preferences arise in the absence of conventions, replicating previous findings from production experiments. In addition, we show that participants regularise the input they learn in the manual modality in two ways, such that, while the preference for systematic order patterns increases through learning, it exists in competition with the natural ordering preference, that conditions order on the semantics of the event. Using our experimental data in a computational model of cultural transmission, we show that this pattern is expected to persist over generations, suggesting that we should expect to see evidence of semantically-conditioned word order variability in at least some languages.

Die Geste der Kunst

Die Geste wurde von der römischen Rhetorik über die Physiognomik bis hin zu den Gesture Studies vordergründig als eine funktionale körperliche Stütze für sprachliche Kommunikation oder als sinnliche Spur geistiger Zustände aufgefasst. Gesten begleiten unsere Diskurse und Gemütslagen und machen diese wahrnehmbar. Über diesen traditionell gültigen Standpunkt hinaus verlagert Luca Viglialoro den Fokus auf den somatisch-medialen Kunstcharakter der Geste, um diese als prozesshafte Technik (ars) für die Modellierung der aísthesis zu konturieren. Dafür werden die komplexen Relationen zwischen Gesten und Kunstreflexionen am Beispiel von medial unterschiedlichen Konfigurationen (von der Schrift und der Zeichnung bis zur Fotografie und Installationskunst) analysiert.

Kinesic Humor

Literature is one of the richest sources of information concerning the ways in which human beings are able to play with cognition. According to the theory of embodied cognition, human cognition is grounded in sensorimotricity, i.e., the ability to feel, perceive, and move. The pervading cognitive process called perceptual simulation, which is activated when we cognitively process a gesture in a real-life situation, is also recruited when we read about actions, movements, and gestures in texts. Kinesic Humor examines literary works written by major authors—including Chrétien de Troyes, Cervantes, Milton, Saint-Simon, Rousseau, Sterne, and Stendhal—in which perceptual simulations of complex sensorimotor events and kinesic interactions trigger humorous effects. Such works create anticipations regarding movements and sensations, which are unexpectedly thwarted, thus producing cognitive shifts typical of humor. By bringing together literary studies, cognitive studies, gesture studies, and humor studies, this book offers original perspectives on such important artworks as Paradise Lost, Don Quixote, and Le Rouge et le Noir. In it, the importance of rhythm and tonicity in the perception of movements and gestures is a focus of attention. The interactional significance of gestures often lies in their dynamics, and this fact also applies to the cognitive retrieval of narrated gestures during the act of reading. The method of kinesic analysis practiced in this book takes into account such cognitive features in correlation with the historical and cultural contexts in which the literary works were written.

Controlling Video Stimuli in Sign Language and Gesture Research: The OpenPoseR Package for Analyzing OpenPose Motion-Tracking Data in R

Researchers in the fields of sign language and gesture studies frequently present their participants with video stimuli showing actors performing linguistic signs or co-speech gestures. Up to now, such video stimuli have been mostly controlled only for some of the technical aspects of the video material (e.g., duration of clips, encoding, framerate, etc.), leaving open the possibility that systematic differences in video stimulus materials may be concealed in the actual motion properties of the actor’s movements. Computer vision methods such as OpenPose enable the fitting of body-pose models to the consecutive frames of a video clip and thereby make it possible to recover the movements performed by the actor in a particular video clip without the use of a point-based or markerless motion-tracking system during recording. The OpenPoseR package provides a straightforward and reproducible way of working with these body-pose model data extracted from video clips using OpenPose, allowing researchers in the fields of sign language and gesture studies to quantify the amount of motion (velocity and acceleration) pertaining only to the movements performed by the actor in a video clip. These quantitative measures can be used for controlling differences in the movements of an actor in stimulus video clips or, for example, between different conditions of an experiment. In addition, the package also provides a set of functions for generating plots for data visualization, as well as an easy-to-use way of automatically extracting metadata (e.g., duration, framerate, etc.) from large sets of video files.

Evolução da Linguagem e Ordem Natural de Palavras os verbos reversíveis e não reversíveis da Língua Gestual de São Tomé e Príncipe num estudo piloto

search on language evolution has recently focused on the issue of natural word order, that is, word order in the phylogenetic and cognitive sense (Pagel 2009; Gell-Mann and Ruhlen 2011). Sign language and gesture studies have inspired this discussion in the literature, with special emphasis on the seminal study by Goldin-Meadow and colleagues (2008). The results of this study revealed that participants tend to produce SVO and SOV word order, regardless of the syntax of their native language. This finding has been corroborated in later studies (Gibson et al. 2013; Hall et al. 2013; Sandler et al. 2005). Our study aims to verify if there is dominant word order, or not, in linguistic emergence of Sign Language of São Tomé and Príncipe.

Temporality, social interaction, and power in an anthropology of gesture

Current anthropological studies of gesture give extensive attention to communities of study from a synchronic perspective while also focusing on semantic, cognitive, and linguistic analyses of gesture. However, less well explored is how the uses and meanings of gestures can change over time within societies and the role of gesture in social interactions. In addition, individual, interpersonal, and societal level politics can also influence what gestures mean and how they are strategically used. This paper uses careful analysis of European missionary reports and trader accounts written in the late 17th and early 18th centuries to focus on shifting power relations in the pre-colonial era Kongo Kingdom in West Central Africa. Larger social transformations will be used to contextualize three key incidents where gestures were at the center of complex negotiations about meaning and power. The paper argues for gesture studies scholars to consider deep, contextual, and historically grounded examinations of gestures and the role they play in shaping relationships and societies.

What is an anthropology of gesture?

For gesture research outside anthropology, the promise – and challenge – of anthropological method stems from one or more of its core commitments: its pursuit of human variation, both diachronic and synchronic; its insistence on naturalistic rather than experimental research design; and its integrative sensibility that situates human behavior in relation to an expansive sociocultural context. This essay reflects on this last sensibility. As we envision an anthropology of gesture and weigh its potential for gesture studies, we should pause and reflect on the fitful history of gesture in anthropology. As a parable for the present, I revisit a neglected anthropological voice from twentieth-century gesture research: Ray L. Birdwhistell, whose ambitious postwar science of kinesics teamed film-based microanalysis with American linguistic structuralism. At stake in Birdwhistell’s work was a problem that looms large here, that of how and at what cost a science of gesture can contextualize its object integratively.

Using Gesture to Facilitate L2 Phoneme Acquisition: The Importance of Gesture and Phoneme Complexity

Most language learners have difficulties acquiring the phonemes of a second language (L2). Unfortunately, they are often judged on their L2 pronunciation, and segmental inaccuracies contribute to miscommunication. Therefore, we aim to determine how to facilitate phoneme acquisition. Given the close relationship between speech and co-speech gesture, previous work unsurprisingly reports that gestures can benefit language acquisition, e.g., in (L2) word learning. However, gesture studies on L2 phoneme acquisition present contradictory results, implying that both specific properties of gestures and phonemes used in training, and their combination, may be relevant. We investigated the effect of phoneme and gesture complexity on L2 phoneme acquisition. In a production study, Dutch natives received instruction on the pronunciation of two Spanish phonemes, /u/ and /θ/. Both are typically difficult to produce for Dutch natives because their orthographic representation differs between both languages. Moreover, /θ/ is considered more complex than /u/, since the Dutch phoneme inventory contains /u/ but not /θ/. The instruction participants received contained Spanish examples presented either via audio-only, audio-visually without gesture, audio-visually with a simple, pointing gesture, or audio-visually with a more complex, iconic gesture representing the relevant speech articulator(s). Preceding and following training, participants read aloud Spanish sentences containing the target phonemes. In a perception study, Spanish natives rated the target words from the production study on accentedness and comprehensibility. Our results show that combining gesture and speech in L2 phoneme training can lead to significant improvement in L2 phoneme production, but both gesture and phoneme complexity affect successful learning: Significant learning only occurred for the less complex phoneme /u/ after seeing the more complex iconic gesture, whereas for the more complex phoneme /θ/, seeing the more complex gesture actually hindered acquisition. The perception results confirm the production findings and show that items containing /θ/ produced after receiving training with a less complex pointing gesture are considered less foreign-accented and more easily comprehensible as compared to the same items after audio-only training. This shows that gesture can facilitate task performance in L2 phonology acquisition, yet complexity affects whether certain gestures work better for certain phonemes than others.

Controlling video stimuli in sign language and gesture research: The OpenPoseR package for analyzing OpenPose motion tracking data in R

Researchers in the fields of sign language and gesture studies frequently present their participants with video stimuli showing actors performing linguistic signs or co-speech gestures. Up to now, such video stimuli have been mostly controlled only for some of the technical aspects of the video material (e.g., duration of clips, encoding, framerate, etc.), leaving open the possibility that systematic differences in video stimulus materials may be concealed in the actual motion properties of the actor’s movements. Computer vision methods such as OpenPose enable the fitting of body-pose models to the consecutive frames of a video clip and thereby make it possible to recover the movements performed by the actor in a particular video clip without the use of a point-based or markerless motion-tracking system during recording. The OpenPoseR package provides a straightforward and reproducible way of working with these body-pose model data extracted from video clips using OpenPose, allowing researchers in the fields of sign language and gesture studies to quantify the amount of motion (velocity and acceleration) pertaining only to the movements performed by the actor in a video clip. These quantitative measures can be used for controlling differences in the movements of an actor in stimulus video clips or, for example, between different conditions of an experiment. In addition, the package also provides a set of functions for generating plots for data visualization, as well as an easy-to-use way of automatically extracting metadata (e.g., duration, framerate, etc.) from large sets of video files.