scholarly journals Role of Gesture in Language Processing

2018 ◽  
pp. 591-607
Author(s):  
Asli Özyürek
Keyword(s):  
Language Processing ◽  
Relevant Information ◽  
Hand Gestures ◽  
Face To Face ◽  
Syntactic Information ◽  
Different Levels

Use of language in face-to-face context is multimodal. Production and perception of speech take place in the context of visual articulators such as lips, face, or hand gestures which convey relevant information to what is expressed in speech at different levels of language. While lips convey information at the phonological level, gestures contribute to semantic, pragmatic, and syntactic information, as well as to discourse cohesion. This chapter overviews recent findings showing that speech and gesture (e.g. a drinking gesture as someone says, “Would you like a drink?”) interact during production and comprehension of language at the behavioral, cognitive, and neural levels. Implications of these findings for current psycholinguistic theories and how they can be expanded to consider the multimodal context of language processing are discussed.

Testing the automaticity of syntax using masked visual priming

2021 ◽  
Author(s):  
Elena Pyatigorskaya ◽  
Matteo Maran ◽  
Emiliano Zaccarella
Keyword(s):  
Language Processing ◽  
Language Comprehension ◽  
Syntactic Priming ◽  
Visual Words ◽  
Visual Priming ◽  
Priming Paradigm ◽  
Syntactic Information ◽  
Fast Pace ◽  
Behavioural Experiments

Language comprehension proceeds at a very fast pace. It is argued that context influences the speed of language comprehension by providing informative cues for the correct processing of the incoming linguistic input. Priming studies investigating the role of context in language processing have shown that humans quickly recognise target words that share orthographic, morphological, or semantic information with their preceding primes. How syntactic information influences the processing of incoming words is however less known. Early syntactic priming studies reported faster recognition for noun and verb targets (e.g., apple or sing) following primes with which they form grammatical phrases or sentences (the apple, he sings). The studies however leave open a number of questions about the reported effect, including the degree of automaticity of syntactic priming, the facilitative versus inhibitory nature, and the specific mechanism underlying the priming effect—that is, the type of syntactic information primed on the target word. Here we employed a masked syntactic priming paradigm in four behavioural experiments in German language to test whether masked primes automatically facilitate the categorization of nouns and verbs presented as flashing visual words. Overall, we found robust syntactic priming effects with masked primes—thus suggesting high automaticity of the process—but only when verbs were morpho-syntactically marked (er kau-t; he chew-s). Furthermore, we found that, compared to baseline, primes slow down target categorisation when the relationship between prime and target is syntactically incorrect, rather than speeding it up when the prime-target relationship is syntactically correct. This argues in favour of an inhibitory nature of syntactic priming. Overall, the data indicate that humans automatically extract abstract syntactic features from word categories as flashing visual words, which has an impact on the speed of successful language processing during language comprehension.

scholarly journals Title-Guided Encoding for Keyphrase Generation

2019 ◽  
Vol 33 ◽  
pp. 6268-6275 ◽  
Author(s):  
Wang Chen ◽  
Yifan Gao ◽  
Jiani Zhang ◽  
Irwin King ◽  
Michael R. Lyu
Keyword(s):  
Language Processing ◽  
Deep Neural Networks ◽  
State Of The Art ◽  
Relevant Information ◽  
The State ◽  
Main Body ◽  
Generation Task ◽  
Leading Role ◽  
Very High

Keyphrase generation (KG) aims to generate a set of keyphrases given a document, which is a fundamental task in natural language processing (NLP). Most previous methods solve this problem in an extractive manner, while recently, several attempts are made under the generative setting using deep neural networks. However, the state-of-the-art generative methods simply treat the document title and the document main body equally, ignoring the leading role of the title to the overall document. To solve this problem, we introduce a new model called Title-Guided Network (TG-Net) for automatic keyphrase generation task based on the encoderdecoder architecture with two new features: (i) the title is additionally employed as a query-like input, and (ii) a titleguided encoder gathers the relevant information from the title to each word in the document. Experiments on a range of KG datasets demonstrate that our model outperforms the state-of-the-art models with a large margin, especially for documents with either very low or very high title length ratios.

scholarly journals Top-down information is more important in noisy situations: Exploring the role of pragmatic, semantic, and syntactic information in language processing

2019 ◽  
Author(s):  
Fabio Trecca ◽  
Kristian Tylén ◽  
Riccardo Fusaroli ◽  
Christer Johansson ◽  
Morten H. Christiansen
Keyword(s):  
Language Processing ◽  
Spoken Language ◽  
Top Down ◽  
Speech Input ◽  
Processing Strategies ◽  
Syntactic Information ◽  
Noisy Conditions ◽  
Relative Weighting ◽  
Different Sources

Language processing depends on the integration of bottom-up information with top-down cues from several different sources—primarily our knowledge of the real world, of discourse contexts, and of how language works. Previous studies have shown that factors pertaining to both the sender and the receiver of the message affect the relative weighting of such information. Here, we suggest another factor that may change our processing strategies: perceptual noise in the environment. We hypothesize that listeners weight different sources of top-down information more in situations of perceptual noise than in noise-free situations. Using a sentence-picture matching experiment with four forced-choice alternatives, we show that degrading the speech input with noise compels the listeners to rely more on top-down information in processing. We discuss our results in light of previous findings in the literature, highlighting the need for a unified model of spoken language comprehension in different ecologically valid situations, including under noisy conditions.

scholarly journals The predictive potential of hand gestures during conversation: An investigation of the timing of gestures in relation to speech

2020 ◽  
Author(s):  
Marlijn ter Bekke ◽  
Linda Drijvers ◽  
JUDITH HOLLER
Keyword(s):  
Response Time ◽  
Language Processing ◽  
Semantic Information ◽  
Hand Gestures ◽  
Lexical Information ◽  
Face To Face ◽  
Movement Component ◽  
Natural Conversation ◽  
Representational Gestures ◽  
Predictive Language Processing

In face-to-face conversation, recipients might use the bodily movements of the speaker (e.g. gestures) to facilitate language processing. It has been suggested that one way through which this facilitation may happen is prediction. However, for this to be possible, gestures would need to precede speech, and it is unclear whether this is true during natural conversation. In a corpus of Dutch conversations, we annotated hand gestures that represent semantic information and occurred during questions, and the word(s) which corresponded most closely to the gesturally depicted meaning. Thus, we tested whether representational gestures temporally precede their lexical affiliates. Further, to see whether preceding gestures may indeed facilitate language processing, we asked whether the gesture-speech asynchrony predicts the response time to the question the gesture is part of. Gestures and their strokes (most meaningful movement component) indeed preceded the corresponding lexical information, thus demonstrating their predictive potential. However, while questions with gestures got faster responses than questions without, there was no evidence that questions with larger gesture-speech asynchronies get faster responses. These results suggest that gestures indeed have the potential to facilitate predictive language processing, but further analyses on larger datasets are needed to test for links between asynchrony and processing advantages.

scholarly journals The Role of Synchrony and Ambiguity in Speech–Gesture Integration during Comprehension

2011 ◽  
Vol 23 (8) ◽  
pp. 1845-1854 ◽  
Author(s):  
Boukje Habets ◽  
Sotaro Kita ◽  
Zeshu Shao ◽  
Asli Özyurek ◽  
Peter Hagoort
Keyword(s):  
Semantic Integration ◽  
Time Span ◽  
Hand Movements ◽  
Hand Gestures ◽  
Face To Face ◽  
Significant Difference ◽  
Speech Segments

During face-to-face communication, one does not only hear speech but also see a speaker's communicative hand movements. It has been shown that such hand gestures play an important role in communication where the two modalities influence each other's interpretation. A gesture typically temporally overlaps with coexpressive speech, but the gesture is often initiated before (but not after) the coexpressive speech. The present ERP study investigated what degree of asynchrony in the speech and gesture onsets are optimal for semantic integration of the concurrent gesture and speech. Videos of a person gesturing were combined with speech segments that were either semantically congruent or incongruent with the gesture. Although gesture and speech always overlapped in time, gesture and speech were presented with three different degrees of asynchrony. In the SOA 0 condition, the gesture onset and the speech onset were simultaneous. In the SOA 160 and 360 conditions, speech was delayed by 160 and 360 msec, respectively. ERPs time locked to speech onset showed a significant difference between semantically congruent versus incongruent gesture–speech combinations on the N400 for the SOA 0 and 160 conditions. No significant difference was found for the SOA 360 condition. These results imply that speech and gesture are integrated most efficiently when the differences in onsets do not exceed a certain time span because of the fact that iconic gestures need speech to be disambiguated in a way relevant to the speech context.

THE ROLE OF GESTURE IN DOCUMENT EXPLANATION BY EMBODIED CONVERSATIONAL AGENTS

2008 ◽  
Vol 02 (01) ◽  
pp. 47-70 ◽  
Author(s):  
TIMOTHY BICKMORE ◽  
LAURA PFEIFER ◽  
LANGXUAN YIN
Keyword(s):  
Domain Knowledge ◽  
Empirical Studies ◽  
Evaluation Study ◽  
Embodied Conversational Agents ◽  
Conversational Agents ◽  
Hand Gestures ◽  
Embodied Conversational Agent ◽  
Face To Face ◽  
Low Levels

We describe two empirical studies of how professionals explain documents to lay clients who have varying levels of knowledge about the domain under discussion. We find that hand gestures, and in particular deictic gestures by the professional at various parts of the document play a major role in explanations of documents with clients in face-to-face settings. We describe a preliminary computational model of document explanation by an embodied conversational agent, in which appropriate form and location of hand gestures are used by the agent in explaining a document to a user. Results from a pilot evaluation study indicate that individuals with low levels of domain knowledge prefer receiving explanations from such an agent rather than from a human. Examples are drawn from the healthcare domain, in which research consent forms and hospital discharge instruction forms are used as the documents being explained, and health literacy is used as the measure of client domain knowledge.

On keeping cool: The role of inhibition in bilingual word processing

2018 ◽  
Vol 22 (04) ◽  
pp. 699-700 ◽  
Author(s):  
MATHIEU DECLERCK ◽  
GABRIELA MEADE ◽  
JONATHAN GRAINGER
Keyword(s):  
Language Processing ◽  
Word Processing ◽  
Data Set ◽  
Lexical Representations ◽  
Target Article ◽  
One Step ◽  
Different Levels ◽  
Language Membership ◽  
Target Data

One of the cool aspects of the original implementation of the BIA model (van Heuven, Dijkstra & Grainger, 1998) was the discovery that inhibitory connections between language nodes and lexical representations was a necessary feature for the model to be able to simulate the target data set at that time. This demonstrates the importance of computational modeling, a key point of the present target article, since inhibitory connections were postulated to occur only between representations at the same level in the conceptual model (Grainger & Dijkstra, 1992). Top-down inhibition was subsequently dropped in the BIA+ model (Dijkstra & van Heuven, 2002), and the Multilink model of the present target article (Dijkstra, Wahl, Buytenhuijs, van Halem, Al-jibouri, de Korte & Rekké, 2018) goes one step further by removing all kinds of inhibitory connections, both between and within levels. Instead, the authors of the model propose that bilingual language processing relies on bidirectional excitatory connections between representations at different levels. This is curious given that even more evidence has accumulated in favor of inhibition since the original implementation of the BIA model, both between neighboring lexical representations (i.e., lateral inhibition) and from language membership representations (e.g., language nodes and tags) down to lexical representations. In this commentary, we focus on whether the exclusion of these two inhibitory processes is warranted, and how the inclusion of these processes might benefit future developments of the model.

scholarly journals The Role of Vocabulary Mediation to Discover and Represent Relevant Information in Privacy Policies

2020 ◽  
Author(s):  
Valentina Leone ◽  
Luigi Di Caro
Keyword(s):  
Language Processing ◽  
Data Protection ◽  
Data Privacy ◽  
Relevant Information ◽  
Privacy Policies ◽  
Domain Specific ◽  
Formal Taxonomy ◽  
Semantic Resources ◽  
Machine Readable

To date, the effort made by existing vocabularies to provide a shared representation of the data protection domain is not fully exploited. Different natural language processing (NLP) techniques have been applied to the text of privacy policies without, however, taking advantage of existing vocabularies to provide those documents with a shared semantic superstructure. In this paper we show how a recently released domain-specific vocabulary, i.e. the Data Privacy Vocabulary (DPV), can be used to discover, in privacy policies, the information that is relevant with respect to the concepts modelled in the vocabulary itself. We also provide a machine-readable representation of this information to bridge the unstructured textual information to the formal taxonomy modelled in it. This is the first approach to the automatic processing of privacy policies that relies on the DPV, fuelling further investigation on the applicability of existing semantic resources to promote the reuse of information and the interoperability between systems in the data protection domain.

Collection of Ego-Centered Network Data with Computer-Assisted Interviews

Methodology ◽  
2006 ◽  
Vol 2 (1) ◽  
pp. 7-15 ◽  
Author(s):  
Joachim Gerich ◽  
Roland Lehner
Keyword(s):  
Survey Methods ◽  
A Priori ◽  
Dynamic Structure ◽  
Computer Assisted ◽  
Network Data ◽  
Great Effort ◽  
Self Administration ◽  
Face To Face ◽  
Full Network

Although ego-centered network data provide information that is limited in various ways as compared with full network data, an ego-centered design can be used without the need for a priori and researcher-defined network borders. Moreover, ego-centered network data can be obtained with traditional survey methods. However, due to the dynamic structure of the questionnaires involved, a great effort is required on the part of either respondents (with self-administration) or interviewers (with face-to-face interviews). As an alternative, we will show the advantages of using CASI (computer-assisted self-administered interview) methods for the collection of ego-centered network data as applied in a study on the role of social networks in substance use among college students.

Value-driven issues throughout the development of sociological theory

2020 ◽  
pp. 147-160
Author(s):  
Gulbarshyn Chepurko ◽  
Valerii Pylypenko
Keyword(s):  
Social Interactions ◽  
Social System ◽  
Sociological Theory ◽  
Social Scientists ◽  
Social Sphere ◽  
Technological Advances ◽  
The Social ◽  
The Individual ◽  
Different Levels

The paper examines and compares how the major sociological theories treat axiological issues. Value-driven topics are analysed in view of their relevance to society in times of crisis, when both societal life and the very structure of society undergo dramatic change. Nowadays, social scientists around the world are also witnessing such a change due to the emergence of alternative schools of sociological thought (non-classical, interpretive, postmodern, etc.) and, subsequently, the necessity to revise the paradigms that have been existed in sociology so far. Since the above-mentioned approaches are often used to address value-related issues, building a solid theoretical framework for these studies takes on considerable significance. Furthermore, the paradigm revision has been prompted by technological advances changing all areas of people’s lives, especially social interactions. The global human community, integral in nature, is being formed, and production of human values now matters more than production of things; hence the “expansion” of value-focused perspectives in contemporary sociology. The authors give special attention to collectivities which are higher-order units of the social system. These units are described as well-organised action systems where each individual performs his/her specific role. Just as the role of an individual is distinct from that of the collectivity (because the individual and the collectivity are different as units), so too a distinction is drawn between the value and the norm — because they represent different levels of social relationships. Values are the main connecting element between the society’s cultural system and the social sphere while norms, for the most part, belong to the social system. Values serve primarily to maintain the pattern according to which the society is functioning at a given time; norms are essential to social integration. Apart from being the means of regulating social processes and relationships, norms embody the “principles” that can be applied beyond a particular social system. The authors underline that it is important for Ukrainian sociology to keep abreast of the latest developments in the field of axiology and make good use of those ideas because this is a prerequisite for its successful integration into the global sociological community.

Sign in / Sign up

Export Citation Format

Share Document