scholarly journals Lexical processing in sign language comprehension and production – experimental perspectives

2019 ◽  
Author(s):  
Eva Gutierrez-Sigut ◽  
Cristina Baus
Keyword(s):  
Sign Language ◽  
Language Processing ◽  
Language Comprehension ◽  
Lexical Processing ◽  
Spatial Information ◽  
Field Research ◽  
Cognitive Mechanisms ◽  
Comprehensive Overview ◽  
Natural Languages ◽  
Universal Language

The study of sign language has received increasing interest in the last decades. Within this growing field, research on sign language processing – including both comprehension and production – has also received a remarkable boost in recent years. At initial stages of research, efforts were concentrated on demonstrating universal aspects of language processing; thus, little attention was paid to the differences between modalities or to the specific aspects of the sign-modality. However, the wide recognition of sign languages as natural languages has supported a greater interest in furthering our understanding of modality specific factors (e.g., the use of proprioceptive and spatial information for phonological encoding or the greater potential for iconicity). This chapter offers a comprehensive overview of the most relevant studies of sign language comprehension and production that focus on the lexical level of processing. Results from behavioural studies, as well as evidence of similar neural substrates underlying speech and sign processing, have led to the widely accepted assumption that universal language processing principles can explain lexical access in both signed and spoken languages. However, although psycholinguistic and cognitive mechanisms as well as neural networks underlying speech and sign processing are strikingly similar, they are not identical. We propose that the study of the differences in processing of speech and signs can lead to a more complete picture of human language processing. Acknowledging these differences can also point researchers to factors influencing spoken language processing that might have been under-researched so far.

scholarly journals Hand and Mouth: Cortical Correlates of Lexical Processing in British Sign Language and Speechreading English

2008 ◽  
Vol 20 (7) ◽  
pp. 1220-1234 ◽  
Author(s):  
Cheryl M. Capek ◽  
Dafydd Waters ◽  
Bencie Woll ◽  
Mairéad MacSweeney ◽  
Michael J. Brammer ◽  
...  
Keyword(s):  
Sign Language ◽  
Language Processing ◽  
Lexical Processing ◽  
Vocal Tract ◽  
Temporal Cortex ◽  
Brain Network ◽  
British Sign Language ◽  
Signed Language ◽  
Native Signers ◽  
British Sign

Spoken languages use one set of articulators—the vocal tract, whereas signed languages use multiple articulators, including both manual and facial actions. How sensitive are the cortical circuits for language processing to the particular articulators that are observed? This question can only be addressed with participants who use both speech and a signed language. In this study, we used functional magnetic resonance imaging to compare the processing of speechreading and sign processing in deaf native signers of British Sign Language (BSL) who were also proficient speechreaders. The following questions were addressed: To what extent do these different language types rely on a common brain network? To what extent do the patterns of activation differ? How are these networks affected by the articulators that languages use? Common peri-sylvian regions were activated both for speechreading English words and for BSL signs. Distinctive activation was also observed reflecting the language form. Speechreading elicited greater activation in the left mid-superior temporal cortex than BSL, whereas BSL processing generated greater activation at the temporo-parieto-occipital junction in both hemispheres. We probed this distinction further within BSL, where manual signs can be accompanied by different types of mouth action. BSL signs with speech-like mouth actions showed greater superior temporal activation, whereas signs made with non-speech-like mouth actions showed more activation in posterior and inferior temporal regions. Distinct regions within the temporal cortex are not only differentially sensitive to perception of the distinctive articulators for speech and for sign but also show sensitivity to the different articulators within the (signed) language.

scholarly journals Enabling the Disabled using Machine Learning and Natural Language Processing

2021 ◽  
pp. 172-179
Author(s):  
Nafisa Mapari ◽  
Abdullah Shaikh ◽  
Atik Shaikh ◽  
Zaid Siddiqui
Keyword(s):  
Machine Learning ◽  
Natural Language ◽  
Sign Language ◽  
Language Processing ◽  
Body Language ◽  
Natural Languages ◽  
Social Lives ◽  
Communication Method ◽  
The Disabled ◽  
Head Gestures

Humans communicate with each other through natural language channels such as words and writing, or through body language (gestures) such as hand and head gestures, facial expression, lip motion, etc. There are some examples of natural languages that people use to communicate with each other. We all know that understanding natural language is essential, learning sign language is also very important. For disable people, sign language is the primary communication method for hearing. As there is no translator for communicating among them, so they face problems in communicating. So this is the motivation for us to create a system that recognizes sign language to impact deaf people's social lives significantly.

The Opposition of Surprisal and Semantic Similarity in the Prediction of Language Processing: Evidence from Eye-tracking Data

2020 ◽  
Author(s):  
Kun Sun
Keyword(s):  
Eye Tracking ◽  
Semantic Similarity ◽  
Cognitive Processing ◽  
Language Processing ◽  
Language Comprehension ◽  
Word Processing ◽  
Reading Time ◽  
Computational Models ◽  
Tracking Data ◽  
Dynamic Approach

Expectations or predictions about upcoming content play an important role during language comprehension and processing. One important aspect of recent studies of language comprehension and processing concerns the estimation of the upcoming words in a sentence or discourse. Many studies have used eye-tracking data to explore computational and cognitive models for contextual word predictions and word processing. Eye-tracking data has previously been widely explored with a view to investigating the factors that influence word prediction. However, these studies are problematic on several levels, including the stimuli, corpora, statistical tools they applied. Although various computational models have been proposed for simulating contextual word predictions, past studies usually preferred to use a single computational model. The disadvantage of this is that it often cannot give an adequate account of cognitive processing in language comprehension. To avoid these problems, this study draws upon a massive natural and coherent discourse as stimuli in collecting the data on reading time. This study trains two state-of-art computational models (surprisal and semantic (dis)similarity from word vectors by linear discriminative learning (LDL)), measuring knowledge of both the syntagmatic and paradigmatic structure of language. We develop a `dynamic approach' to compute semantic (dis)similarity. It is the first time that these two computational models have been merged. Models are evaluated using advanced statistical methods. Meanwhile, in order to test the efficiency of our approach, one recently developed cosine method of computing semantic (dis)similarity based on word vectors data adopted is used to compare with our `dynamic' approach. The two computational and fixed-effect statistical models can be used to cross-verify the findings, thus ensuring that the result is reliable. All results support that surprisal and semantic similarity are opposed in the prediction of the reading time of words although both can make good predictions. Additionally, our `dynamic' approach performs better than the popular cosine method. The findings of this study are therefore of significance with regard to acquiring a better understanding how humans process words in a real-world context and how they make predictions in language cognition and processing.

Neural Mechanisms of Music and Language

2019 ◽  
pp. 906-952
Author(s):  
Mattson Ogg ◽  
L. Robert Slevc
Keyword(s):  
Auditory Cortex ◽  
Language Processing ◽  
Auditory Processing ◽  
Cortical Plasticity ◽  
Speaker Identification ◽  
Auditory Pathway ◽  
Cognitive Mechanisms ◽  
Cross Domain ◽  
Music And Language ◽  
Cortical Regions

Music and language are uniquely human forms of communication. What neural structures facilitate these abilities? This chapter conducts a review of music and language processing that follows these acoustic signals as they ascend the auditory pathway from the brainstem to auditory cortex and on to more specialized cortical regions. Acoustic, neural, and cognitive mechanisms are identified where processing demands from both domains might overlap, with an eye to examples of experience-dependent cortical plasticity, which are taken as strong evidence for common neural substrates. Following an introduction describing how understanding musical processing informs linguistic or auditory processing more generally, findings regarding the major components (and parallels) of music and language research are reviewed: pitch perception, syntax and harmonic structural processing, semantics, timbre and speaker identification, attending in auditory scenes, and rhythm. Overall, the strongest evidence that currently exists for neural overlap (and cross-domain, experience-dependent plasticity) is in the brainstem, followed by auditory cortex, with evidence and the potential for overlap becoming less apparent as the mechanisms involved in music and speech perception become more specialized and distinct at higher levels of processing.

scholarly journals Listener’s personality traits predict changes in pupil size during auditory language comprehension

2021 ◽  
Vol 11 (1) ◽  
Author(s):  
Isabell Hubert Lyall ◽  
Juhani Järvikivi
Keyword(s):  
Personality Traits ◽  
Language Processing ◽  
Language Comprehension ◽  
Pupil Size ◽  
Pupil Dilation ◽  
Systematic Research ◽  
Cognitive Resources ◽  
Personality Profiles ◽  
Pupillary Responses ◽  
Grammatical Errors

AbstractResearch suggests that listeners’ comprehension of spoken language is concurrently affected by linguistic and non-linguistic factors, including individual difference factors. However, there is no systematic research on whether general personality traits affect language processing. We correlated 88 native English-speaking participants’ Big-5 traits with their pupillary responses to spoken sentences that included grammatical errors, "He frequently have burgers for dinner"; semantic anomalies, "Dogs sometimes chase teas"; and statements incongruent with gender stereotyped expectations, such as "I sometimes buy my bras at Hudson's Bay", spoken by a male speaker. Generalized additive mixed models showed that the listener's Openness, Extraversion, Agreeableness, and Neuroticism traits modulated resource allocation to the three different types of unexpected stimuli. No personality trait affected changes in pupil size across the board: less open participants showed greater pupil dilation when processing sentences with grammatical errors; and more introverted listeners showed greater pupil dilation in response to both semantic anomalies and socio-cultural clashes. Our study is the first one demonstrating that personality traits systematically modulate listeners’ online language processing. Our results suggest that individuals with different personality profiles exhibit different patterns of the allocation of cognitive resources during real-time language comprehension.

scholarly journals The role of the superior parietal lobule in lexical processing of sign language: Insights from fMRI and TMS

Cortex ◽  
2021 ◽  
Vol 135 ◽  
pp. 240-254
Author(s):  
A. Banaszkiewicz ◽  
Ł. Bola ◽  
J. Matuszewski ◽  
M. Szczepanik ◽  
B. Kossowski ◽  
...  
Keyword(s):  
Sign Language ◽  
Lexical Processing ◽  
Superior Parietal Lobule ◽  
Parietal Lobule

scholarly journals 3D Avatar Approach for Continuous Sign Movement Using Speech/Text

2021 ◽  
Vol 11 (8) ◽  
pp. 3439
Author(s):  
Debashis Das Chakladar ◽  
Pradeep Kumar ◽  
Shubham Mandal ◽  
Partha Pratim Roy ◽  
Masakazu Iwamura ◽  
...  
Keyword(s):  
Language Learning ◽  
Sign Language ◽  
Language Processing ◽  
Hearing Impaired ◽  
Learning System ◽  
English Sentence ◽  
Impaired People ◽  
Indian Sign Language ◽  
Impaired Person ◽  
3D Avatar

Sign language is a visual language for communication used by hearing-impaired people with the help of hand and finger movements. Indian Sign Language (ISL) is a well-developed and standard way of communication for hearing-impaired people living in India. However, other people who use spoken language always face difficulty while communicating with a hearing-impaired person due to lack of sign language knowledge. In this study, we have developed a 3D avatar-based sign language learning system that converts the input speech/text into corresponding sign movements for ISL. The system consists of three modules. Initially, the input speech is converted into an English sentence. Then, that English sentence is converted into the corresponding ISL sentence using the Natural Language Processing (NLP) technique. Finally, the motion of the 3D avatar is defined based on the ISL sentence. The translation module achieves a 10.50 SER (Sign Error Rate) score.

Similarity of Computations Across Domains Does Not Imply Shared Implementation: The Case of Language Comprehension

2021 ◽  
Vol 30 (6) ◽  
pp. 526-534
Author(s):  
Evelina Fedorenko ◽  
Cory Shain
Keyword(s):  
Language Processing ◽  
Language Comprehension ◽  
Fluid Intelligence ◽  
Linguistic Knowledge ◽  
Domain Specific ◽  
Cognitive Operations ◽  
High Level ◽  
General Circuits ◽  
The Mind ◽  
Language Network

Understanding language requires applying cognitive operations (e.g., memory retrieval, prediction, structure building) that are relevant across many cognitive domains to specialized knowledge structures (e.g., a particular language’s lexicon and syntax). Are these computations carried out by domain-general circuits or by circuits that store domain-specific representations? Recent work has characterized the roles in language comprehension of the language network, which is selective for high-level language processing, and the multiple-demand (MD) network, which has been implicated in executive functions and linked to fluid intelligence and thus is a prime candidate for implementing computations that support information processing across domains. The language network responds robustly to diverse aspects of comprehension, but the MD network shows no sensitivity to linguistic variables. We therefore argue that the MD network does not play a core role in language comprehension and that past findings suggesting the contrary are likely due to methodological artifacts. Although future studies may reveal some aspects of language comprehension that require the MD network, evidence to date suggests that those will not be related to core linguistic processes such as lexical access or composition. The finding that the circuits that store linguistic knowledge carry out computations on those representations aligns with general arguments against the separation of memory and computation in the mind and brain.

Mask Region-Based Convolutional Neural Networks (R-CNN) for Sinhala Sign Language to Text Conversion

2021 ◽  
Author(s):  
R. D. Rusiru Sewwantha ◽  
T. N. D. S. Ginige
Keyword(s):  
Natural Language ◽  
Object Detection ◽  
American Sign Language ◽  
Sign Language ◽  
Language Processing ◽  
Sri Lankan ◽  
American Sign ◽  
Disabled People ◽  
Communication Difficulties ◽  
Communication Gap

Sign Language is the use of various gestures and symbols for communication. It is mainly used by disabled people with communication difficulties due to their speech or hearing impediments. Due to the lack of knowledge on sign language, natural language speakers like us, are not able to communicate with such people. As a result, a communication gap is created between sign language users and natural language speakers. It should also be noted that sign language differs from country to country. With American sign language being the most commonly used, in Sri Lanka, we use Sri Lankan/Sinhala sign language. In this research, the authors propose a mobile solution using a Region Based Convolutional Neural Network for object detection to reduce the communication gap between the sign users and language speakers by identifying and interpreting Sinhala sign language to Sinhala text using Natural Language Processing (NLP). The system is able to identify and interpret still gesture signs in real-time using the trained model. The proposed solution uses object detection for the identification of the signs.

Sign in / Sign up

Export Citation Format

Share Document