Classification and Synthesis of Emotion in Sign Languages Using Neutral Expression Deviation Factor and 4D Trajectories

AbstractThe use of filters and editing tools for perfecting selfies is increasing. While some aesthetic experts have touted the ability of this technology to help patients convey their aesthetic goals, others have expressed concerns about the unrealistic expectations that may come from the ability for individuals to digitally alter their own photos in these so-called “super-selfies.” The aim of the study is to determine the changes that individuals seek when enhancing selfies. Twenty subjects participated in this study between July 25 and September 24, 2019. Subjects had two sets of headshots taken (neutral and smile) and were provided an introduction on the use of the Facetune2 app. Subjects received a digital copy of their photographs and were asked to download the free mobile app. After 1 week of trialing the different tools for enhancing their appearance, subjects submitted their self-determined most attractive edited photographs. Changes in marginal reflex distance (MRD) 1 and 2, nose height and width, eyebrow height, facial width, skin smoothness, skin hue, and saturation as well as overall image brightness were recorded. Paired two-tailed t-test was used to evaluate pre- and post-facial measurements. There were no statistically significant changes identified in the analysis of the altered photos in neutral expression. Analysis of all smiling photographs revealed that subjects increased their smile angle (right: +2.92 mm, p = 0.04; left: +3.58 mm, p < 0.001). When smiling photographs were assessed by gender, females were found to significantly increase their MRD2 (right: +0.64 mm, p = 0.04; left: +0.74 mm, p = 0.05) and their smile angle (right: +1.90 mm, p = 0.03; left: +2.31 mm, p = 0.005) while also decreasing their nose height (−2.8 mm, p = 0.04). Males did not significantly alter any of the facial measurements assessed. This study identifies the types of changes that individuals seek when enhancing selfies and specifies the different aspects of image adjustment that may be sought based on a patient's gender.

Download Full-text

Crosslinguistic similarity and variation in the simultaneous morphology of sign languages

The Linguistic Review ◽

10.1515/tlr-2020-2055 ◽

2020 ◽

Vol 37 (4) ◽

pp. 571-608

Author(s):

Diane Brentari ◽

Laura Horton ◽

Susan Goldin-Meadow

Keyword(s):

American Sign Language ◽

Sign Language ◽

Optimality Theory ◽

Sign Languages ◽

British Sign Language ◽

Verb Phrases ◽

Classifier Predicates ◽

Crosslinguistic Variation ◽

Hong Kong Sign Language ◽

British Sign

Abstract Two differences between signed and spoken languages that have been widely discussed in the literature are: the degree to which morphology is expressed simultaneously (rather than sequentially), and the degree to which iconicity is used, particularly in predicates of motion and location, often referred to as classifier predicates. In this paper we analyze a set of properties marking agency and number in four sign languages for their crosslinguistic similarities and differences regarding simultaneity and iconicity. Data from American Sign Language (ASL), Italian Sign Language (LIS), British Sign Language (BSL), and Hong Kong Sign Language (HKSL) are analyzed. We find that iconic, cognitive, phonological, and morphological factors contribute to the distribution of these properties. We conduct two analyses—one of verbs and one of verb phrases. The analysis of classifier verbs shows that, as expected, all four languages exhibit many common formal and iconic properties in the expression of agency and number. The analysis of classifier verb phrases (VPs)—particularly, multiple-verb predicates—reveals (a) that it is grammatical in all four languages to express agency and number within a single verb, but also (b) that there is crosslinguistic variation in expressing agency and number across the four languages. We argue that this variation is motivated by how each language prioritizes, or ranks, several constraints. The rankings can be captured in Optimality Theory. Some constraints in this account, such as a constraint to be redundant, are found in all information systems and might be considered non-linguistic; however, the variation in constraint ranking in verb phrases reveals the grammatical and arbitrary nature of linguistic systems.

Download Full-text

Interpretation of Swedish Sign Language Using Convolutional Neural Networks and Transfer Learning

SN Computer Science ◽

10.1007/s42979-021-00612-w ◽

2021 ◽

Vol 2 (3) ◽

Author(s):

Gustaf Halvardsson ◽

Johanna Peterson ◽

César Soto-Valero ◽

Benoit Baudry

Keyword(s):

Neural Networks ◽

Sign Language ◽

Transfer Learning ◽

Convolutional Neural Networks ◽

Web Application ◽

Training Dataset ◽

Motion Processing ◽

Image Perception ◽

Sign Languages ◽

High Level

AbstractThe automatic interpretation of sign languages is a challenging task, as it requires the usage of high-level vision and high-level motion processing systems for providing accurate image perception. In this paper, we use Convolutional Neural Networks (CNNs) and transfer learning to make computers able to interpret signs of the Swedish Sign Language (SSL) hand alphabet. Our model consists of the implementation of a pre-trained InceptionV3 network, and the usage of the mini-batch gradient descent optimization algorithm. We rely on transfer learning during the pre-training of the model and its data. The final accuracy of the model, based on 8 study subjects and 9400 images, is 85%. Our results indicate that the usage of CNNs is a promising approach to interpret sign languages, and transfer learning can be used to achieve high testing accuracy despite using a small training dataset. Furthermore, we describe the implementation details of our model to interpret signs as a user-friendly web application.

Download Full-text

The Glow Still Shows: Effects of Facial Masking on Perceptions of Duchenne Versus Social Smiles

Perception ◽

10.1177/03010066211027052 ◽

2021 ◽

pp. 030100662110270

Author(s):

Kennon M. Sheldon ◽

Ryan Goffredi ◽

Mike Corcoran

Keyword(s):

Facial Expressions ◽

Lower Half ◽

Communicative Functions ◽

The Face ◽

Facial Expressions Of Emotion ◽

Duchenne Smiles ◽

Neutral Expression

Facial expressions of emotion have important communicative functions. It is likely that mask-wearing during pandemics disrupts these functions, especially for expressions defined by activity in the lower half of the face. We tested this by asking participants to rate both Duchenne smiles (DSs; defined by the mouth and eyes) and non-Duchenne or “social” smiles (SSs; defined by the mouth alone), within masked and unmasked target faces. As hypothesized, masked SSs were rated much lower in “a pleasant social smile” and much higher in “a merely neutral expression,” compared with unmasked SSs. Essentially, masked SSs became nonsmiles. Masked DSs were still rated as very happy and pleasant, although significantly less so than unmasked DSs. Masked DSs and SSs were both rated as displaying more disgust than the unmasked versions.

Download Full-text

Covariance Mean-to-Standard-Deviation Factor for Ultrasound Imaging

2020 IEEE International Ultrasonics Symposium (IUS) ◽

10.1109/ius46767.2020.9251390 ◽

2020 ◽

Author(s):

Yaunguo Wang ◽

Chichao Zheng ◽

Hu Peng

Keyword(s):

Standard Deviation ◽

Ultrasound Imaging ◽

Deviation Factor

Download Full-text

Computer-assisted sign language translation: a study of translators’ practice to specify CAT software

Machine Translation ◽

10.1007/s10590-021-09278-w ◽

2021 ◽

Author(s):

Marion Kaczmarek ◽

Michael Filhol

Keyword(s):

Sign Language ◽

Language Translation ◽

Computer Assisted ◽

Translation Process ◽

Sign Languages ◽

Subjective Data ◽

Do So

AbstractProfessional Sign Language translators, unlike their text-to-text counterparts, are not equipped with computer-assisted translation (CAT) software. Those softwares are meant to ease the translators’ tasks. No prior study as been conducted on this topic, and we aim at specifying such a software. To do so, we based our study on the professional Sign Language translators’ practices and needs. The aim of this paper is to identify the necessary steps in the text-to-sign translation process. By filming and interviewing professionals for both objective and subjective data, we build a list of tasks and see if they are systematic and performed in a definite order. Finally, we reflect on how CAT tools could assist those tasks, how to adapt the existing tools to Sign Language and what is necessary to add in order to fit the needs of Sign Language translation. In the long term, we plan to develop a first prototype of CAT software for sign languages.

Download Full-text

The FATE Landscape of Sign Language AI Datasets

ACM Transactions on Accessible Computing ◽

10.1145/3436996 ◽

2021 ◽

Vol 14 (2) ◽

pp. 1-45

Author(s):

Danielle Bragg ◽

Naomi Caselli ◽

Julie A. Hochgesang ◽

Matt Huenerfauth ◽

Leah Katz-Hernandez ◽

...

Keyword(s):

Sign Language ◽

New Technology ◽

Individual Data ◽

Community Members ◽

Sign Languages ◽

Language Data ◽

History Of ◽

Language Technologies ◽

And Storage ◽

Access Mechanisms

Sign language datasets are essential to developing many sign language technologies. In particular, datasets are required for training artificial intelligence (AI) and machine learning (ML) systems. Though the idea of using AI/ML for sign languages is not new, technology has now advanced to a point where developing such sign language technologies is becoming increasingly tractable. This critical juncture provides an opportunity to be thoughtful about an array of Fairness, Accountability, Transparency, and Ethics (FATE) considerations. Sign language datasets typically contain recordings of people signing, which is highly personal. The rights and responsibilities of the parties involved in data collection and storage are also complex and involve individual data contributors, data collectors or owners, and data users who may interact through a variety of exchange and access mechanisms. Deaf community members (and signers, more generally) are also central stakeholders in any end applications of sign language data. The centrality of sign language to deaf culture identity, coupled with a history of oppression, makes usage by technologists particularly sensitive. This piece presents many of these issues that characterize working with sign language AI datasets, based on the authors’ experiences living, working, and studying in this space.

Download Full-text