iPoet: interactive painting poetry creation with visual multimodal analysis

2021 ◽  
Author(s):  
Yingchaojie Feng ◽  
Jiazhou Chen ◽  
Keyu Huang ◽  
Jason K. Wong ◽  
Hui Ye ◽  
...  
Keyword(s):  
Multimodal Analysis

Embodied displacements in young German children’s storytelling

2019 ◽  
Vol 3 (1-2) ◽  
pp. 168-195 ◽  
Author(s):  
Vivien Heller
Keyword(s):  
Picture Book ◽  
Book Reading ◽  
Lexical Resources ◽  
Narrative Space ◽  
Multimodal Analysis ◽  
Different Types ◽  
German Speaking ◽  
Picture Book Reading ◽  
The Given

This paper is concerned with embodied processes of joint imagination in young children’s narrative interactions. Based on Karl Bühler’s notion of ‘deixis in the imagination’, it examines in detail how a 19-month-old German-speaking child, engaged in picture book reading with his mother, brings about different subtypes of deixis in the imagination by either ‘displacing’ what is absent into the given order of perception (e.g. by using the hand as a token for an object) or displacing his origo to an imagined space (e.g. by kinaesthetically aligning his body with an imagined body and animating his movements). Drawing on multimodal analysis and the concept of layering in interaction, the study analyses the ways in which the picture book as well as deictic, depictive, vocal and lexical resources are coordinated to evoke a narrative space, co-enact the storybook character’s experiences and produce reciprocal affect displays. Findings demonstrate that different types of displacement are in play quite early in childhood; displacements in the dimension of space and person are produced through layerings of spaces, voices and bodies.

Construction of risk in road safety advertising: a multimodal analysis of Vietnamese child helmet awareness advertisements

2021 ◽  
Vol 0 (0) ◽  
Author(s):  
Hoang Van Nguyen
Keyword(s):  
Road Safety ◽  
Expert Knowledge ◽  
Multimodal Approach ◽  
Traditional Values ◽  
Multimodal Analysis ◽  
Multimodal Discourse Analysis ◽  
Multimodal Discourse ◽  
Comprehensive Picture ◽  
Media And Communication

AbstractThe discourses of risk serve to organise the ways in which we understand and respond to potential harms and threats, which have become a major concern in our daily life. However, the discourses of risk have not been extensively investigated using linguistic text-based methods on the multimodal level, nor deeply examined beyond Western contexts. Grounded in the literature of risk and multimodal discourse, the aim of the study is to demonstrate Multimodal Discourse Analysis from a Systemic Functional Linguistics perspective as a potential methodology to investigate how risk discourses are constructed in and through semiotic resources in a non-Western setting. Through a case study of child helmet awareness advertisements in Vietnam, the multimodal analysis reveals a comprehensive picture of risk discourses constructed across various semiotic modes. In this analysis, the discourses of risk are constructed through a negotiation of expert knowledge and traditional values to encourage the audience to take actions and provide helmets for their children. Findings of the study demonstrate the use of Systemic Functional multimodal approach to media and communication to provide evidence for risk discourses in the Vietnamese setting, which are at odds with the current literature and can potentially be extended to other contexts.

scholarly journals Darker shades of “fairness” in India: Male attractiveness and colorism in commercials

2020 ◽  
Vol 6 (1) ◽  
pp. 225-248
Author(s):  
Sayantan Mukherjee
Keyword(s):  
Critical Discourse Analysis ◽  
Skin Tone ◽  
Indian Society ◽  
Primary Data ◽  
Television Commercials ◽  
Multimodal Analysis ◽  
Light Skin ◽  
Indian Men ◽  
Skin Lightening

AbstractThe skin-lightening products for men in India and their mode of advertising have been shaping the concept of attractiveness for Indian men by portraying lighter skin tone as the most fundamental quality of being attractive, always desirable, and successful. Although women’s skin-lightening products in India have received attention by a few scholars lately, men’s products are still underresearched. Hence, this study aims to investigate the issue of colorism augmented by television commercials for men’s “fairness” (light skin tone) products in India. The primary data for this study are six Hindi television commercials for men’s skin-lightening products which were broadcast from 2005 to 2015 and were available on YouTube during data collection. The commercials are by one popular brand, Emami Fair and Handsome. The target commercials are significant for their categorical distinction in directness as well as for their nature of storytelling that helps facilitate the discourse of colorism itself. The methodology is a combination of multimodal analysis, critical discourse analysis, and advertisement analysis. The overall goal of this study is to bring visibility to this subtle and multilayered problem of colorism in Indian society which is being reinforced by the skin-lightening products for men.

scholarly journals What You Say or How You Say It? Depression Detection Through Joint Modeling of Linguistic and Acoustic Aspects of Speech

2021 ◽  
Author(s):  
Nujud Aloshban ◽  
Anna Esposito ◽  
Alessandro Vinciarelli
Keyword(s):  
Short Term Memory ◽  
Joint Modeling ◽  
Joint Analysis ◽  
Health Issues ◽  
Multimodal Analysis ◽  
Sequence Modeling ◽  
Depression Detection ◽  
Long Short Term Memory ◽  
Joint Representation ◽  
Better Than

AbstractDepression is one of the most common mental health issues. (It affects more than 4% of the world’s population, according to recent estimates.) This article shows that the joint analysis of linguistic and acoustic aspects of speech allows one to discriminate between depressed and nondepressed speakers with an accuracy above 80%. The approach used in the work is based on networks designed for sequence modeling (bidirectional Long-Short Term Memory networks) and multimodal analysis methodologies (late fusion, joint representation and gated multimodal units). The experiments were performed over a corpus of 59 interviews (roughly 4 hours of material) involving 29 individuals diagnosed with depression and 30 control participants. In addition to an accuracy of 80%, the results show that multimodal approaches perform better than unimodal ones owing to people’s tendency to manifest their condition through one modality only, a source of diversity across unimodal approaches. In addition, the experiments show that it is possible to measure the “confidence” of the approach and automatically identify a subset of the test data in which the performance is above a predefined threshold. It is possible to effectively detect depression by using unobtrusive and inexpensive technologies based on the automatic analysis of speech and language.

scholarly journals “Do you understand (me)?” negotiating mutual understanding by using gaze and environmentally coupled gestures between two deaf signing participants

2020 ◽  
Vol 0 (0) ◽  
Author(s):  
Nina Sivunen ◽  
Elina Tapio
Keyword(s):  
Asylum Seeker ◽  
Asylum Seekers ◽  
The State ◽  
Mutual Understanding ◽  
Multimodal Analysis ◽  
Semiotic Resources ◽  
Challenges And Opportunities ◽  
Language Competencies ◽  
Unrealistic Expectations ◽  
Insight Into

AbstractIn this paper we explore the use of multimodal and multilingual semiotic resources in interactions between two deaf signing participants, a researcher and an asylum seeker. The focus is on the use of gaze and environmentally coupled gestures. Drawing on multimodal analysis and linguistic ethnography, we demonstrate how gaze and environmentally coupled gestures are effective semiotic resources for reaching mutual understanding. The study provides insight into the challenges and opportunities (deaf) asylum seekers, researchers, and employees of reception centres or the state may encounter because of the asymmetrical language competencies. Our concern is that such asymmetrical situations may be created and maintained by ignoring visual and embodied resources in interaction and, in the case of deaf asylum seekers, by unrealistic expectations towards conventionalized forms of international sign.

CORNICLE LENGTH AS A CRITERION FOR SEPARATING FIELD-COLLECTED NYMPHAL INSTARS OF THE PEA APHID, ACYRTHOSIPHON PISUM (HOMOPTERA: APHIDIDAE)

1983 ◽  
Vol 115 (12) ◽  
pp. 1615-1619 ◽  
Author(s):  
William D. Hutchison ◽  
David B. Hogg
Keyword(s):  
Acyrthosiphon Pisum ◽  
Pea Aphid ◽  
Field Sample ◽  
Rearing Temperature ◽  
Multimodal Analysis

AbstractCornicle length measuremetns of Acyrthosiphon pisum (Harris) nymphs reared in the laboratory were instar-specific and unaffected by rearing temperature. A multimodal analysis of cornicle lengths of field-collected aphids clearly detected four distribution peaks (i.e., instars) in five different field populations, and there was generaaly little overlap between successive instar distributions. However, third and fourth instars in the spring field sample could not be separated accurately due to the shorter cornicle length of nymphs that developed from overwintered eggs. Cornicle length proved to be a useful criterion for separating virginoparous A. pisum instars in samples collected in southern Wisconsin.

Sign in / Sign up

Export Citation Format

Share Document