Heroes and villains: multimodal identity construction in children’s wartime visual narratives

The article investigates the Self and Other binaries in wartime visual literature published in Bengali-language children’s periodicals in West Bengal, India during the Bangladesh Liberation Struggle 1971. The study applies a critical multimodal framework using the Social Actors Approach and Social Semiotics within the Discourse-Historical Approach. The binaries are defined by the representation and subsequent differentiation of physical, linguistic, and cultural features of the Bengali and non-Bengali social actors and through their actions in the plots. The representation of social actors in the texts conforms to as well as deviates from typical wartime propaganda.

Multimodal positioning of artifacts in interaction in a collaborative elementary engineering club

This exploratory case study uses multimodal positioning analysis to determine and describe how a purposefully crafted emergent artifact comes to influence and/or manipulate social dynamics, structure, and positionings of one design team comprised of five third-graders in an afterschool elementary engineering and literacy club. In addition to social semiotic theories of multimodality (e.g., Kress, G. (2010). Multimodality: a social semiotic approach to contemporary communication. New York, NY: Routledge) and multimodal interactional analysis (Norris, S. (2004). Analyzing multimodal interaction: a methodological framework. New York, NY: Routledge, Norris, S. (2019). Systematically working with multimodal data: research methods in multimodal discourse analysis. Hoboken, NJ: Wiley-Blackwell), Positioning Theory (Harré, R. and Van Langenhove, L. (1991). Varieties of positioning. J. Theor. Soc. Behav. 21: 393–407) is used to examine group interactions with the artifact, with observational data collected from audio, video, researcher field notes, analytic memos, photographs, student artifacts (e.g., drawn designs, built designs), and transcriptions of audio and video data. Analysis of interactions of the artifact as it unfolds demonstrates multiple types of role-based positioning with students (e.g., builder, helper, idea-sharer). Foregrounding analysis of the artifact, rather than the student participants, exposed students’ alignment or opposition with their groupmates during the project. This study contributes to multimodal and artifactual scholarship through a close examination of positions emergent across time through multimodal communicative actions and illustrates how perspectives on multimodality may be analytically combined with Positioning Theory.

Conceptual blending and (im)politeness in political cartooning

This paper is meant to give an account of multimodal (im)politeness in political cartoons, drawing primarily on critical discourse studies (CDS) (in particular, Teun van Dijk’s notion of “context models” and Paul Chilton’s concept of “critical discourse moments”), blending theory (Fauconnier and Turner 2002), and speech act theory (especially Geoffrey Leech’s most recent revisions of Penelope Brown and Stephen C. Levinson’s notions of negative and positive face). There is of course an abundant literature on blending theory, but the potential of this theory for analysing face-enhancing or face-threatening multimodal discourse has not been fully realised. It is shown that political cartoons can exemplify not only face attack but also face enhancement, and that blending theory can contribute to the comprehending and critique of sociopolitical action or linguistic and nonlinguistic forms of control that may operate in the world. The article thus demonstrates the value that results from merging critical cognitive linguistics and sociopragmatics.

Engaging students in multimodal literacy practices in a university ESP context: towards understanding identity and ideology in government debates

This paper focuses on issues of multimodal literacy practices in ESP higher education settings. In particular, the research explores how students become engaged in various literacy activities aimed at enhancing their critical-thinking skills and interpretation of images. For this purpose, two datasets consisting of video clips were extracted from a larger multimodal corpus and developed for teaching applications: one involved a UK live parliament debate and the other a US House of Representatives debate. The main objective is to identify the key verbal strategies reflecting persuasive, argumentative rhetoric and the non-verbal features accompanying these verbal utterances such as prosodic stress, body/head movements, gaze, gesture. Thus, the focus of the analysis is on how different semiotic modes of communication construct meaning, especially in terms of how they reinforce the construction of identity and ideological stance. The results were systematically categorized and applied on a practical level to a teaching unit on ‘identity and ideology’.

Towards developing multimodal literacies in the ESP classroom: methodological insights and practical applications

In this article, we provide an introduction to this special issue of Multimodal Communication entitled “Multimodal approaches in ESP: Innovative research and practice”. The Special Issue showcases innovative research presented at the 2019 International Conference on Knowledge Dissemination and Multimodal Literacy: Research Perspectives on ESP in a Digital Age. After briefly discussing the multimodal approach in language teaching and specifically in English for Specific Purposes (ESP) and its key role in developing multimodal competence, each of the five featured contributions is previewed. The contributions offer theoretically grounded and research-informed applications of the multimodal approach in the ESP classroom.

Exploring digital interactivity: a multimodal and social semiotic approach to intersemiotic construction in COVID China website

This paper explores intersemiotic construction and digital interactivity through a multimodal social semiotic lens examining the semiotic instantiations of interactive signs on the homepage of “Fighting COVID-19 the Chinese Way” (covid-19.chinadaily.com.cn), a website created to disseminate information on news and development of COVID prevention and control practices in China to contribute to global efforts to fight the pandemic. The study focuses on how digital interactivity is afforded by the COVID China website, where the interactive signs are ideationally and compositionally constructed for meaning representation and interpersonally for text-viewer relation construal. From the multimodal social semiotic perspective, systemic functional model-based cluster and intersemiotic analyses are applied to explore the visual, spatial, and linguistic features that contribute to the design and construction of interactive semiotic signs on the COVID China website and afford digital interactivity for viewers’ action potentials. This study extends the analytical focus to the semiotic instantiations of interactive signs and their intersemiotic construction process that stimulate the enabling of interactivity, instead of the interactivity per se, and demonstrates how different semiotic instantiations of interactive signs are featured and interact to afford digital interactivity. It argues for an integrated lens in analysis to look at the interactive signs not only as signs of action with action-enabling forms but also as signs of meanings that afford user-page interactivity.

Teaching communication strategies for the workplace: a multimodal framework

Recent changes in Higher Education (HE) approaches to content delivery, coupled with breakthroughs in the Information and Communications Technology field, have led to a whole new multimodal approach to teaching (Jewitt, C. 2009). In: Jewitt, C. (Ed.), The Routledge handbook of multimodal analysis. Routledge, London & New York; Jewitt, C. (2013). Multimodal methods for researching digital technologies. In: Jewitt, C. and Brown, B. (Eds.), The SAGE handbook of digital technology research. Sage, London, pp. 250–265; Kress, G. and van Leeuwen, T. (2001). Multimodal discourse. Bloomsbury Academic, London). Multimodality in language teaching increasingly draws on multiple channels of communication and not simply text on a page. Multimodal awareness and competence are also paramount in intercultural and interpersonal communication, which has become increasingly common in today’s global workplace. Through the description of the activities implemented in the English for Professional Purposes (EPP) course entitled English for the World of Work, held at the University of Verona, we will illustrate our multimodal, EPP framework based on Littlewood’s learning continuum, which ranges from analytical study to experiential practice (2014). Our principal aim, however, is to highlight ways in which the didactic framework fosters an awareness of and competence in key areas such as multimodal competence and intercultural awareness as skills required for effective communication in today’s world of work.

The designing of ocean threat comics by elementary students

A paucity of research has been conducted with learners in elementary classrooms on both the use of and the student creation of science comics. During the classroom-based research featured in this article, Grade 4 students designed ocean threat comics for the culminating activity of an interdisciplinary Ocean Literacy unit, one component of a larger study. Throughout the research, the students were afforded with opportunities to develop their visual meaning-making skills and competences, as well as their aesthetic understanding of and critical thinking about multimodal ensembles through participation in activities that focused on various elements of visual art and design, and conventions of the medium of comics. The visual and descriptive analysis of one student’s ocean threat comics, which includes excerpts from the interview about her work, reveals her motivations for selecting and orchestrating specific semiotic resources to represent and express particular meanings that realized her objectives as a sign-maker. Overall, the descriptions of the pedagogy featured during the research and the student’s ocean threat comics demonstrate how the development of student knowledge about elements of visual art and design, and conventions of the medium of comics can inform and deepen students’ semiotic work of comprehending, interpreting and designing science comics.

Tam pam pam pam and mi – fa – sol: constituting musical instructions through multimodal interaction in orchestra rehearsals

Using multimodal (inter)action/conversation analysis, the present contribution inventories the repertoire of higher-level actions that constitute musical instruction in orchestra rehearsals. The study describes the modal complexity of the instructional actions as built from a varied combination of speech, gesture, gaze, vocalizing and body posture/movement. A high modal intensity of speech and vocalizing is explained with recourse to their contextually useful modal reaches. While some modes, like vocalizing and body posture appear to be action-specific, others turn out to be pervasive default modes. Besides modal intensity, the study also attends to the transitioning between higher-level actions through gaze and the role of the score as frozen action. The analyses help demystify orchestra rehearsals as a special type of professional communicative interaction, which builds on a rich multimodal texture motivated by recurring instructional functions. The methodological rationale demonstrated will be suited to exploring the social variation of instructional interaction in orchestra rehearsals.