The Modalities of Media II: An Expanded Model for Understanding Intermedial Relations

This chapter is a significantly expanded and improved version of ‘The Modalities of Media: A Model for Understanding Intermedial Relations’ from 2010. It suggests an elaborated theoretical framework for distinguishing the multimodal character of media products, which are understood as those entities and phenomena that make inter-human communication possible. It offers a foundational model for describing and analysing the most basic similarities, differences and interrelations among all conceivable forms of media. The chapter also explains some basic mechanisms for categorising media products into media types and the intricate nature of media borders: how they are identified, construed and crossed. Finally, it broadly discusses two general intermedial perspectives: media integration and media transformation.

Electronic Screens in Film Diegesis: Modality Modes and Qualifying Aspects of a Formation Enhanced by the Post-digital Era

The proliferation of television screens, video monitors and computer or mobile screens in film diegetic worlds is an apparently simple numeric increase of certain objects within the filmed space, conditioned by, and thus mirroring, contemporary technological changes. However, one should consider this intermediary screenic formation as a complex and versatile audiovisual and narrative method that could have emerged in this frequency only in our current post-digital era. This chapter argues that fine-tuning the model of media functioning presented in Elleström’s “The Modalities of Media” for this specific phenomenon enables a more precise description of the process along which the three presemiotic media modalities morph into the semiotic one. By presenting a systematic description of electronic screens in film diegetic worlds, and a general assessment of the intermedial processes at work, the chapter examines Euro-American films influenced by the video, respectively, the digital era and technology.

Reading Audiobooks

The development of digital media technologies like the MP3 file and the smartphone has changed the status of the audiobook from being a by-product of the printed book to being a mass medium in its own right. This chapter takes a context and user perspective on audiobooks and asks the fundamental question: to what extent can one say that one ‘reads’ an audiobook? Based on the Danish author Helle Helle’s novel Ned til hundene (Down to the Dogs, 2008), the authors discuss how the audiobook experience as a whole can be analysed regarding ‘technological framing’, ‘reading situations’ and ‘the performing voice’. They also investigate audiobook reading in relation to the experience of time and depth.

A Recalibration of Theatre’s Hypermediality

It is often proposed that the unique capacity of theatre is that it allows all media hosted within it to manifest themselves in their own particular forms. This chapter pursues Lars Elleström’s notion that theatre is indeed highly multimodal and integrates many basic and qualified media, prompting a recalibration of current definitions of a hypermedium incorporating all arts and media. The author contends that, alongside the significance of material mobility, there are specific temporal, spatial and sensorial modes that are fundamental in defining the mechanics and the potential of the hypermedium. This interplay of modalities creates new forms of hybrid signification through particular dialogues of immediacy and hypermediacy, participant authorship, angles of mediation and angles of exclusivity, transporting theatre into new and sometimes challenging relationships with other assertive qualified media types, notably what the author refers to as the architecture of commerce.

Language in Digital Motion: From ABCs to Intermediality and Why This Matters for Language Learning

Linguistic communication has moved beyond simple alphabetic encoding to multimedia design, challenging the fit of structural theories of language to digital communication. This transition is barely evident in mobile language learning contexts where top-selling apps present language as a linguistic structure to be drilled, ironically bypassing the complex communicative potential of smart devices. This chapter overviews changing language norms from language as structure to language within multimodality and comparatively discusses multimodality from a social semiotics paradigm nested in linguistic theory and from Elleström’s intermediality paradigm. To illustrate how one could conceptualize multimodality from a perspective decentred from linguistics and leveraged to explain language use in multimedia contexts, the author examines two novel features of digital communication: emoji and conversational digital agents.

Truthfulness and Affect via Digital Mediation in Audiovisual Storytelling

This chapter investigates different ways in which the film techniques of digitally mediated images—such as found footage, diegetic camera, and computer screen—achieve story truthfulness and affective engagement in the viewer’s narrative interpretation process. The pursuit of truthful storytelling is to demonstrate objective facts, while mediated images in film are predominantly subjective. The chapter starts by reviewing the perennial paradox of two seemingly mutually exclusive narrative functions and then tackles the paradox by proposing a multi-leveled framework, synthesizing semiotic conceptualization and cognitive research findings. It also analyzes the various forms of digital mediated images in films over the last two decades and sheds light on how the functions of truthfulness and affective engagement can be closely intertwined rather than in conflict.

Multimodal Acting and Performing

This chapter considers how one might understand the work of actors and performers with reference to ideas of multimodality. The actor is not a ‘medium’ in the way of the media types in which she performs (theatre, film, radio drama, etc.). Yet, the actor does have a communicative function. It is argued that the modes of communicative media operate in a contemporary field of modal expansion, interrelation and transposition and that an account of acting and performing that considers such a field amends the longstanding focus on the actor’s ‘intention’ and ‘presence’. Instead, such an account delineates a technical infrastructure that suggests new insights into the actor’s work. Is the actor’s mode of performing expressed, shaped or constrained by the shifting media modalities in which her or his work is conveyed? The chapter examines instances of performances in theatre, in film, in ‘reality trend’ events and online.