Multimodal literacy is a term that originates in social semiotics, and refers to the study of language that combines two or more modes of meaning. The related term, multimodality, refers to the constitution of multiple modes in semiosis or meaning making. Modes are defined differently across schools of thought, and the classification of modes is somewhat contested. However, from a social semiotic approach, modes are the socially and culturally shaped resources or semiotic structure for making meaning. Specific examples of modes from a social semiotic perspective include speech, gesture, written language, music, mathematical notation, drawings, photographic images, or moving digital images.
Language and literacy practices have always been multimodal, because communication requires attending to diverse kinds of meanings, whether of spoken or written words, visual images, gestures, posture, movement, sound, or silence. Yet, undeniably, the affordances of people-driven digital media and textual production have given rise to an exponential increase in the circulation of multimodal texts in networked digital environments. Multimodal text production has become a central part of everyday life for many people throughout the life course, and across cultures and societies. This has been enabled by the ease of producing and sharing digital images, music, video games, apps, and other digital media via the Internet and mobile technologies.
The increasing significance of multimodal literacy for communication has led to a growing body of research and theory to address the differing potentials of modes and their intermodality for making meaning. The study of multimodal literacy learning in schools and society is an emergent field of research, which begins with the important recognition that reading and writing are rarely practiced as discrete skills, but are intimately connected to the use of multimodal texts, often in digital contexts of use. The implications of multimodal literacy for pedagogy, curriculum, and assessment in education is an expanding field of multimodal research. In addition, there is a growing attention to multimodal literacy practices that are practiced in informal social contexts, from early childhood to adolescence and adulthood, such as in homes, recreational sites, communities, and workplaces.
Chapter 7. Having a Feel for What Works: Polymedia, Emotion, and Literacy Practices with Mobile Technologies
