Multi-modal language input: A learned superadditive effect

Review of psychological and language acquisition research into seeing faces while listening, seeing gesture while listening, illustrated text, reading while listening, and same language subtitled video, confirms that bi-modal input has a consistently positive effect on language learning over a variety of input types. This effect is normally discussed using a simple additive model where bi-modal input increases the total amount of data and adds redundancy to duplicated input thus increasing comprehension and then learning. Parallel studies in neuroscience suggest that bi-modal integration is a general effect using common brain areas and following common neural paths. Neuroscience also shows that bi-modal effects are more complex than simple addition, showing early integration of inputs, a learning/developmental effect, and a superadditive effect for integrated bi-modal input. The different bodies of research produce a revised model of bi-modal input as a learned, active system. The implications for language learning are that bi- or multi-modal input can powerfully enhance language learning and that the learning benefits of such input will increase alongside the development of neurological integration of the inputs.