Visual Prosody Supports Reading Aloud Expressively

Type is not expressive enough. Even the youngest speakers are able to express a full range of emotions with their voice, while young readers read aloud monotonically as if to convey robotic boredom. We augmented type to convey expression similarly to our voices. Specifically, we wanted to convey in text words that are spoken louder, words that drawn out and spoken longer, and words that are spoken at a higher pitch. We then asked children to read sentences with these new kinds of type to see if children would read these with greater expression. We found that children would ignore the augmentation if they weren't explicitly told about it. But when children were told about the augmentation, they were able to read aloud with greater vocal inflection. This innovation holds great promise for helping both children and adults to read aloud with greater expression and fluency.

Visual prosody supports reading aloud expressively for deaf readers

Type is a wonderful tool to represent speech visually. Therefore, it can provide deaf individuals the information that they miss auditorily. Still, type does not represent all the information available in speech: it misses an exact indication of prosody. Prosody is the motor of expressive speech through speech variations in loudness, duration, and pitch. The speech of deaf readersis often less expressive because deafness impedes the perception and production of prosody. Support can be provided by visual cues that provide information about prosody—visual prosody—supporting both the training of speech variations and expressive reading. We will describe the influence of visual prosody on the reading expressiveness of deaf readers between age 7 and 18 (in this study, ‘deaf readers’ means persons with any kind of hearing loss, with or without hearing devices, who still developed legible speech). A total of seven cues visualize speech variations: a thicker/thinner font corresponds with a louder/quieter voice; a wider/narrower font relates to a lower/faster speed; a font raised above/lowered below the baseline suggests a higher/lower pitch; wider spaces between words suggest longer pauses. We evaluated the seven cues with questionnaires and a reading aloud test. Deaf readers relate most cues to the intendedspeech variation and read most of them aloud correctly. Only the raised cue is di#cult to connect to the intended speech variation at first, and a faster speed and lower pitch prove challenging to vocalize. Despite those two difficulties, this approach to visual prosody is elective in supporting speech prosody. The applied materials can form an example for typographers, type designers, graphic designers, teachers, speech therapists, and researchers developing expressive reading materials.

preliminary study exploring the relation between visual prosody and the prosodic components in sign language

Type enriched with visual prosody is a powerful tool to encourage expressive reading. Visual prosody adds cues to text to guide vocal variations in loud-ness, duration, and pitch. More vocal variations result in a less monotonous voice and thus more expression. A positive e!ect of visual prosody is known on the voice of normal hearing readers and of signed bilingual deaf readers who developed signed language and spoken language. These deaf readers rely on speech as well as sign language and both modalities can be used interchangeably to compensate each other. This preliminary study explores visual prosody in text in relation to Flemish Sign Language to see if sign language can be used to explain prosody. We asked deaf readers between 7 and 18 to relate prosodic cues to videos presenting prosodic components of Flemish Sign Language. We found that those readers connect the prosodic cues with the components in Flemish Sign Language as intended. Larger word-spacing cor-relates with a pause between signs, a wider font with a sign with ‘longer du-ration’, a thicker font with more ‘displacement’ in the sign, a raised font with a ‘faster velocity’ in the sign. However, some confusion occurred as participants seemed to extract only two prosodic components in the sign language: both the ‘faster velocity’ and ‘longer duration’ were referred to in terms of 'speed' and were not perceived as separate prosodic components. Participants were confused about why there were three cues in the text. Therefore, it is advised to re-evaluate and to re-design visual prosody for sign language with only ‘displacement’ and ‘speed’ in mind.

Visual Prosody Across Cultures

Communication partners not only exchange information through the auditory channel but also use a wide range of visual cues, such as facial expressions, eye gaze, hand gestures, and other types of body language. The latter set of features can be subsumed under the term ‘visual prosody’. Visual cues have been shown to be communicatively relevant, as they not only help to improve speech intelligibility, especially in noisy conditions, but can also signal information that is not expressed through the words or the syntactic structure of an utterance. This chapter examines how visual prosody can be put to use to highlight specific communicative functions, and how such cues relate to auditory forms of prosody. It also presents results of comparative analyses of visual prosody to shed light on resemblances and differences in how it is being exploited in different cultures.