Listening to Dura Europos: An Experiment in Archaeological Image Sonification

We present an experiment in sonifying archival archaeological imagery to make the act of looking at photography strange and weird. The sounds produced will then arrest us and slow us down, and make apparent to us the different ways that archaeological vision is constructed to particular effect/affect. It makes us alive to what is hidden or elided in the image itself; in slowing down, listening/looking/moving at one, we are moved towards enchantment, and engage in a kind of digital hermeneutics that reveals more than what the lens may have captured.

Listening to Dura Europos: An Experiment in Archaeological Image Sonification

We present an experiment in sonifying archival archaeological imagery to make the act of looking at photography strange and weird. The sounds produced will then arrest us and slow us down, and make apparent to us the different ways that archaeological vision is constructed to particular effect/affect. It makes us alive to what is hidden or elided in the image itself; in slowing down, listening/looking/moving at one, we are moved towards enchantment, and engage in a kind of digital hermeneutics that reveals more than what the lens may have captured.

Preliminary Guidelines on the Sonification of Visual Artworks: Linking Music, Sonification & Visual Arts

Sonification and data processing algorithms have advanced over the years to reach practical applications in our everyday life. Similarly, image processing techniques have improved over time. While a number of image sonification methods have already been developed, few have delved into potential synergies through the combined use of multiple data and image processing techniques. Additionally, little has been done on the use of image sonification for artworks, as most research has been focused on the transcription of visual data for people with visual impairments. Our goal is to sonify paintings reflecting their art style and genre to improve the experience of both sighted and visually impaired individuals. To this end, we have designed initial sonifications for paintings of abstractionism and realism, and conducted interviews with visual and auditory experts to improve our mappings. We believe the recommendations and design directions we have received will help develop a multidimensional sonification algorithm that can better transcribe visual art into appropriate music.

Cross-Modal Correspondences Enhance Performance on a Colour-to-Sound Sensory Substitution Device

Visual sensory substitution devices (SSDs) can represent visual characteristics through distinct patterns of sound, allowing a visually impaired user access to visual information. Previous SSDs have avoided colour and when they do encode colour, have assigned sounds to colour in a largely unprincipled way. This study introduces a new tablet-based SSD termed the ‘Creole’ (so called because it combines tactile scanning with image sonification) and a new algorithm for converting colour to sound that is based on established cross-modal correspondences (intuitive mappings between different sensory dimensions). To test the utility of correspondences, we examined the colour–sound associative memory and object recognition abilities of sighted users who had their device either coded in line with or opposite to sound–colour correspondences. Improved colour memory and reduced colour-errors were made by users who had the correspondence-based mappings. Interestingly, the colour–sound mappings that provided the highest improvements during the associative memory task also saw the greatest gains for recognising realistic objects that also featured these colours, indicating a transfer of abilities from memory to recognition. These users were also marginally better at matching sounds to images varying in luminance, even though luminance was coded identically across the different versions of the device. These findings are discussed with relevance for both colour and correspondences for sensory substitution use.