Synaesthetic Sanctuary

<p><b>Traditional scientific methods of visualising sound data have focused on techniques that attempt to capture distinct elements of the audio signal, such as volume and length. However, existing methods such as spectrograms and waveform analysis are limited in their expression of the characteristics associated with complex sounds such as bird song. This research explores strategies to visualise sound in an aesthetically engaging manner. It uses sound data from native New Zealand birds as a design tool for creating an audio-visual design system. The distinct focus on timing and pitch within these songs makes the data suitable for visual comparison. The design techniques explored throughout this research project attempt to express the unique characteristics of a variety of New Zealand bird songs and calls. It investigates how artistic audio-visual methods can be integrated with scientific techniques so that the auditory data can be made more accessible to non-specialists.</b></p> <p>More specifically, this research aims to take advantage of the natural phonaesthetic connections people make between sonic and visual elements. The final output of this research consists of a generative design system that uses auditory data to create visualisations of New Zealand bird song. These visualisations have a mathematical basis, as well as being audio-visual artworks in themselves.</p>

Synaesthetic Sanctuary

<p><b>Traditional scientific methods of visualising sound data have focused on techniques that attempt to capture distinct elements of the audio signal, such as volume and length. However, existing methods such as spectrograms and waveform analysis are limited in their expression of the characteristics associated with complex sounds such as bird song. This research explores strategies to visualise sound in an aesthetically engaging manner. It uses sound data from native New Zealand birds as a design tool for creating an audio-visual design system. The distinct focus on timing and pitch within these songs makes the data suitable for visual comparison. The design techniques explored throughout this research project attempt to express the unique characteristics of a variety of New Zealand bird songs and calls. It investigates how artistic audio-visual methods can be integrated with scientific techniques so that the auditory data can be made more accessible to non-specialists.</b></p> <p>More specifically, this research aims to take advantage of the natural phonaesthetic connections people make between sonic and visual elements. The final output of this research consists of a generative design system that uses auditory data to create visualisations of New Zealand bird song. These visualisations have a mathematical basis, as well as being audio-visual artworks in themselves.</p>

Synaesthetic Sanctuary

<p><b>Traditional scientific methods of visualising sound data have focused on techniques that attempt to capture distinct elements of the audio signal, such as volume and length. However, existing methods such as spectrograms and waveform analysis are limited in their expression of the characteristics associated with complex sounds such as bird song. This research explores strategies to visualise sound in an aesthetically engaging manner. It uses sound data from native New Zealand birds as a design tool for creating an audio-visual design system. The distinct focus on timing and pitch within these songs makes the data suitable for visual comparison. The design techniques explored throughout this research project attempt to express the unique characteristics of a variety of New Zealand bird songs and calls. It investigates how artistic audio-visual methods can be integrated with scientific techniques so that the auditory data can be made more accessible to non-specialists.</b></p> <p>More specifically, this research aims to take advantage of the natural phonaesthetic connections people make between sonic and visual elements. The final output of this research consists of a generative design system that uses auditory data to create visualisations of New Zealand bird song. These visualisations have a mathematical basis, as well as being audio-visual artworks in themselves.</p>

Application of the Kalman Filter in Functional Magnetic Resonance Image Data

The Kalman-Bucy filter was applied on the preprocessing of the functional magnetic resonance image-fMRI. Numerical simulations of hemodynamic response added Gaussian noise were performed to evaluate the performance of the filter. After the proceeding was applied in auditory real data. The Kohonen’s self-organized map was employed as tools to compare the performance of the Kalman’s filter with another type of pre-processing. The results of the application of Kalman-Bucy filter for simulated data and real auditory data showed that it can be used as a tool in the temporal filtering step in fMRI data.

Associative Variational Auto-Encoder with Distributed Latent Spaces and Associators

In this paper, we propose a novel structure for a multi-modal data association referred to as Associative Variational Auto-Encoder (AVAE). In contrast to the existing models using a shared latent space among modalities, our structure adopts distributed latent spaces for multi-modalities which are connected through cross-modal associators. The proposed structure successfully associates even heterogeneous modality data and easily incorporates the additional modality to the entire network via the associator. Furthermore, in our structure, only a small amount of supervised (paired) data is enough to train associators after training auto-encoders in an unsupervised manner. Through experiments, the effectiveness of the proposed structure is validated on various datasets including visual and auditory data.

Deaf Qualitative Health Research: Leveraging Technology to Conduct Linguistically and Sociopolitically Appropriate Methods of Inquiry

One of the most understudied health disparity populations in the United States is the Deaf community—a sociolinguistic minority group of at least 500,000 individuals who communicate using American Sign Language. Research within this population is lacking, in part, due to researchers’ use of methodologies that are inaccessible to Deaf sign language users. Traditional qualitative methods were developed to collect and analyze participants’ spoken language. There is, therefore, a paradigm shift that must occur to move from an auditory data schema to one that prioritizes the collection and analysis of visual data. To effectively navigate this shift when working with Deaf sign language users, there are unique linguistic and sociopolitical considerations that should be taken into account. The current article explores these considerations and outlines an emerging method of conducting qualitative analysis that, we argue, has the potential to enhance qualitative researchers’ work regardless of the population of focus.