Discrimination of natural acoustic variation in vocal signals

Studies of acoustic communication often focus on the categories and units of vocalizations, but subtle variation also occurs in how these signals are uttered. In human speech, it is not only phonemes and words that carry information but also the timbre, intonation, and stress of how speech sounds are delivered (often referred to as “paralinguistic content”). In non-human animals, variation across utterances of vocal signals also carries behaviorally relevant information across taxa. However, the discriminability of these cues has been rarely tested in a psychophysical paradigm. Here, we focus on acoustic communication in the zebra finch (Taeniopygia guttata), a songbird species in which the male produces a single stereotyped motif repeatedly in song bouts. These motif renditions, like the song repetitions of many birds, sound very similar to the casual human listener. In this study, we show that zebra finches can easily discriminate between the renditions, even at the level of single song syllables, much as humans can discriminate renditions of speech sounds. These results support the notion that sensitivity to fine acoustic details may be a primary channel of information in zebra finch song, as well as a shared, foundational property of vocal communication systems across species.

New Avenues in Audio Intelligence: Towards Holistic Real-life Audio Understanding

Computer audition (i.e., intelligent audio) has made great strides in recent years; however, it is still far from achieving holistic hearing abilities, which more appropriately mimic human-like understanding. Within an audio scene, a human listener is quickly able to interpret layers of sound at a single time-point, with each layer varying in characteristics such as location, state, and trait. Currently, integrated machine listening approaches, on the other hand, will mainly recognise only single events. In this context, this contribution aims to provide key insights and approaches, which can be applied in computer audition to achieve the goal of a more holistic intelligent understanding system, as well as identifying challenges in reaching this goal. We firstly summarise the state-of-the-art in traditional signal-processing-based audio pre-processing and feature representation, as well as automated learning such as by deep neural networks. This concerns, in particular, audio interpretation, decomposition, understanding, as well as ontologisation. We then present an agent-based approach for integrating these concepts as a holistic audio understanding system. Based on this, concluding, avenues are given towards reaching the ambitious goal of ‘holistic human-parity’ machine listening abilities.

Speech Enhancement Based on Fusion of Both Magnitude/Phase-Aware Features and Targets

Recently, supervised learning methods have shown promising performance, especially deep neural network-based (DNN) methods, in the application of single-channel speech enhancement. Generally, those approaches extract the acoustic features directly from the noisy speech to train a magnitude-aware target. In this paper, we propose to extract the acoustic features not only from the noisy speech but also from the pre-estimated speech, noise and phase separately, then fuse them into a new complementary feature for the purpose of obtaining more discriminative acoustic representation. In addition, on the basis of learning a magnitude-aware target, we also utilize the fusion feature to learn a phase-aware target, thereby further improving the accuracy of the recovered speech. We conduct extensive experiments, including performance comparison with some typical existing methods, generalization ability evaluation on unseen noise, ablation study, and subjective test by human listener, to demonstrate the feasibility and effectiveness of the proposed method. Experimental results prove that the proposed method has the ability to improve the quality and intelligibility of the reconstructed speech.

Japanese Young Women Did not Discriminate between Robots and Humans as Listeners for Their Self-Disclosure -Pilot Study-

Disclosing personal matters to other individuals often contributes to the maintenance of our mental health and social bonding. However, in face-to-face situations, it can be difficult to prompt others to self-disclose because people often feel embarrassed disclosing personal matters to others. Although artificial agents without strong social pressure for listeners to induce self-disclosure is a promising engineering method that can be applied in daily stress management and reduce depression, gender difference is known to make a drastic difference of the attitude toward robots. We hypothesized that, as compared to men, women tend to prefer robots as a listener for their self-disclosure. The experimental results that are based on questionnaires and the actual self-disclosure behavior indicate that men preferred to self-disclose to the human listener, while women did not discriminate between robots and humans as listeners for their self-disclosure in the willingness and the amount of self-disclosure. This also suggests that the gender difference needs to be considered when robots are used as a self-disclosure listener.

ЗВУКОВОЙ ОБРАЗ «В ЛАНДШАФТЕ», НЕПОСРЕДСТВЕННЫЙ И ИММЕРСИВНЫЙ

В статье рассматриваются возможности подхода к образу не только через область визуального. Указывая на звуковую материю как дополнительный источник образов, автор отмечает, с одной стороны, зависимость sound studies от визуальных исследований: методологические ходы, связанные с миметическим выделением пар терминов «слышать/слушать», «слышимое/неслышимое» (подобные «смотреть/видеть», «видимое/невидимое»); расширение предметного поля за счёт включения примеров использования звука, которые не рассматривались в истории и теории музыки (подобно тому, как история и теория искусств практически не рассматривала примеры, предшествовавшие складыванию живописного канона или появившиеся после его трансформации). С другой стороны, автор демонстрирует, как не отягощённые миметизмом визуального экологические sound studies, уравнивающие слушателя-человека с любыми объектами окружающего пространства в качестве препятствия на пути распространения звуковых волн, попутно существенно сужают возможность говорить об образе. Акцентируя безусловно важные характеристики звукового воздействия – иммерсивность и непосредственность, эти исследования подводят к выводу об отсутствующих столкновениях и специфических констелляциях, результатом которых мог бы быть образ. Выбранные для примера эксперименты Х. Ноака призваны показать, как нетипичный художественный жест, не отрицающий значимость «природы», раскрывает различные каналы восприятия навстречу друг другу и является генеративным в отношении констеллирующих эффектов вспыхивания образа.The article aims to investigate if there are possibilities to put questions of image beyond the field of the visual. Considering the sonic matter as a complementary source of images, the author demonstrates, on the one hand, the dependence of sound studies on visual studies. It can be seen in methodology that uses a pair of terms ‘hearing / listening’, ‘audible / unaudible’ similar to ‘look / regard’, ‘visible / invisible’, as well as in expansion of the subject field by including examples of dealing with sound beyond the frame of classical music theory. On the other hand, ecologically charged sound studies developing out of communication with visual studies, that equate a human listener with any object standing in the way of sound waves, diminish the possibility to speak about image. These studies emphasize the main characteristics of the impact of the sonic – immersion and immediacy, but at the same time accentuate the absence of tensions and specific constellations, the result of which could be an image. Contrary to the latter, Hunter Noack’s actions exemplify by means of a “natural” artistic gesture the opening of different sense perceptions one to the other in their generative functions to flash images.

Sonification With Music for Cybersecurity Situational Awareness

Cyber defenders work in stressful, information-rich, and high-stakes environments. While other researchers have considered sonification for security operations centers (SOCs), the mappings of network events to sound parameters have produced aesthetically unpleasing results. This paper proposes a novel sonification pro-cess for transforming data about computer network traffic into music. The musical cues relate to notable network events in such a way as to minimize the amount of training time a human listener would need in order to make sense of the cues. We demonstrate our technique on a dataset of 708 million authentication events over nine continuous months from an enterprise network. We il-lustrate a volume-centric approach in relation to the amplitude of the input data, and also a volumetric approach mapping the input data signal into the number of notes played. The resulting music prioritizes aesthetics over bandwidth to balance performance with adoption.

Chromes from Chromatin: Sonification of the Epigenome

The epigenetic modifications are organized in patterns determining the functional properties of the underlying genome. Such patterns, typically measured by ChIP-seq assays of histone modifications, can be combined and translated into musical scores, summarizing multiple signals into a single waveform. As music is recognized as a universal way to convey meaningful information (1), we wanted to investigate properties of music obtained by sonification of ChIP-seq data. We show that the music produced by such quantitative signals is perceived by human listener as more pleasant than that produced from randomized signals. Moreover, the waveform can be analyzed to predict phenotypic properties, such as differential gene expression.

Numerical Modeling of Head-Related Transfer Functions Using the Boundary Source Representation

A technique based on the virtual source representation is presented for modeling head-related transfer functions (HRTFs). This method is motivated by the theory of simple layer potential and the principle of wave superposition. Using the virtual source representation, the HRTFs for a human head with pinnae are calculated with a minimal amount of computation. In the process, a special regularization scheme is required to calculate the equivalent strengths of virtual sources. To justify the proposed method, tests were carried out to compare the virtual source method with the boundary element method (BEM) and a direct HRTF measurement. The HRTFs obtained using the virtual source method agrees reasonably well in terms of frequency response, directional response, and impulse response with the other methods. From the numerical perspectives, the virtual source method obviates the singularity problem as commonly encountered in the BEM, and is less computationally demanding than the BEM in terms of computational time and memory storage. Subjective experiments are also conducted using the calculated and the measured HRTFs. The results reveal that the spatial characteristics of sound localization are satisfactorily reproduced as a human listener would naturally perceive by using the virtual source HRTFs.