A practical guide for generating unsupervised, spectrogram-based latent space representations of animal vocalizations

The manual detection, analysis, and classification of animal vocalizations in acoustic recordings is laborious and requires expert knowledge. Hence, there is a need for objective, generalizable methods that detect underlying patterns in these data, categorize sounds into distinct groups, and quantify similarities between them. Among all computational methods that have been proposed to accomplish this, neighborhood-based dimensionality reduction of spectrograms to produce a latent-space representation of calls stands out for its conceptual simplicity and effectiveness. Using a dataset of manually annotated meerkat (Suricata suricatta) vocalizations, we demonstrate how this method can be used to obtain meaningful latent space representations that reflect the established taxonomy of call types. We analyze strengths and weaknesses of the proposed approach, give recommendations for its usage and show application examples, such as the classification of ambiguous calls and the detection of mislabeled calls. All analyses are accompanied by example code to help researchers realize the potential of this method for the study of animal vocalizations.

Honest signaling in mouse lemur vocalizations?

Animal vocalizations may provide information about a sender’s condition or motivational state and, hence, mediate social interactions. In this study, we examined whether vocalizations of gray mouse lemurs (Microcebus murinus) emitted in aggressive contexts (grunts, tsaks) co-vary with physical condition, which would underly and indicate honest signaling. We recorded calls from captive individuals that were subjected to a caloric restricted (CR) or ad libitum (AL) diet, assuming that individuals on an ad libitum dietary regime were in better condition. We analyzed 828 grunts produced by seven CR and nine AL individuals and 270 tsaks by eight CR and five AL individuals. Grunts consisted of two separate elements, with the 1st element having more energy in higher frequencies than the 2nd element. Body mass correlated negatively with acoustic features of grunts, and heavier individuals produced lower-frequency grunts. Acoustic features of grunts did not differ between sexes. Acoustic features of tsaks were predicted by neither body mass nor sex. However, tsaks produced by AL individuals were noisier than those of CR individuals. Hence, manipulation of body condition via dietary regimes affected acoustic features of calls given during aggression in different ways: acoustic features of grunts varied according to the rule of acoustic allometry, and can be considered as honest signals. Acoustic features of tsaks, however, varied according to motivational structural rules. Longitudinal studies are now indicated to examine whether intra-individual changes in body mass are also reflected in the acoustic structure of calls, allowing callers to signal more flexible variation in condition.

Rabbits use both spectral and temporal cues to discriminate the fundamental frequency of harmonic complexes with missing fundamentals

The pitch of harmonic complex tones (HCT) common in speech, music and animal vocalizations plays a key role in the perceptual organization of sound. Unraveling the neural mechanisms of pitch perception requires animal models but little is known about complex pitch perception by animals, and some species appear to use different pitch mechanisms than humans. Here, we tested rabbits' ability to discriminate the fundamental frequency (F0) of HCTs with missing fundamentals using a behavioral paradigm inspired by foraging behavior in which rabbits learned to harness a spatial gradient in F0 to find the location of a virtual target within a room for a food reward. Rabbits were initially trained to discriminate HCTs with F0s in the range 400-800 Hz and with harmonics covering a wide frequency range (800-16,000 Hz), and then tested with stimuli differing either in spectral composition to test the role of harmonic resolvability (Experiment 1), or in F0 range (Experiment 2), or both F0 and spectral content (Experiment 3). Together, these experiments show that rabbits can discriminate HCTs over a wide F0 range (200-1600 Hz) encompassing the range of conspecific vocalizations, and can use either the spectral pattern of harmonics resolved by the cochlea for higher F0s or temporal envelope cues resulting from interaction between unresolved harmonics for lower F0s. The qualitative similarity of these results to human performance supports using rabbits as an animal model for studies of pitch mechanisms providing species differences in cochlear frequency selectivity and F0 range of vocalizations are taken into account.

Utilizing DeepSqueak for automatic detection and classification of mammalian vocalizations: a case study on primate vocalizations

Bioacoustic analyses of animal vocalizations are predominantly accomplished through manual scanning, a highly subjective and time-consuming process. Thus, validated automated analyses are needed that are usable for a variety of animal species and easy to handle by non-programing specialists. This study tested and validated whether DeepSqueak, a user-friendly software, developed for rodent ultrasonic vocalizations, can be generalized to automate the detection/segmentation, clustering and classification of high-frequency/ultrasonic vocalizations of a primate species. Our validation procedure showed that the trained detectors for vocalizations of the gray mouse lemur (Microcebus murinus) can deal with different call types, individual variation and different recording quality. Implementing additional filters drastically reduced noise signals (4225 events) and call fragments (637 events), resulting in 91% correct detections (Ntotal = 3040). Additionally, the detectors could be used to detect the vocalizations of an evolutionary closely related species, the Goodman’s mouse lemur (M. lehilahytsara). An integrated supervised classifier classified 93% of the 2683 calls correctly to the respective call type, and the unsupervised clustering model grouped the calls into clusters matching the published human-made categories. This study shows that DeepSqueak can be successfully utilized to detect, cluster and classify high-frequency/ultrasonic vocalizations of other taxa than rodents, and suggests a validation procedure usable to evaluate further bioacoustics software.

Vocal size exaggeration may have contributed to the origins of vocalic complexity

Vocal tract elongation, which uniformly lowers vocal tract resonances (formant frequencies) in animal vocalizations, has evolved independently in several vertebrate groups as a means for vocalizers to exaggerate their apparent body size. Here, we propose that smaller speech-like articulatory movements that alter only individual formants can serve a similar yet less energetically costly size-exaggerating function. To test this, we examine whether uneven formant spacing alters the perceived body size of vocalizers in synthesized human vowels and animal calls. Among six synthetic vowel patterns, those characterized by the lowest first and second formant (the vowel /u/ as in ‘boot’) are consistently perceived as produced by the largest vocalizer. Crucially, lowering only one or two formants in animal-like calls also conveys the impression of a larger body size, and lowering the second and third formants simultaneously exaggerates perceived size to a similar extent as rescaling all formants. As the articulatory movements required for individual formant shifts are minor compared to full vocal tract extension, they represent a rapid and energetically efficient mechanism for acoustic size exaggeration. We suggest that, by favouring the evolution of uneven formant patterns in vocal communication, this deceptive strategy may have contributed to the origins of the phonemic diversification required for articulated speech. This article is part of the theme issue ‘Voice modulation: from origin and mechanism to social impact (Part II)’.

Bio-acoustic tracking and localization using heterogeneous, scalable microphone arrays

Microphone arrays are an essential tool in the field of bioacoustics as they provide a non-intrusive way to study animal vocalizations and monitor their movement and behavior. Microphone arrays can be used for passive localization and tracking of sound sources while analyzing beamforming or spatial filtering of the emitted sound. Studying free roaming animals usually requires setting up equipment over large areas and attaching a tracking device to the animal which may alter their behavior. However, monitoring vocalizing animals through arrays of microphones, spatially distributed over their habitat has the advantage that unrestricted/unmanipulated animals can be observed. Important insights have been achieved through the use of microphone arrays, such as the convergent acoustic field of view in echolocating bats or context-dependent functions of avian duets. Here we show the development and application of large flexible microphone arrays that can be used to localize and track any vocalizing animal and study their bio-acoustic behavior. In a first experiment with hunting pallid bats the acoustic data acquired from a dense array with 64 microphones revealed details of the bats’ echolocation beam in previously unseen resolution. We also demonstrate the flexibility of the proposed microphone array system in a second experiment, where we used a different array architecture allowing to simultaneously localize several species of vocalizing songbirds in a radius of 75 m. Our technology makes it possible to do longer measurement campaigns over larger areas studying changing habitats and providing new insights for habitat conservation. The flexible nature of the technology also makes it possible to create dense microphone arrays that can enhance our understanding in various fields of bioacoustics and can help to tackle the analytics of complex behaviors of vocalizing animals.

Neurometric correlates to sensitive high-frequency sound amplitude modulation detection by mice

Amplitude modulation (AM) is a common feature of natural sounds, including speech and animal vocalizations. Here, we used operant conditioning and in vivo electrophysiology to determine the AM detection threshold of mice as well as its underlying neuronal encoding. Mice were trained in a Go-NoGo task to detect the transition to AM within a noise stimulus designed to prevent the use of spectral side-bands or a change in intensity as alternative cues. Our results indicate that mice, in comparison with other species, detect high modulation frequencies up to 512 Hz exceptionally well, but show poor performance at low frequencies. Our in vivo multielectrode recordings in the inferior colliculus (IC) of both anesthetized and awake mice revealed a few single units with remarkable phase-locking ability to 512 Hz modulation, but not sufficient to explain the good behavioral detection. Using a model of the population response that combined dimensionality reduction with threshold detection, we reproduced the general high-pass characteristics of behavioral detection based on a subset of neurons showing the largest firing rate change (both increase and decrease) in response to AM. Our data thus identify candidate neurons in the IC to explain the high-pass transfer function for AM detection in the mouse.

Experimental characterization and automatic identification of stridulatory sounds inside wood

ABSTRACTThe propagation of animal vocalizations in water and in air is a well-studied phenomenon, but sound produced by bark and wood boring insects, which feed and reproduce inside trees, is poorly understood. Often being confined to the dark and chemically-saturated habitat of wood, many bark- and woodborers have developed stridulatory mechanisms to communicate acoustically. Despite their ecological and economic importance and the unusual medium used for acoustic communication, very little is known about sound production in these insects, or their acoustic interactions inside trees. Here, we use bark beetles (Scolytinae) as a model system to study the effects of wooden tissue on the propagation of insect stridulations and propose algorithms for their automatic identification. We characterize distance-dependence of the spectral parameters of stridulatory sounds, propose data-based models for the power decay of the stridulations in both outer and inner bark, provide optimal spectral ranges for stridulation detectability, and develop automatic methods for their detection and identification. We also discuss the acoustic discernibility of species cohabitating the same log. The species tested can be acoustically identified with 99% of accuracy at distances up to 20 cm and detected to the greatest extent in the 2-6 kHz frequency band. Phloem was a better medium for sound transmission than bark.

Phylogenetic signal in the vocalizations of vocal learning and vocal non-learning birds

Some animal vocalizations develop reliably in the absence of relevant experience, but an intriguing subset of animal vocalizations is learned: they require acoustic models during ontogeny in order to develop, and the learner's vocal output reflects those models. To what extent do such learned vocalizations reflect phylogeny? We compared the degree to which phylogenetic signal is present in vocal signals from a wide taxonomic range of birds, including both vocal learners (songbirds) and vocal non-learners. We used publically available molecular phylogenies and developed methods to analyse spectral and temporal features in a carefully curated collection of high-quality recordings of bird songs and bird calls, to yield acoustic distance measures. Our methods were initially developed using pairs of closely related North American and European bird species, and then applied to a non-overlapping random stratified sample of European birds. We found strong similarity in acoustic and genetic distances, which manifested itself as a significant phylogenetic signal, in both samples. In songbirds, both learned song and (mostly) unlearned calls allowed reconstruction of phylogenetic trees nearly isomorphic to the phylogenetic trees derived from genetic analysis. We conclude that phylogeny and inheritance constrain vocal structure to a surprising degree, even in learned birdsong. This article is part of the theme issue ‘Vocal learning in animals and humans’.

On the Importance of Passive Acoustic Monitoring Filters

Passive acoustic monitoring (PAM) is a noninvasive technique to supervise wildlife. Acoustic surveillance is preferable in some situations such as in the case of marine mammals, when the animals spend most of their time underwater, making it hard to obtain their images. Machine learning is very useful for PAM, for example to identify species based on audio recordings. However, some care should be taken to evaluate the capability of a system. We defined PAM filters as the creation of the experimental protocols according to the dates and locations of the recordings, aiming to avoid the use of the same individuals, noise patterns, and recording devices in both the training and test sets. It is important to remark that the filters proposed here were not intended to improve the accuracy rates. Indeed, these filters tended to make it harder to obtain better rates, but at the same time, they tended to provide more reliable results. In our experiments, a random division of a database presented accuracies much higher than accuracies obtained with protocols generated with PAM filters, which indicates that the classification system learned other components presented in the audio. Although we used the animal vocalizations, in our method, we converted the audio into spectrogram images, and after that, we described the images using the texture. These are well-known techniques for audio classification, and they have already been used for species classification. Furthermore, we performed statistical tests to demonstrate the significant difference between the accuracies generated with and without PAM filters with several well-known classifiers. The configuration of our experimental protocols and the database were made available online.