An Interactive Approach to Emotional Expression Through Musical Cues

Previous literature suggests that structural and expressive cues affect the emotion expressed in music. However, only a few systematic explorations of cues have been done, usually focussing on a few cues or a limited amount of predetermined arbitrary cue values. This paper presents three experiments investigating the effect of six cues and their combinations on the music's perceived emotional expression. Twenty-eight musical pieces were created with the aim of providing flexible, ecologically valid, unfamiliar, new stimuli. In Experiment 1, 96 participants assessed which emotions were expressed in the pieces using Likert scale ratings. In Experiment 2, a subset of the stimuli was modified by participants (N = 42) via six available cues (tempo, mode, articulation, pitch, dynamics, and brightness) to convey seven emotions (anger, sadness, fear, joy, surprise, calmness, and power), addressing the main aim of exploring the impact of cue levels to expressions. Experiment 3 investigated how well the variations of the original stimuli created by participants in Experiment 2 expressed their intended emotion. Participants (N = 91) rated them alongside the seven original pieces, allowing the exploration of similarities and differences between the two sets of related pieces. An overall pattern of cue combinations was identified for each emotion. Some findings corroborate previous studies: mode and tempo were the most impactful cues in shaping emotions, and sadness and joy were amongst the most accurately recognised emotions. Novel findings include soft dynamics being used to convey anger, and dynamics and brightness being the least informative cues. These findings provide further motivation to investigate the effect of cues on emotions in music as combinations of multiple cues rather than as individual cues, as one cue might not give enough information to portray a specific emotion. The new findings and discrepancies are discussed in relation to current theories of music and emotions.

Multimodal Affect Models: An Investigation of Relative Salience of Audio and Visual Cues for Emotion Prediction

People perceive emotions via multiple cues, predominantly speech and visual cues, and a number of emotion recognition systems utilize both audio and visual cues. Moreover, the perception of static aspects of emotion (speaker's arousal level is high/low) and the dynamic aspects of emotion (speaker is becoming more aroused) might be perceived via different expressive cues and these two aspects are integrated to provide a unified sense of emotion state. However, existing multimodal systems only focus on single aspect of emotion perception and the contributions of different modalities toward modeling static and dynamic emotion aspects are not well explored. In this paper, we investigate the relative salience of audio and video modalities to emotion state prediction and emotion change prediction using a Multimodal Markovian affect model. Experiments conducted in the RECOLA database showed that audio modality is better at modeling the emotion state of arousal and video for emotion state of valence, whereas audio shows superior advantages over video in modeling emotion changes for both arousal and valence.

Multiple cues guarantee successful predation by a Neotropical wasp

Predatory social wasps are well studied in several aspects; however, foraging behaviour, especially that which takes place away from the nest at often unpredictable locations, or specialized behaviours to find and subdue prey are not well understood. In the Brazilian tropical savanna, the Polistinae wasp Brachygastra lecheguana is specialized in preying on some endophytic weevil larvae which develops inside floral buds. We hypothesized that these wasps utilize a combination of different mechanisms such as visual, chemical (odour) and possible tactile cues to find the weevil larvae. Using a combination of experimental manipulations (visual; chemical; visual/chemical) we tested the wasp’s ability to detect the endophytic larvae in the field. Additionally, we checked the ability of this wasp to detect vibrations produced by the weevils inside the buds. Our results suggest that the B. lecheguana wasp utilizes a sequence of eco-physiological mechanisms to find the endophytic larva inside floral buds: sight, smell, and perhaps touch. The use of multiple cues by this wasp guarantees such a high rate of predation on endophytic beetles that the wasp may have positive implications (reduction in weevils’ infestation) for the future of the host plant’s reproduction.

Chemotactic and temperature-dependent responses of the Strongyloidoidea superfamily of nematodes

Host-seeking behaviour and how a parasite identifies the correct host to infect remains a poorly understood area of parasitology. What is currently known is that host sensation and seeking behaviour is formed from a complex mixture of chemo-, thermo- and mechanosensory behaviours, of which chemosensation is the best studied. Previous studies of olfaction in parasitic nematodes suggested that this behaviour appears to be more closely related to target host and infection mode than phylogeny. However, there has not yet been a study comparing the chemotactic and temperature-dependent behaviours of very closely related parasitic and non-parasitic nematodes. To this end, we examined the temperature-dependent and chemotactic responses of the Strongyloidoidea superfamily of nematodes. We found differences in temperature response between the different species and within infective larvae. Chemotactic responses were highly divergent, with different attraction profiles between all species studied. When examining direct stimulation with fur, we found that it was insufficient to cause an attractive response. Overall, our results support the notion that olfactory sensation is more closely related to lifestyle and host range than phylogeny, and that multiple cues are required to initiate host-seeking behaviour.

A model of head direction and landmark coding in complex environments

Environmental information is required to stabilize estimates of head direction (HD) based on angular path integration. However, it is unclear how this happens in real-world (visually complex) environments. We present a computational model of how visual feedback can stabilize HD information in environments that contain multiple cues of varying stability and directional specificity. We show how combinations of feature-specific visual inputs can generate a stable unimodal landmark bearing signal, even in the presence of multiple cues and ambiguous directional specificity. This signal is associated with the retrosplenial HD signal (inherited from thalamic HD cells) and conveys feedback to the subcortical HD circuitry. The model predicts neurons with a unimodal encoding of the egocentric orientation of the array of landmarks, rather than any one particular landmark. The relationship between these abstract landmark bearing neurons and head direction cells is reminiscent of the relationship between place cells and grid cells. Their unimodal encoding is formed from visual inputs via a modified version of Oja’s Subspace Algorithm. The rule allows the landmark bearing signal to disconnect from directionally unstable or ephemeral cues, incorporate newly added stable cues, support orientation across many different environments (high memory capacity), and is consistent with recent empirical findings on bidirectional HD firing reported in the retrosplenial cortex. Our account of visual feedback for HD stabilization provides a novel perspective on neural mechanisms of spatial navigation within richer sensory environments, and makes experimentally testable predictions.

The contribution of metamemory beliefs to the font size effect on judgments of learning: Is word frequency a moderating factor?

Previous studies found that metamemory beliefs dominate the font size effect on judgments of learning (JOLs). However, few studies have investigated whether beliefs about font size contribute to the font size effect in circumstances of multiple cues. The current study aims to fill this gap. Experiment 1 adopted a 2 (font size: 70 pt vs. 9 pt) * 2 (word frequency (WF): high vs. low) within-subjects design. The results showed that beliefs about font size did not mediate the font size effect on JOLs when multiple cues (font size and WF) were simultaneously provided. Experiment 2 further explored whether WF moderates the contribution of beliefs about font size to the font size effect, in which a 2 (font size: 70 pt vs. 9 pt, as a within-subjects factor) * 2 (WF: high vs. low, as a between-subjects factor) mixed design was used. The results showed that the contribution of beliefs about font size to the font size effect was present in a pure list of low-frequency words, but absent in a pure list of high-frequency words. Lastly, a meta-analysis showed evidence supporting the proposal that the contribution of beliefs about font size to the font size effect on JOLs is moderated by WF. Even though numerous studies suggested beliefs about font size play a dominant role in the font size effect on JOLs, the current study provides new evidence suggesting that such contribution is conditional. Theoretical implications are discussed.

Little evidence that Eurasian jays protect their caches by responding to cues about a conspecific’s desire and visual perspective

Eurasian jays have been reported to protect their caches by responding to cues about either the visual perspective or current desire of an observing conspecific, similarly to other corvids. Here, we used established paradigms to test whether these birds can – like humans – integrate multiple cues about different mental states and perform an optimal response accordingly. Across five experiments, which also include replications of previous work, we found little evidence that our jays adjusted their caching behaviour in line with the visual perspective and current desire of another agent, neither by integrating these social cues nor by responding to only one type of cue independently. These results raise questions about the reliability of the previously reported effects and highlight several key issues affecting reliability in comparative cognition research.

The Influence of Focus Marking on Pronoun Resolution in Dialogue Context

Using visual world eye-tracking, we examined whether adults (N = 58) and children (N = 37; 3;1–6;3) use linguistic focussing devices to help resolve ambiguous pronouns. Participants listened to English dialogues about potential referents of an ambiguous pronoun he. Four conditions provided prosodic focus marking to the grammatical subject or to the object, which were either additionally it-clefted or not. A reference condition focussed neither the subject nor object. Adult online data revealed that linguistic focussing via prosodic marking enhanced subject preference, and overrode it in the case of object focus, regardless of the presence of clefts. Children’s processing was also influenced by prosodic marking; however, their performance across conditions showed some differences from adults, as well as a complex interaction with both their memory and language skills. Offline interpretations showed no effects of focus in either group, suggesting that while multiple cues are processed, subjecthood and first mention dominate the final interpretation in cases of conflict.