Multimodal Affect Models: An Investigation of Relative Salience of Audio and Visual Cues for Emotion Prediction

People perceive emotions via multiple cues, predominantly speech and visual cues, and a number of emotion recognition systems utilize both audio and visual cues. Moreover, the perception of static aspects of emotion (speaker's arousal level is high/low) and the dynamic aspects of emotion (speaker is becoming more aroused) might be perceived via different expressive cues and these two aspects are integrated to provide a unified sense of emotion state. However, existing multimodal systems only focus on single aspect of emotion perception and the contributions of different modalities toward modeling static and dynamic emotion aspects are not well explored. In this paper, we investigate the relative salience of audio and video modalities to emotion state prediction and emotion change prediction using a Multimodal Markovian affect model. Experiments conducted in the RECOLA database showed that audio modality is better at modeling the emotion state of arousal and video for emotion state of valence, whereas audio shows superior advantages over video in modeling emotion changes for both arousal and valence.

Relative salience of chord-type and chord-voicing changes: A two-oddball paradigm

Our research project investigated the effect of background and stimuli factors on the relative salience of chord-type and chord-voicing changes. Earlier studies have shown that surface features tend to be easier to perceive than deeper features and that musical training attenuates this general tendency. For further studying how deeper-level and surface-level musical features are perceived, we used a two-oddball paradigm. Each item consisted of a succession of five same-root chords: one chord-type oddball (deeper feature), one voicing oddball (surface feature), and three standards. Participants chose the chord that sounded most different to them. All chord-type pairings formed of major, minor, dominant seventh, major seventh, and minor seventh chords were tested. Chord-type oddball and voicing oddball were chosen equally often, together forming the majority of the responses. Musical training and conceptual knowledge of chords affected the chord-type oddball responses, but not the voicing-oddball responses. However, chord-type oddballs were chosen regardless of the musical training. Chord-type responses were easiest for pairs consisting of a major-based and a minor-based chord and for pairs involving two pitch-class changes. Our results suggest that musical training and conceptual knowledge about chords is not the only factor influencing the relative salience of chord-type changes over voicing changes.

Speech-related auditory salience detection in the posterior superior temporal region

Processing auditory human speech requires both detection (early and transient) and analysis (sustained). We analyzed high gamma (70-110Hz) activity of intracranial electroencephalography waveforms acquired during an auditory task that paired forward speech, reverse speech, and signal correlated noise. We identified widespread superior temporal sites with sustained activity responding only to forward and reverse speech regardless of paired order. More localized superior temporal auditory onset sites responded to all stimulus types when presented first in a pair and responded in recurrent fashion to the second paired stimulus in select conditions even in the absence of interstimulus silence; a novel finding. Auditory onset activity to a second paired sound recurred according to relative salience, with evidence of partial suppression during linguistic processing. We propose that temporal lobe auditory onset sites facilitate a salience detector function with hysteresis of 200ms and are influenced by cortico-cortical feedback loops involving linguistic processing and articulation.

Asante Queen Mothers in Ghana

The queen mothers of Asante are linked together with chiefs in a dual-gender system of leadership. The symbol of authority and leadership in Asante is a stool (like a throne in England). Throughout the polities of Asante, each queen mother occupies her own stool, and each chief occupies his own stool, representing the authority of chieftaincy in a town or a paramountcy. This political model shapes Asante like a pyramid: queen mothers and chiefs of towns and villages at the base, paramount queen mothers and chiefs at the next level with authority over those of towns and villages, and the king of Asante, the Asantehene, and the queen mother of Asante, the Asantehemaa, at the top ruling over all of Asante. The king of Asante occupies the Golden Stool, the symbol of the Asante nation, which holds the souls of the Asante people according to popular belief. Although the position of queen mother has survived challenges, the relative salience of specific features of her authority has varied. Colonialism ignored queen mothers, and yet Yaa Asantewaa led a war and became a symbol of Asante identity. When the global women’s movement provided inspiration, queen mothers joined together to reclaim their authority.

The proposer or the proposal? An experimental analysis of constitutional beliefs

In many countries, constitutional amendments require the direct approval of voters, but the consequences of fundamental changes to the powers and operations of the state are difficult to anticipate. The referendums literature suggests that citizens weigh their prior beliefs about the merits of proposals against the heuristic provided by the partisanship of the proposer, but the relative salience of these factors across constitutional issue areas remains underexplored. This paper examines the determinants of citizen preferences on 12 diverse constitutional issues, based on a novel survey experiment in Japan. We show that support for amendments is greater when its proposer is described as non-partisan. However, constitutional ideology moderates this effect. Those who prefer idealistic constitutions that elevate national traditions tend to value proposals that expand government powers, compared to those who prefer pragmatic constitutions that constrain government authority. These results highlight the significance of constitutional beliefs that are independent of partisanship.

A Comparative Analysis of Emotion-Related Cultural Norms in Popular American and Chinese Storybooks

Storybooks written for young children contain rich information on emotions and act as important educational tools for children’s emotion socialization. The current study aims to investigate how cultural norms regarding emotions are portrayed in the narratives of popular storybooks across cultures. Thus, in this study, 38 bestselling Chinese storybooks written by Chinese authors and 42 bestselling American storybooks by European-American writers were compared. The narratives were coded with a focus on emotion-related content and further analysed using binary logistic regressions. The findings revealed that American storybooks were more likely to present positive (vs. negative) emotions, negative powerful (vs. negative powerless) emotions, and supportive (vs. unsupportive and teaching) responses to negative emotions than Chinese storybooks, but less likely to present social (vs. personal) themes, other-based (vs. self-based) attribution, and teaching (vs. supportive and unsupportive) responses to negative emotions. However, the results found no cultural variation in the prevalence of intrinsic (vs. extrinsic) interpersonal emotion regulation. The findings suggest that elements of emotion-related content coexist in both cultures although the relative salience of such content differs across cultures.

Conformity and reactions to deviance in the time of COVID-19

We have seen massive global behavioral change as billions of people radically altered their ways of life in response to COVID-19. Here, we review how research on conformity and deviance can inform understanding of and effective responses to the pandemic. Group identities are critical for understanding who is influenced by whom, as well as how partisan divisions can obstruct cohesive and coordinated action. We identify several questions highlighted by the pandemic, including when people will react more harshly to ingroup members who violate health-protective norms (black sheep effect) or to outgroup members violating the same norms (intergroup hypocrisy). As a working hypothesis, we propose a goal-based approach, positing that differentially negative reactions to ingroup and outgroup deviants are likely influenced by the relative salience of goals to protect ingroup image, enforce group norms, maintain intergroup boundaries, and seek justice for potential victims put at risk by norm-violating behavior.

You Won’t Believe Our Results! But They Might: Heterogeneity in Beliefs About the Accuracy of Online Media

“Clickbait” media has long been espoused as an unfortunate consequence of the rise of digital journalism. But little is known about why readers choose to read clickbait stories. Is it merely curiosity, or might voters think such stories are more likely to provide useful information? We conduct a survey experiment in Italy, where a major political party enthusiastically embraced the esthetics of new media and encouraged their supporters to distrust legacy outlets in favor of online news. We offer respondents a monetary incentive for correct answers to manipulate the relative salience of the motivation for accurate information. This incentive increases differences in the preference for clickbait; older and less educated subjects become even more likely to opt to read a story with a clickbait headline when the incentive to produce a factually correct answer is higher. Our model suggests that a politically relevant subset of the population prefers Clickbait Media because they trust it more.