Avatars with faces of real people: A construction method for scientific experiments in virtual reality

Experimental psychology research typically employs methods that greatly simplify the real-world conditions within which cognition occurs. This approach has been successful for isolating cognitive processes, but cannot adequately capture how perception operates in complex environments. In turn, real-world environments rarely afford the access and control required for rigorous scientific experimentation. In recent years, technology has advanced to provide a solution to these problems, through the development of affordable high-capability virtual reality (VR) equipment. The application of VR is now increasing rapidly in psychology, but the realism of its avatars, and the extent to which they visually represent real people, is captured poorly in current VR experiments. Here, we demonstrate a user-friendly method for creating photo-realistic avatars of real people and provide a series of studies to demonstrate their psychological characteristics. We show that avatar faces of familiar people are recognised with high accuracy (Study 1), replicate the familiarity advantage typically observed in real-world face matching (Study 2), and show that these avatars produce a similarity-space that corresponds closely with real photographs of the same faces (Study 3). These studies open the way to conducting psychological experiments on visual perception and social cognition with increased realism in VR.

Emotion increases the similarity between neural representations of pictures, but not their perceived similarity.

Emotional similarity refers to the tendency to group stimuli together because they evoke the same feelings in us, even when they look different and have different semantic meanings. It is still unclear which features define the similarity space of emotional categories. Additionally, whether emotional stimuli are perceived as more similar than neutral ones, and whether this difference is paralleled by differences in their neural representations, has never been investigated. We conducted a series of experiments to quantify behavioural similarity, and one that used fMRI and Representation similarity analysis to compute neural similarity. We hypothesised that the similarity between emotional stimuli will be greater than between non-emotional stimuli, paralleled by higher neural similarity among emotional stimuli. We tested these hypotheses with two measures of similarity and two different databases of complex negative and neutral pictures, the second of which allowed us to control semantic similarity. For the first time, we found a decoupling between subjective and objective measures of emotional similarity. Pictures taken from two distinct emotional and neutral categories were perceived as equally similar; however, their neural similarity was higher. This effect was detected in brain clusters localised in a constrained search volume. We conclude that features representing participants similarity space may have different weights in these regions than they do in explicit ratings.

Behavioral and Neural Representations en route to Intuitive Action Understanding

When we observe another person's actions, we process many kinds of information -- from how their body moves to the intention behind their movements. What kinds of information underlie our intuitive understanding about how similar actions are to each other? To address this question, we measured the intuitive similarities among a large set of everyday action videos using multi-arrangement experiments, then used a modeling approach to predict this intuitive similarity space along three hypothesized properties. We found that similarity in the actors' inferred goals predicted the intuitive similarity judgments the best, followed by similarity in the actors' movements, with little contribution from the videos' visual appearance. In opportunistic fMRI analyses assessing brain-behavior correlations, we found evidence for an action processing hierarchy, in which these three kinds of action similarities are reflected in the structure of brain responses along a posterior-to-anterior gradient on the lateral surface of the visual cortex. Altogether, this work joins existing literature suggesting that humans are naturally tuned to process others' intentions, and that the visuo-motor cortex computes the perceptual precursors of the higher-level representations over which intuitive action perception operates.

Designing Exhibits to Support Relational Learning in a Science Museum

Science museums aim to provide educational experiences for both children and adults. To achieve this goal, museum displays must convey scientifically-relevant relationships, such as the similarities that unite members of a natural category, and the connections between scientific models and observable objects and events. In this paper, we explore how research on comparison could be leveraged to support learning about such relationships. We describe how museum displays could promote educationally-relevant comparisons involving natural specimens and scientific models. We also discuss how these comparisons could be supported through the design of a display—in particular, by using similarity, space, and language to facilitate relational thinking for children and their adult companions. Such supports may be pivotal given the informal nature of learning in museums.

The Duality of Similarity and Metric Spaces

We introduce a new mathematical basis for similarity space. For the first time, we describe the relationship between distance and similarity from set theory. Then, we derive generally valid relations for the conversion between similarity and a metric and vice versa. We present a general solution for the normalization of a given similarity space or metric space. The derived solutions lead to many already used similarity and distance functions, and combine them into a unified theory. The Jaccard coefficient, Tanimoto coefficient, Steinhaus distance, Ruzicka similarity, Gaussian similarity, edit distance and edit similarity satisfy this relationship, which verifies our fundamental theory.

Generalizing Psychological Similarity Spaces to Unseen Stimuli

The cognitive framework of conceptual spaces proposes to represent concepts as regions in psychological similarity spaces. These similarity spaces are typically obtained through multidimensional scaling (MDS), which converts human dissimilarity ratings for a fixed set of stimuli into a spatial representation. One can distinguish metric MDS (which assumes that the dissimilarity ratings are interval or ratio scaled) from nonmetric MDS (which only assumes an ordinal scale). In our first study, we show that despite its additional assumptions, metric MDS does not necessarily yield better solutions than nonmetric MDS. In this chapter, we furthermore propose to learn a mapping from raw stimuli into the similarity space using artificial neural networks (ANNs) in order to generalize the similarity space to unseen inputs. In our second study, we show that a linear regression from the activation vectors of a convolutional ANN to similarity spaces obtained by MDS can be successful and that the results are sensitive to the number of dimensions of the similarity space.

Convolutional neural net face recognition works in non-human-like ways

Convolutional neural networks (CNNs) give the state-of-the-art performance in many pattern recognition problems but can be fooled by carefully crafted patterns of noise. We report that CNN face recognition systems also make surprising ‘errors'. We tested six commercial face recognition CNNs and found that they outperform typical human participants on standard face-matching tasks. However, they also declare matches that humans would not, where one image from the pair has been transformed to appear a different sex or race. This is not due to poor performance; the best CNNs perform almost perfectly on the human face-matching tasks, but also declare the most matches for faces of a different apparent race or sex. Although differing on the salience of sex and race, humans and computer systems are not working in completely different ways. They tend to find the same pairs of images difficult, suggesting some agreement about the underlying similarity space.

Moving Beyond the Mean: Subgroups and dimensions of brain activity and cognitive performance across domains

Human neuroimaging during cognitive tasks has provided unique and important insights into the neurobiology of cognition. However, the vast majority of research relies upon group aggregate or average statistical maps of activity, which do not fully capture the rich variability which exists across individuals. To better characterize individual variability, hierarchical clustering was performed separately on six fMRI tasks in 822 participants from the Human Connectome Project. Across all tasks, clusters ranged from a predominantly ‘deactivating’ pattern towards a more ‘activating’ pattern of brain activity, with differences in out-of-scanner cognitive test scores between clusters. Cluster stability was assessed via bootstrapping approach. Cluster probability did not indicate distinct/clear clustering. However, when participants were plotted in a dimensionally reduced ‘similarity space’ derived from bootstrapping, variability in brain activity among participants was best represented multidimensionally. A ‘positive to negative’ axis of activity was the strongest driver of individual differences.

Stop Caring about Consciousness

The best empirically grounded theory of first-personal phenomenal consciousness is global workspace theory. This, combined with the success of the phenomenal-concept strategy, means that consciousness can be fully reductively explained in terms of globally broadcast representational content. So there are no qualia (and there is no mental paint). As a result, the question of which other creatures besides ourselves are phenomenally conscious is of no importance, and doesn’t admit of a factual answer in most cases. What is real, and what does matter, is a multidimensional similarity space of functionally organized minds.