From Social Gaze to Indirect Speech Constructions: How to Induce the Impression That Your Companion Robot Is a Conscious Creature

We implemented different modes of social gaze behavior in our companion robot, F-2, to evaluate the impression of the gaze behaviors on humans in three symmetric communicative situations: (a) the robot telling a story, (b) the person telling a story to the robot, and (c) both parties communicating about objects in the real world while solving a Tangram puzzle. In all the situations the robot localized the human’s eyes and directed its gaze between the human, the environment, and the object of interest in the problem space (if it existed). We examined the balance between different gaze directions as the novel key element to maintaining a feeling of social connection with the robot in humans. We extended the computer model of the robot in order to simulate realistic gaze behavior in the robot and create the impression of the robot changing its internal cognitive states. Other novel results include the implicit, rather than explicit, character of the robot gaze perception for many of our subjects and the role of individual differences, especially the level of emotional intelligence, in terms of human sensitivity to the robotic gaze. Therefore, in this study, we used an iterative approach, extending the applied cognitive architecture in order to simulate the balance between different behavioral reactions and to test it in the experiments. In such a way, we came to a description of the key behavioral cues that suggest to a person that the particular robot can be perceived as an emotional and even conscious creature.

Robot’s social gaze affects conflict resolution but not conflict adaptations

Robots are a new category of social agents that, thanks to their embodiment, can be used to train and support cognitive skills such as cognitive control. Several studies showed that cognitive control mechanisms are sensitive to affective states induced by humor, mood, and symbolic feedback such as monetary rewards. In the present study, we investigated whether the social gaze of a humanoid robot can affect cognitive control mechanisms. To this end, in two experiments, we evaluated both the conflict resolution and trial-by-trial adaptations during an auditory Simon task, as a function of the type of feedback participants received in the previous trial from the iCub robot, namely, mutual or avoiding gaze behaviour. Across three experiments, we compared the effect of mutual, avoiding (Exp1 and Exp2), and neutral (Exp3) gaze feedback between screen-based (Exp1) and physically embodied setups (Exp2 and Exp3). Results showed that iCub’s social gaze feedback modulated conflict resolution, but not conflict adaptations. Specifically, the Simon effect was increased following mutual gaze feedback from iCub. Moreover, the modulatory effect was observed for the embodied setup in which the robot could engage or avoid eye contact in real-time (Exp2) but not for the screen-based setting (Exp1). Our findings showed for the first time that social feedback in Human-Robot Interaction, such as social gaze, can be used to modulate cognitive control. The results highlight the advantage of using robots to evaluate and train complex cognitive skills in both healthy and clinical populations.

Single-cell foundations of live social gaze interaction in the prefrontal cortex and amygdala

Social gaze interaction powerfully shapes interpersonal communication in humans and other primates. However, little is known about the neural underpinnings of these social behavioral exchanges. Here, we studied neural responses associated with naturalistic, face-to-face, social gaze interactions between pairs of macaques. We examined spiking activity in a large number of neurons spanning four different brain regions involved in social behaviors - the amygdala, orbitofrontal cortex, anterior cingulate cortex, and dorsomedial prefrontal cortex. We observed widespread single-cell representations of social gaze interaction functionalities in these brain regions - social discriminability, social gaze monitoring, and mutual eye contact selectivity. Many of these neurons discriminated looking at social versus non-social stimuli with rich temporal heterogeneity, or parametrically tracked the gaze positions of oneself or the conspecific. Furthermore, many neurons displayed selectivity for mutual eye contact as a function of the initiator or follower of mutual gaze events. Crucially, a significant proportion of neurons coded for more than one of these three signatures of social gaze interaction, supporting the recruitment of partially overlapping neuronal ensembles. Our findings emphasize integrated contributions of the amygdala and prefrontal circuits within the social interaction networks in processing real-life social interactions.

Embodied Religious Belief: The Experience of Syahadatain Sufi Order in Indonesia

Several types of research on religion in Indonesia emphasise more on religious knowledge and belief and less on other approaches such as material aspects. Religion is always related to material aspects such as mosque buildings, veils or robes for prayer, or holy water obtained from grave visitors. This study uses embodiment approach and material theory of religion to the imposition of special fashion in prayer that gives consequences on awareness and attitude of a new morality in-group cohesion. This study is a case study through participatory observations and interviews of new members of Syahadatain congregation, Cirebon, Indonesia for three years. The result of the study shows two significant findings namely first, the establishment of rituals through special fashions exerts an influence on discursive awareness and moral behaviour; and second, the driving factor of the emergence of new moral behaviour from the practice is the social gaze. This study recommends the need for the study of material artifacts such as clothing can be an alternative to the study of religion in Indonesia using the embodiment approach.

Interpersonal synchrony modulates infants’ reactions to caregiver-infant miscoordination

In the present study we examined the role of behavioural synchrony and endocrinological responses in 4-month-old infants during social exchanges with their mothers for their responses during miscoordinated interactions, where mothers did not reciprocate with them. A total of 40 mother-infant dyads were observed during three conditions: baseline, natural interaction (NI), and a modified interaction (MI) where mothers delivered an emotionless speech about facts, while continually looking at, but refraining from engaging with or reacting to the infant. During NI we observed gaze and affect synchrony, and during MI we measured infant social gaze, positive and negative behaviours. Four saliva samples were collected from infants throughout the visit to determine OT, and we used changes in infant OT from before to after NI for analyses. Results showed that both gaze and affect synchrony were positively associated with social gaze and negatively with negative behaviours during MI, while we did not find a correlation between behavioural synchrony and infant positive behaviours during MI. Interestingly, increase in infant OT from before to after NI was positively associated with both social gaze and negative behaviours, but negatively with positive behaviours during MI. These findings suggest that prior synchrony experiences with caregivers allow infants to notice instances of miscoordination, and to self-regulate during these moments. Moreover, OT seems to enable infants to notice changes in regular interaction patterns, but it is likely only in combination with synchrony experiences that OT also contributes to the development of regulatory capacities.

Use of machine learning to identify autism with natural social gaze behavior (Preprint)

BACKGROUND Abnormal gaze behavior is a prominent feature of the autism spectrum disorder (ASD). Previous eye tracking studies had participants watch images (i.e., picture, video and webpage), and the application of machine learning (ML) on these data showed promising results in identify ASD individuals. Given the fact that gaze behavior differs in face-to-face interaction from image viewing tasks, no study has investigated whether natural social gaze behavior could accurately identify ASD. OBJECTIVE The objective of this study was to examine whether and what area of interest (AOI)-based features extracted from the natural social gaze behavior could identify ASD. METHODS Both children with ASD and typical development (TD) were eye-tracked when they were engaged in a face-to-face conversation with an interviewer. Four ML classifiers (support vector machine, SVM; linear discriminant analysis, LDA; decision tree, DT; and random forest, RF) were used to determine the maximum classification accuracy and the corresponding features. RESULTS A maximum classification accuracy of 84.62% were achieved with three classifiers (LDA, DT and RF). Results showed that the mouth, but not the eyes AOI, was a powerful feature in detecting ASD. CONCLUSIONS Natural gaze behavior could be leveraged to identify ASD, suggesting that ASD might be objectively screened with eye tracking technology in everyday social interaction. In addition, the comparison between our and previous findings suggests that eye tracking features that could identify ASD might be culture dependent and context sensitive.