Patch Attention Layer of Embedding Handcrafted Features in CNN for Facial Expression Recognition

Recognizing facial expression has attracted much more attention due to its broad range of applications in human–computer interaction systems. Although facial representation is crucial to final recognition accuracy, traditional handcrafted representations only reflect shallow characteristics and it is uncertain whether the convolutional layer can extract better ones. In addition, the policy that weights are shared across a whole image is improper for structured face images. To overcome such limitations, a novel method based on patches of interest, the Patch Attention Layer (PAL) of embedding handcrafted features, is proposed to learn the local shallow facial features of each patch on face images. Firstly, a handcrafted feature, Gabor surface feature (GSF), is extracted by convolving the input face image with a set of predefined Gabor filters. Secondly, the generated feature is segmented as nonoverlapped patches that can capture local shallow features by the strategy of using different local patches with different filters. Then, the weighted shallow features are fed into the remaining convolutional layers to capture high-level features. Our method can be carried out directly on a static image without facial landmark information, and the preprocessing step is very simple. Experiments on four databases show that our method achieved very competitive performance (Extended Cohn–Kanade database (CK+): 98.93%; Oulu-CASIA: 97.57%; Japanese Female Facial Expressions database (JAFFE): 93.38%; and RAF-DB: 86.8%) compared to other state-of-the-art methods.

Mental Representations and Facial Impressions of Muslim Men in Japan From 2015 to 2017

There is prejudice against Muslims in many nations, including Japan. This prejudice would be related to biased mental representations of Muslim faces. Moreover, in 2015, the increased news coverage linking Muslims to terrorism in Japan would have enhanced such negative mental representations. In the present study, Japanese participants were asked to imagine Muslim men, and from two faces with a random noise pattern added, participants were instructed to choose the face they imagined to be more Muslim. Typical Muslim facial representations were visualized in 2015, 2016, and 2017 by averaging all selected noise patterns using reverse correlation. The visualized representations were evaluated using the dimensions of warmth, competence, and basic emotions. The results showed that the warmth scores for the visualized facial representation were lower in 2015 than in 2017, whereas competence scores did not differ between the representations in 2015, 2016, and 2017. “Angry” and “disgusted” scores for the facial representation in 2015 were higher than those in 2017, whereas “happy” scores in 2015 were lower than those in 2017. The decreased “angry” score and increased “happy” score predicted an increase in the impression of warmth from 2015 to 2017.