The Effectiveness of Facial Expression Recognition in Detecting Emotional Responses to Sound Interventions in Older Adults With Dementia

This research uses facial expression recognition software (FaceReader) to explore the influence of different sound interventions on the emotions of older people with dementia. The field experiment was carried out in the public activity space of an older adult care facility. Three intervention sound sources were used, namely, music, stream, and birdsong. Data collected through the Self-Assessment Manikin Scale (SAM) were compared with facial expression recognition (FER) data. FaceReader identified differences in the emotional responses of older people with dementia to different sound interventions and revealed changes in facial expressions over time. The facial expression of the participants had significantly higher valence for all three sound interventions than in the intervention without sound (p < 0.01). The indices of sadness, fear, and disgust differed significantly between the different sound interventions. For example, before the start of the birdsong intervention, the disgust index initially increased by 0.06 from 0 s to about 20 s, followed by a linear downward trend, with an average reduction of 0.03 per 20 s. In addition, valence and arousal were significantly lower when the sound intervention began before, rather than concurrently with, the start of the activity (p < 0.01). Moreover, in the birdsong and stream interventions, there were significant differences between intervention days (p < 0.05 or p < 0.01). Furthermore, facial expression valence significantly differed by age and gender. Finally, a comparison of the SAM and FER results showed that, in the music intervention, the valence in the first 80 s helps to predict dominance (r = 0.600) and acoustic comfort (r = 0.545); in the stream sound intervention, the first 40 s helps to predict pleasure (r = 0.770) and acoustic comfort (r = 0.766); for the birdsong intervention, the first 20 s helps to predict dominance (r = 0.824) and arousal (r = 0.891).

Download Full-text

IDENTIFYING CUSTOMER’S EMOTIONAL RESPONSES TOWARDS GUEST-ROOM DESIGN BY USING FACIAL EXPRESSION RECOGNITION, IN HOTEL’S VIRTUAL AND REAL ENVIRONMENTS

Journal of Indonesian Tourism Hospitality and Recreation ◽

10.17509/jithor.v2i1.16681 ◽

2019 ◽

Vol 2 (1) ◽

Author(s):

Iván Alvarez León ◽

Begoña Juliá Nehme ◽

So Yeon Yoon

Keyword(s):

Facial Expression ◽

Facial Expression Recognition ◽

Emotional Responses ◽

Recognition System ◽

Expression Recognition ◽

City Image ◽

Peak Intensity ◽

Room Design ◽

Repeat Visit ◽

First Time

MENGIDENTIFIKASI RESPON EMOSIONAL PELANGGAN TERHADAP DESAIN KAMAR DENGAN MENGGUNAKAN PENGENALAN EKSPRESI WAJAH, DI LINGKUNGAN VIRTUAL DAN AKTUAL SEBUAH HOTELABSTRAKSangat umum diketahui bahwa emosi memainkan peran penting dalam pengalaman pelanggan dalam pariwisata. Mengukur emosi dapat memberikan informasi berharga tentang persepsi pelanggan tentang ruang hotel. Tujuan dari penelitian ini adalah untuk mengidentifikasi jenis lingkungan hotel mana, Virtual atau Asli, yang lebih efektif dalam memunculkan respon emosional dari responden yang ditunjukkan untuk pertama kalinya. Selain itu, penelitian ini bertujuan untuk mengungkap komponen kamar tamu mana, contohnya pemandangan alam ke luar atau interior yang mampu memicu respon emosional yang lebih tinggi. Dua percobaan dilakukan untuk memberikan bukti, yang pertama disajikan Lingkungan Virtual kepada responden melalui video, dan yang kedua dilakukan di lingkungan nyata pada jenis kamar presidential suite. Respon emosional dianalisis menggunakan perangkat lunak FaceReader, sistem pengenalan ekspresi wajah yang mengidentifikasi tujuh emosi, gairah fisiologis, dan kesenangan. Hasil menunjukkan nilai intensitas rata-rata keseluruhan yang rendah dari setiap emosi di lingkungan virtual dan nyata. Namun demikian, perbedaan signifikan dalam nilai intensitas puncak maksimum ditemukan antara lingkungan virtual dan nyata dengan nilai intensitas lebih tinggi di ruang tamu nyata. Tidak ada perbedaan signifikan yang ditemukan dalam tanggapan emosional terhadap pemandangan ke luar atau pandangan dari interior kamar. Kata kunci: Desain Emosional, Desain Hotel, Pengalaman Pelanggan, Pengenalan Ekspresi Wajah Otomatis IDENTIFYING CUSTOMER’S EMOTIONAL RESPONSES TOWARDS GUEST-ROOM DESIGN BY USING FACIAL EXPRESSION RECOGNITION, IN HOTEL’S VIRTUAL AND REAL ENVIRONMENTS ABSTRACTIt is well known that emotions play a key role in the customer experience in tourism. Measuring emotions can provide valuable information about customer’s perceptions regarding hotel spaces. The purpose of this study is to identify which type of hotel environment, Virtual or Real, is more effective in eliciting emotional responses from participants who are shown a scenario for the first time. Furthermore, this study aims to uncover which of the components of guestrooms, e.g., natural views to the outside or interiors are capable of triggering higher emotional responses. Two experiments were conducted to provide evidence, the first presented a Virtual Environment to participants via video, and while the second one was conducted in Real Environments of presidential suites. Emotional responses were analyzed using FaceReader software, a facial expression recognition system that identifies seven emotions, physiological arousal and pleasure. Results showed low overall mean intensity values of each emotion in both virtual and real environments. Nevertheless, significant differences in the maximum peak intensity values were found between virtual and real environments with intensity values being higher in the real guestroom. No significant differences were found in emotional responses to the views to the outside or views of the guestroom interiors. Keywords: City Image, Cognitive Image, Unique Image, Affective Image, Repeat Visit.

Download Full-text