Humans use two different means to exchange information: language and nonverbal communication. Often nonverbal signals emphasize and specify what is being said with words. Yet sometimes they collide, and the words are contradicted by what seeps through facial expression, gesture, and posture. This chapter discusses two theoretical frameworks for studying these nonverbal behaviors. The first approach (the emotional model) aims at unveiling the emotional state from facial expression and gesture. The second approach (the behavioral ecology model) analyzes the social meaning of nonverbal behavior, regardless of the emotional state of the sender of nonverbal signals. The two models are not incompatible and can be integrated to study nonverbal behavior. Yet, the behavioral ecology model explains some findings that are not accounted for by the emotional model. The final part of the chapter deals with neuropsychiatric conditions, such as Williams syndrome and prosopagnosia, that alter the encoding and decoding of nonverbal signals. The impact of these conditions on real-life social behavior can be dramatic, which shows the adaptive relevance of nonverbal communication.
Emotional State Recognition Using Facial Expression, Voice, and Physiological Signal
