Recognition of emotions using Kinects

Emotion recognition can improve the quality of patient care, product development and human-machine interaction.Psychological studies indicate that emotional state can be expressed in the way people walk,and the human gait can be used to reveal a person's emotional state.This paper proposes a novel method to do emotion recognition by using Microsoft Kinect to record gait patterns and train machine learning algorithms for emotion recognition. 59 subjects are recruited, and their gait patterns are recorded by two Kinect cameras.Joint selection, coordinate system transformation, sliding window gauss filtering,differential operation, and data segmentation are used for data preprocessing.We run Fourier transformation to extract features from the gait patterns and utilize Principal Component Analysis(PCA) for feature selection. By using NaiveBayes, RandomForests, LibSVM and SMO classifiers, the accuracy of recognition between natural and angry emotions can reach 80%,and the accuracy of recognition between natural and happy emotions can reach above 70%.The result indicates that Kinect can be used in the recognition of emotions with fairly well performance.

Download Full-text

Recognition of emotions using Kinects

10.7287/peerj.preprints.1419v1 ◽

2015 ◽

Author(s):

Shun Li ◽

Liqing Cui ◽

Changye Zhu ◽

Nan Zhao ◽

Baobin Li ◽

...

Keyword(s):

Emotion Recognition ◽

Principal Component ◽

Machine Learning Algorithms ◽

Microsoft Kinect ◽

Human Gait ◽

Well Performance ◽

System Transformation ◽

Gait Patterns ◽

Quality Of Patient Care ◽

Recognition Of Emotions

Emotion recognition can improve the quality of patient care, product development and human-machine interaction.Psychological studies indicate that emotional state can be expressed in the way people walk,and the human gait can be used to reveal a person's emotional state.This paper proposes a novel method to do emotion recognition by using Microsoft Kinect to record gait patterns and train machine learning algorithms for emotion recognition. 59 subjects are recruited, and their gait patterns are recorded by two Kinect cameras.Joint selection, coordinate system transformation, sliding window gauss filtering,differential operation, and data segmentation are used for data preprocessing.We run Fourier transformation to extract features from the gait patterns and utilize Principal Component Analysis(PCA) for feature selection. By using NaiveBayes, RandomForests, LibSVM and SMO classifiers, the accuracy of recognition between natural and angry emotions can reach 80%,and the accuracy of recognition between natural and happy emotions can reach above 70%.The result indicates that Kinect can be used in the recognition of emotions with fairly well performance.

Download Full-text

Emotion recognition using Kinect motion capture data of human gaits

PeerJ ◽

10.7717/peerj.2364 ◽

2016 ◽

Vol 4 ◽

pp. e2364 ◽

Cited By ~ 18

Author(s):

Shun Li ◽

Liqing Cui ◽

Changye Zhu ◽

Baobin Li ◽

Nan Zhao ◽

...

Keyword(s):

Emotion Recognition ◽

Emotional State ◽

Low Cost ◽

Information Source ◽

Principal Component ◽

Microsoft Kinect ◽

Main Body ◽

Emotional States ◽

System Transformation ◽

Neutral State

Automatic emotion recognition is of great value in many applications, however, to fully display the application value of emotion recognition, more portable, non-intrusive, inexpensive technologies need to be developed. Human gaits could reflect the walker’s emotional state, and could be an information source for emotion recognition. This paper proposed a novel method to recognize emotional state through human gaits by using Microsoft Kinect, a low-cost, portable, camera-based sensor. Fifty-nine participants’ gaits under neutral state, induced anger and induced happiness were recorded by two Kinect cameras, and the original data were processed through joint selection, coordinate system transformation, sliding window gauss filtering, differential operation, and data segmentation. Features of gait patterns were extracted from 3-dimentional coordinates of 14 main body joints by Fourier transformation and Principal Component Analysis (PCA). The classifiers NaiveBayes, RandomForests, LibSVM and SMO (Sequential Minimal Optimization) were trained and evaluated, and the accuracy of recognizing anger and happiness from neutral state achieved 80.5% and 75.4%. Although the results of distinguishing angry and happiness states were not ideal in current study, it showed the feasibility of automatically recognizing emotional states from gaits, with the characteristics meeting the application requirements.

Download Full-text

Feature Analysis of Smart Shoe Sensors for Classification of Gait Patterns

Sensors ◽

10.3390/s20216253 ◽

2020 ◽

Vol 20 (21) ◽

pp. 6253

Author(s):

Unang Sunarya ◽

Yuli Sun Hariyani ◽

Taeheum Cho ◽

Jongryun Roh ◽

Joonho Hyeong ◽

...

Keyword(s):

Nearest Neighbor ◽

Pressure Sensors ◽

Machine Learning Algorithms ◽

Sensor Data ◽

Human Gait ◽

Support Vector ◽

K Nearest Neighbor ◽

Gait Patterns ◽

Gait Classification

Gait analysis is commonly used to detect foot disorders and abnormalities such as supination, pronation, unstable left foot and unstable right foot. Early detection of these abnormalities could help us to correct the walking posture and avoid getting injuries. This paper presents extensive feature analyses on smart shoes sensor data, including pressure sensors, accelerometer and gyroscope signals, to obtain the optimum combination of the sensors for gait classification, which is crucial to implement a power-efficient mobile smart shoes system. In addition, we investigated the optimal length of data segmentation based on the gait cycle parameters, reduction of the feature dimensions and feature selection for the classification of the gait patterns. Benchmark tests among several machine learning algorithms were conducted using random forest, k-nearest neighbor (KNN), logistic regression and support vector machine (SVM) algorithms for the classification task. Our experiments demonstrated the combination of accelerometer and gyroscope sensor features with SVM achieved the best performance with 89.36% accuracy, 89.76% precision and 88.44% recall. This research suggests a new state-of-the-art gait classification approach, specifically on detecting human gait abnormalities.

Download Full-text

Emotion Recognition Using Facial Expression

Innovative Research in Attention Modeling and Computer Vision Applications - Advances in Computational Intelligence and Robotics ◽

10.4018/978-1-4666-8723-3.ch013 ◽

2016 ◽

pp. 327-345 ◽

Cited By ~ 1

Author(s):

Santosh Kumar ◽

Shubam Jaiswal ◽

Rahul Kumar ◽

Sanjay Kumar Singh

Keyword(s):

Facial Expression ◽

Emotion Recognition ◽

Principal Component ◽

Machine Learning Algorithms ◽

Face Images ◽

Benchmark Database ◽

Matching Efficiency ◽

Facial Images ◽

Pattern Histogram

Recognition of facial expression is a challenging problem for machine in comparison to human and it has encouraged numerous advanced machine learning algorithms. It is one of the methods for emotion recognition as the emotion of a particular person can be found out by studying his or her facial expressions. In this paper, we proposes a generic algorithms for recognition of emotions and illustrates a fundamental steps of the four algorithms such as Eigenfaces (Principal Component Analysis [PCA]), Fisherfaces, Local Binary Pattern Histogram (LBP) and SURF with FLANN over two databases Cohn-kanade database and IIT BHU student face images as benchmark database.The objective of this book chapter is to recognize the emotions from facial images of individuals and compare the performances of holistic algorithms like Eigenfaces, Fisherfaces, and texture based recognition algorithms LBPH, hybrid algorithm SURF and FLANN. Matching efficiency of individual emotions from facial expression databases are labeled for training and testing phases. The set of features is extracted from labeled dataset for training purpose and test images are matched with discriminative set of feature points. Based on that comparison, we conclude that Eigenfaces and Fisherfaces yields good recognition accuracy on the benchmark database than others and the efficiency of SURF with FLANN algorithm can be enhanced significantly by changing the parameters.

Download Full-text

Emotion Recognition Using Facial Expression

Computer Vision ◽

10.4018/978-1-5225-5204-8.ch074 ◽

2018 ◽

pp. 1768-1787

Author(s):

Santosh Kumar ◽

Shubam Jaiswal ◽

Rahul Kumar ◽

Sanjay Kumar Singh

Keyword(s):

Facial Expression ◽

Emotion Recognition ◽

Principal Component ◽

Machine Learning Algorithms ◽

Face Images ◽

Benchmark Database ◽

Matching Efficiency ◽

Facial Images ◽

Pattern Histogram

Recognition of facial expression is a challenging problem for machine in comparison to human and it has encouraged numerous advanced machine learning algorithms. It is one of the methods for emotion recognition as the emotion of a particular person can be found out by studying his or her facial expressions. In this paper, we proposes a generic algorithms for recognition of emotions and illustrates a fundamental steps of the four algorithms such as Eigenfaces (Principal Component Analysis [PCA]), Fisherfaces, Local Binary Pattern Histogram (LBP) and SURF with FLANN over two databases Cohn-kanade database and IIT BHU student face images as benchmark database.The objective of this book chapter is to recognize the emotions from facial images of individuals and compare the performances of holistic algorithms like Eigenfaces, Fisherfaces, and texture based recognition algorithms LBPH, hybrid algorithm SURF and FLANN. Matching efficiency of individual emotions from facial expression databases are labeled for training and testing phases. The set of features is extracted from labeled dataset for training purpose and test images are matched with discriminative set of feature points. Based on that comparison, we conclude that Eigenfaces and Fisherfaces yields good recognition accuracy on the benchmark database than others and the efficiency of SURF with FLANN algorithm can be enhanced significantly by changing the parameters.

Download Full-text

Recognition of Emotions in Gait Patterns by Means of Artificial Neural Nets

Journal of Nonverbal Behavior ◽

10.1007/s10919-007-0045-3 ◽

2008 ◽

Vol 32 (2) ◽

pp. 79-92 ◽

Cited By ~ 49

Author(s):

Daniel Janssen ◽

Wolfgang I. Schöllhorn ◽

Jessica Lubienetzki ◽

Karina Fölling ◽

Henrike Kokenge ◽

...

Keyword(s):

Neural Nets ◽

Gait Patterns ◽

Artificial Neural Nets ◽

Artificial Neural ◽

Recognition Of Emotions

Download Full-text

Emotion recognition using time–frequency ridges of EEG signals based on multivariate synchrosqueezing transform

Biomedical Engineering / Biomedizinische Technik ◽

10.1515/bmt-2020-0295 ◽

2021 ◽

Vol 0 (0) ◽

Author(s):

Ahmet Mert ◽

Hasan Huseyin Celik

Keyword(s):

Emotion Recognition ◽

Feature Vector ◽

Dimensional Space ◽

Computational Cost ◽

Feature Space ◽

Maximum Energy ◽

Machine Learning Algorithms ◽

Eeg Signals ◽

Time Frequency ◽

Synchrosqueezing Transform

Abstract The feasibility of using time–frequency (TF) ridges estimation is investigated on multi-channel electroencephalogram (EEG) signals for emotional recognition. Without decreasing accuracy rate of the valence/arousal recognition, the informative component extraction with low computational cost will be examined using multivariate ridge estimation. The advanced TF representation technique called multivariate synchrosqueezing transform (MSST) is used to obtain well-localized components of multi-channel EEG signals. Maximum-energy components in the 2D TF distribution are determined using TF-ridges estimation to extract instantaneous frequency and instantaneous amplitude, respectively. The statistical values of the estimated ridges are used as a feature vector to the inputs of machine learning algorithms. Thus, component information in multi-channel EEG signals can be captured and compressed into low dimensional space for emotion recognition. Mean and variance values of the five maximum-energy ridges in the MSST based TF distribution are adopted as feature vector. Properties of five TF-ridges in frequency and energy plane (e.g., mean frequency, frequency deviation, mean energy, and energy deviation over time) are computed to obtain 20-dimensional feature space. The proposed method is performed on the DEAP emotional EEG recordings for benchmarking, and the recognition rates are yielded up to 71.55, and 70.02% for high/low arousal, and high/low valence, respectively.

Download Full-text