scholarly journals Analysis of Micro Facial Expression by Machine and Deep Learning Methods: Haar, CNN, and RNN

2021 ◽  
Vol 16 (1) ◽  
pp. 95-101
Author(s):  
Dibakar Raj Pant ◽  
Rolisha Sthapit
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Facial Expression ◽  
Facial Expressions ◽  
Recurrent Neural Network ◽  
Test Accuracy ◽  
Data Sets ◽  
Learning Methods ◽  
Facial Movements ◽  
Casme Ii

Facial expressions are due to the actions of the facial muscles located at different facial regions. These expressions are two types: Macro and Micro expressions. The second one is more important in computer vision. Analysis of micro expressions categorized by disgust, happiness, anger, sadness, surprise, contempt, and fear are challenging because of very fast and subtle facial movements. This article presents one machine learning method: Haar and two deep learning methods: Convolution Neural Network (CNN) and Recurrent Neural Network (RNN) to perform recognition of micro-facial expression analysis. First, Haar Cascade Classifier is used to detect the face as a pre-image-processing step. Secondly, those detected faces are passed through series of Convolutional Neural Network (CNN) layers for the features extraction. Thirdly, the Recurrent Neural Network (RNN) classifies micro facial expressions. Two types of data sets are used for training and testing of the proposed method: Chinese Academy of Sciences Micro-Expression II (CSAME II) and Spontaneous Actions and Micro-Movements (SAMM) database. The test accuracy of SAMM and CASME II are obtained as 84.76%, and 87% respectively. In addition, the distinction between micro facial expressions and non- micro facial expressions are analyzed by the ROC curve.

scholarly journals Facial Expression Recognition Based on Deep Learning Convolution Neural Network: A Review

2021 ◽  
Vol 2 (1) ◽  
Author(s):  
Sharmeen M. Saleem Abdullah ◽  
◽  
Adnan Mohsin Abdulazeez ◽  
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Facial Expression ◽  
Facial Expressions ◽  
Video Game ◽  
Facial Expression Recognition ◽  
Emotional Processing ◽  
Convolution Neural Network ◽  
Data Sets ◽  
Expression Recognition

Facial emotional processing is one of the most important activities in effective calculations, engagement with people and computers, machine vision, video game testing, and consumer research. Facial expressions are a form of nonverbal communication, as they reveal a person's inner feelings and emotions. Extensive attention to Facial Expression Recognition (FER) has recently been received as facial expressions are considered. As the fastest communication medium of any kind of information. Facial expression recognition gives a better understanding of a person's thoughts or views and analyzes them with the currently trending deep learning methods. Accuracy rate sharply compared to traditional state-of-the-art systems. This article provides a brief overview of the different FER fields of application and publicly accessible databases used in FER and studies the latest and current reviews in FER using Convolution Neural Network (CNN) algorithms. Finally, it is observed that everyone reached good results, especially in terms of accuracy, with different rates, and using different data sets, which impacts the results.

scholarly journals A Review of Plant Phenotypic Image Recognition Technology Based on Deep Learning

Electronics ◽  
2021 ◽  
Vol 10 (1) ◽  
pp. 81
Author(s):  
Jianbin Xiong ◽  
Dezheng Yu ◽  
Shuangyin Liu ◽  
Lei Shu ◽  
Xiaochan Wang ◽  
...  
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Convolutional Neural Network ◽  
Plant Species ◽  
Image Recognition ◽  
Recurrent Neural Network ◽  
Plant Diseases ◽  
Learning Methods ◽  
Smart Agriculture ◽  
Important Branch

Plant phenotypic image recognition (PPIR) is an important branch of smart agriculture. In recent years, deep learning has achieved significant breakthroughs in image recognition. Consequently, PPIR technology that is based on deep learning is becoming increasingly popular. First, this paper introduces the development and application of PPIR technology, followed by its classification and analysis. Second, it presents the theory of four types of deep learning methods and their applications in PPIR. These methods include the convolutional neural network, deep belief network, recurrent neural network, and stacked autoencoder, and they are applied to identify plant species, diagnose plant diseases, etc. Finally, the difficulties and challenges of deep learning in PPIR are discussed.

scholarly journals A Statistics and Deep Learning Hybrid Method for Multivariate Time Series Forecasting and Mortality Modeling

Forecasting ◽  
2021 ◽  
Vol 4 (1) ◽  
pp. 1-25
Author(s):  
Thabang Mathonsi ◽  
Terence L. van Zyl
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Recurrent Neural Network ◽  
Short Term Memory ◽  
Hybrid Methods ◽  
Prediction Interval ◽  
Hybrid Approach ◽  
Computational Cost ◽  
Exponential Smoothing ◽  
Learning Methods

Hybrid methods have been shown to outperform pure statistical and pure deep learning methods at forecasting tasks and quantifying the associated uncertainty with those forecasts (prediction intervals). One example is Exponential Smoothing Recurrent Neural Network (ES-RNN), a hybrid between a statistical forecasting model and a recurrent neural network variant. ES-RNN achieves a 9.4% improvement in absolute error in the Makridakis-4 Forecasting Competition. This improvement and similar outperformance from other hybrid models have primarily been demonstrated only on univariate datasets. Difficulties with applying hybrid forecast methods to multivariate data include (i) the high computational cost involved in hyperparameter tuning for models that are not parsimonious, (ii) challenges associated with auto-correlation inherent in the data, as well as (iii) complex dependency (cross-correlation) between the covariates that may be hard to capture. This paper presents Multivariate Exponential Smoothing Long Short Term Memory (MES-LSTM), a generalized multivariate extension to ES-RNN, that overcomes these challenges. MES-LSTM utilizes a vectorized implementation. We test MES-LSTM on several aggregated coronavirus disease of 2019 (COVID-19) morbidity datasets and find our hybrid approach shows consistent, significant improvement over pure statistical and deep learning methods at forecast accuracy and prediction interval construction.

scholarly journals Facial Expression Recognition Based on Random Forest and Convolutional Neural Network

Information ◽  
2019 ◽  
Vol 10 (12) ◽  
pp. 375 ◽  
Author(s):  
Yingying Wang ◽  
Yibin Li ◽  
Yong Song ◽  
Xuewen Rong
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Random Forest ◽  
Facial Expression ◽  
Convolutional Neural Network ◽  
Facial Expressions ◽  
Facial Expression Recognition ◽  
Recognition Rate ◽  
Great Success ◽  
Expression Recognition

As an important part of emotion research, facial expression recognition is a necessary requirement in human–machine interface. Generally, a face expression recognition system includes face detection, feature extraction, and feature classification. Although great success has been made by the traditional machine learning methods, most of them have complex computational problems and lack the ability to extract comprehensive and abstract features. Deep learning-based methods can realize a higher recognition rate for facial expressions, but a large number of training samples and tuning parameters are needed, and the hardware requirement is very high. For the above problems, this paper proposes a method combining features that extracted by the convolutional neural network (CNN) with the C4.5 classifier to recognize facial expressions, which not only can address the incompleteness of handcrafted features but also can avoid the high hardware configuration in the deep learning model. Considering some problems of overfitting and weak generalization ability of the single classifier, random forest is applied in this paper. Meanwhile, this paper makes some improvements for C4.5 classifier and the traditional random forest in the process of experiments. A large number of experiments have proved the effectiveness and feasibility of the proposed method.

scholarly journals Automatic Facial Expressions and Identification of different face reactions using Convolutional Neural Network

2019 ◽  
Vol 8 (12) ◽  
pp. 5214-5218
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Deep Learning ◽  
Facial Expression ◽  
Facial Expressions ◽  
State Of The Art ◽  
Learning Tools ◽  
Face Expression ◽  
Marketing Team ◽  
Human Facial Expression

Automatic Face expression is the significant device in computer apparition and a predictable knowledge discovery application in automation, personal security and moveable devices. However, the state-of-the-art machine and deep learning (DL) methods has complete this technology game altering and even better human matching part in terms of accurateness. This paper focuses on put on one of the progressive deep learning tools in face expression to achieve higher accuracy. In this paper, we focusses on Automatic Facial Expressions and Identification of different face reactions using Convolution Neural Network. Here, we framed our own data and trained by convolution neural networks. Human behavior can be easily predicted using their facial expression, which helps marketing team, psychological team and other required team to understand the human facial expression more clearly.

scholarly journals A Novel Based 3d Facial Expression Detection Using Recurrent Neural Network

2020 ◽  
pp. 48-53
Author(s):  
Jaswanth K S ◽  
D. Stalin David
Keyword(s):  
Neural Network ◽  
Facial Expression ◽  
Facial Expressions ◽  
Social Relations ◽  
Recurrent Neural Network ◽  
Breakdown Point ◽  
Early Age ◽  
Facial Expression Classification ◽  
Expression Classification ◽  
Highlight Extraction

People periodically have diverse facial expressions and disposition changes in this way. Human facial expression acknowledgment plays a really energetic part in social relations. The acknowledgment of feelings has been an dynamic breakdown point from early age. The real-time location of facial expressions like appall, upbeat, pitiful, irate, anxious, astonish. The proposed framework can recognize 6 diverse facial expression. A facial expression acknowledgment framework needs to perform location and change to 3D image, then the facial highlight extraction, and facial expression classification is worn. Out proposed strategy we should be utilizing Recurrent Neural Network (RNN). This RNN show is prepared on JAFEE and Yale database dataset. This framework has capacity to screen individuals’ feelings, to segregate between feelings and name them fittingly.

scholarly journals Real Time Facial Gesture Recognition using Deep Learning

2021 ◽  
Vol 9 (VI) ◽  
pp. 41-49
Author(s):  
Abaikesh Sharma
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Computer Vision ◽  
Deep Learning ◽  
Facial Expression ◽  
Real Time ◽  
Facial Expressions ◽  
Deep Neural Network ◽  
Discrete Emotion ◽  
Human Faces

The human faces have vibrant frequency of characteristics, which makes it difficult to analyze the facial expression. Automated real time emotions recognition with the help of facial expressions is a work in computer vision. This environment is an important and interesting tool between the humans and computers. In this investigation an environment is created which is capable of analyzing the person’s emotions using the real time facial gestures with the help of Deep Neural Network. It can detect the facial expression from any image either real or animated after facial extraction (muscle position, eye expression and lips position). This system is setup to classify images of human faces into seven discrete emotion categories using Convolutional Neural Networks (CNNs). This type of environment is important for social interaction.

Research on behavior recognition based on feature fusion of automatic coder and recurrent neural network

2020 ◽  
Vol 39 (6) ◽  
pp. 8927-8935
Author(s):  
Bing Zheng ◽  
Dawei Yun ◽  
Yan Liang
Keyword(s):  
Neural Network ◽  
Recurrent Neural Network ◽  
Behavior Pattern ◽  
Rapid Development ◽  
Video Data ◽  
Support Vector ◽  
Behavior Recognition ◽  
Learning Methods ◽  
The Impact ◽  
Internet Of Things Technology

Under the impact of COVID-19, research on behavior recognition are highly needed. In this paper, we combine the algorithm of self-adaptive coder and recurrent neural network to realize the research of behavior pattern recognition. At present, most of the research of human behavior recognition is focused on the video data, which is based on the video number. At the same time, due to the complexity of video image data, it is easy to violate personal privacy. With the rapid development of Internet of things technology, it has attracted the attention of a large number of experts and scholars. Researchers have tried to use many machine learning methods, such as random forest, support vector machine and other shallow learning methods, which perform well in the laboratory environment, but there is still a long way to go from practical application. In this paper, a recursive neural network algorithm based on long and short term memory (LSTM) is proposed to realize the recognition of behavior patterns, so as to improve the accuracy of human activity behavior recognition.

scholarly journals Audio-Based Drone Detection and Identification Using Deep Learning Techniques with Dataset Enhancement through Generative Adversarial Networks

Sensors ◽  
2021 ◽  
Vol 21 (15) ◽  
pp. 4953
Author(s):  
Sara Al-Emadi ◽  
Abdulla Al-Ali ◽  
Abdulaziz Al-Ali
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Recurrent Neural Network ◽  
Learning Algorithms ◽  
Generative Adversarial Networks ◽  
Generative Adversarial Network ◽  
Adversarial Networks ◽  
Detection And Identification ◽  
Learning Techniques ◽  
The Impact

Drones are becoming increasingly popular not only for recreational purposes but in day-to-day applications in engineering, medicine, logistics, security and others. In addition to their useful applications, an alarming concern in regard to the physical infrastructure security, safety and privacy has arisen due to the potential of their use in malicious activities. To address this problem, we propose a novel solution that automates the drone detection and identification processes using a drone’s acoustic features with different deep learning algorithms. However, the lack of acoustic drone datasets hinders the ability to implement an effective solution. In this paper, we aim to fill this gap by introducing a hybrid drone acoustic dataset composed of recorded drone audio clips and artificially generated drone audio samples using a state-of-the-art deep learning technique known as the Generative Adversarial Network. Furthermore, we examine the effectiveness of using drone audio with different deep learning algorithms, namely, the Convolutional Neural Network, the Recurrent Neural Network and the Convolutional Recurrent Neural Network in drone detection and identification. Moreover, we investigate the impact of our proposed hybrid dataset in drone detection. Our findings prove the advantage of using deep learning techniques for drone detection and identification while confirming our hypothesis on the benefits of using the Generative Adversarial Networks to generate real-like drone audio clips with an aim of enhancing the detection of new and unfamiliar drones.

scholarly journals Facial Expression Recognition Based on Multi-Features Cooperative Deep Convolutional Network

2021 ◽  
Vol 11 (4) ◽  
pp. 1428
Author(s):  
Haopeng Wu ◽  
Zhiying Lu ◽  
Jianfeng Zhang ◽  
Xin Li ◽  
Mingyue Zhao ◽  
...  
Keyword(s):  
Facial Expression ◽  
Facial Expressions ◽  
Facial Expression Recognition ◽  
Video Data ◽  
Expression Recognition ◽  
Convolutional Network ◽  
Facial Movements ◽  
The Face ◽  
Deep Convolutional Network ◽  
Selection Of

This paper addresses the problem of Facial Expression Recognition (FER), focusing on unobvious facial movements. Traditional methods often cause overfitting problems or incomplete information due to insufficient data and manual selection of features. Instead, our proposed network, which is called the Multi-features Cooperative Deep Convolutional Network (MC-DCN), maintains focus on the overall feature of the face and the trend of key parts. The processing of video data is the first stage. The method of ensemble of regression trees (ERT) is used to obtain the overall contour of the face. Then, the attention model is used to pick up the parts of face that are more susceptible to expressions. Under the combined effect of these two methods, the image which can be called a local feature map is obtained. After that, the video data are sent to MC-DCN, containing parallel sub-networks. While the overall spatiotemporal characteristics of facial expressions are obtained through the sequence of images, the selection of keys parts can better learn the changes in facial expressions brought about by subtle facial movements. By combining local features and global features, the proposed method can acquire more information, leading to better performance. The experimental results show that MC-DCN can achieve recognition rates of 95%, 78.6% and 78.3% on the three datasets SAVEE, MMI, and edited GEMEP, respectively.

Sign in / Sign up

Export Citation Format

Share Document