Three-Dimensional Evaluation of Soft Tissue Malar Modifications after Zygomatic Valgization Osteotomy via Geometrical Descriptors

Patients with severe facial deformities present serious dysfunctionalities along with an unsatisfactory aesthetic facial appearance. Several methods have been proposed to specifically plan the interventions on the patient’s needs, but none of these seem to achieve a sufficient level of accuracy in predicting the resulting facial appearance. In this context, a deep knowledge of what occurs in the face after bony movements in specific surgeries would give the possibility to develop more reliable systems. This study aims to propose a novel 3D approach for the evaluation of soft tissue zygomatic modifications after zygomatic osteotomy; geometrical descriptors usually involved in face analysis tasks, i.e., face recognition and facial expression recognition, are here applied to soft tissue malar region to detect changes in surface shape. As ground truth for zygomatic changes, a zygomatic openness angular measure is adopted. The results show a high sensibility of geometrical descriptors in detecting shape modification of the facial surface, outperforming the results obtained from the angular evaluation.

Download Full-text

Deep-Emotion: Facial Expression Recognition Using Attentional Convolutional Network

Sensors ◽

10.3390/s21093046 ◽

2021 ◽

Vol 21 (9) ◽

pp. 3046

Author(s):

Shervin Minaee ◽

Mehdi Minaei ◽

Amirali Abdolrashidi

Keyword(s):

Deep Learning ◽

Facial Expression ◽

Facial Expression Recognition ◽

Expression Recognition ◽

Visualization Technique ◽

Convolutional Network ◽

The Past ◽

Multiple Datasets ◽

The Face ◽

Traditional Approaches

Facial expression recognition has been an active area of research over the past few decades, and it is still challenging due to the high intra-class variation. Traditional approaches for this problem rely on hand-crafted features such as SIFT, HOG, and LBP, followed by a classifier trained on a database of images or videos. Most of these works perform reasonably well on datasets of images captured in a controlled condition but fail to perform as well on more challenging datasets with more image variation and partial faces. In recent years, several works proposed an end-to-end framework for facial expression recognition using deep learning models. Despite the better performance of these works, there are still much room for improvement. In this work, we propose a deep learning approach based on attentional convolutional network that is able to focus on important parts of the face and achieves significant improvement over previous models on multiple datasets, including FER-2013, CK+, FERG, and JAFFE. We also use a visualization technique that is able to find important facial regions to detect different emotions based on the classifier’s output. Through experimental results, we show that different emotions are sensitive to different parts of the face.

Download Full-text

Facial Expression Recognition Based on Multi-Features Cooperative Deep Convolutional Network

Applied Sciences ◽

10.3390/app11041428 ◽

2021 ◽

Vol 11 (4) ◽

pp. 1428

Author(s):

Haopeng Wu ◽

Zhiying Lu ◽

Jianfeng Zhang ◽

Xin Li ◽

Mingyue Zhao ◽

...

Keyword(s):

Facial Expression ◽

Facial Expressions ◽

Facial Expression Recognition ◽

Video Data ◽

Expression Recognition ◽

Convolutional Network ◽

Facial Movements ◽

The Face ◽

Deep Convolutional Network ◽

Selection Of

This paper addresses the problem of Facial Expression Recognition (FER), focusing on unobvious facial movements. Traditional methods often cause overfitting problems or incomplete information due to insufficient data and manual selection of features. Instead, our proposed network, which is called the Multi-features Cooperative Deep Convolutional Network (MC-DCN), maintains focus on the overall feature of the face and the trend of key parts. The processing of video data is the first stage. The method of ensemble of regression trees (ERT) is used to obtain the overall contour of the face. Then, the attention model is used to pick up the parts of face that are more susceptible to expressions. Under the combined effect of these two methods, the image which can be called a local feature map is obtained. After that, the video data are sent to MC-DCN, containing parallel sub-networks. While the overall spatiotemporal characteristics of facial expressions are obtained through the sequence of images, the selection of keys parts can better learn the changes in facial expressions brought about by subtle facial movements. By combining local features and global features, the proposed method can acquire more information, leading to better performance. The experimental results show that MC-DCN can achieve recognition rates of 95%, 78.6% and 78.3% on the three datasets SAVEE, MMI, and edited GEMEP, respectively.

Download Full-text

Hybrid Attention Cascade Network for Facial Expression Recognition

Sensors ◽

10.3390/s21062003 ◽

2021 ◽

Vol 21 (6) ◽

pp. 2003 ◽

Cited By ~ 1

Author(s):

Xiaoliang Zhu ◽

Shihao Ye ◽

Liang Zhao ◽

Zhicheng Dai

Keyword(s):

Facial Expression ◽

Facial Expressions ◽

Facial Expression Recognition ◽

Expression Recognition ◽

Spatial Features ◽

Face Images ◽

Temporal Features ◽

The Face ◽

In The Wild ◽

Fusion Features

As a sub-challenge of EmotiW (the Emotion Recognition in the Wild challenge), how to improve performance on the AFEW (Acted Facial Expressions in the wild) dataset is a popular benchmark for emotion recognition tasks with various constraints, including uneven illumination, head deflection, and facial posture. In this paper, we propose a convenient facial expression recognition cascade network comprising spatial feature extraction, hybrid attention, and temporal feature extraction. First, in a video sequence, faces in each frame are detected, and the corresponding face ROI (range of interest) is extracted to obtain the face images. Then, the face images in each frame are aligned based on the position information of the facial feature points in the images. Second, the aligned face images are input to the residual neural network to extract the spatial features of facial expressions corresponding to the face images. The spatial features are input to the hybrid attention module to obtain the fusion features of facial expressions. Finally, the fusion features are input in the gate control loop unit to extract the temporal features of facial expressions. The temporal features are input to the fully connected layer to classify and recognize facial expressions. Experiments using the CK+ (the extended Cohn Kanade), Oulu-CASIA (Institute of Automation, Chinese Academy of Sciences) and AFEW datasets obtained recognition accuracy rates of 98.46%, 87.31%, and 53.44%, respectively. This demonstrated that the proposed method achieves not only competitive performance comparable to state-of-the-art methods but also greater than 2% performance improvement on the AFEW dataset, proving the significant outperformance of facial expression recognition in the natural environment.

Download Full-text

Computer Intelligent Nursing Care of Critically III Patients Based on Improved Patient Facial Expression Recognition Method

Applied Mechanics and Materials ◽

10.4028/www.scientific.net/amm.543-547.2350 ◽

2014 ◽

Vol 543-547 ◽

pp. 2350-2353

Author(s):

Xiao Yan Wan

Keyword(s):

Facial Expression ◽

Critically Ill ◽

Nursing Care ◽

Facial Expression Recognition ◽

Recognition Rate ◽

Support Vector ◽

Appearance Model ◽

Expression Recognition ◽

Recognition Method ◽

The Face

In order to extract the expression features of critically ill patients, and realize the computer intelligent nursing, an improved facial expression recognition method is proposed based on the of active appearance model, the support vector machine (SVM) for facial expression recognition is taken in research, and the face recognition model structure active appearance model is designed, and the attribute reduction algorithm of rough set affine transformation theory is introduced, and the invalid and redundant feature points are removed. The critically ill patient expressions are classified and recognized based on the support vector machine (SVM). The face image attitudes are adjusted, and the self-adaptive performance of facial expression recognition for the critical patient attitudes is improved. New method overcomes the effect of patient attitude to the recognition rate to a certain extent. The highest average recognition rate can be increased about 7%. The intelligent monitoring and nursing care of critically ill patients are realized with the computer vision effect. The nursing quality is enhanced, and it ensures the timely treatment of rescue.

Download Full-text

Facial Appearance and Psychosocial Features in Orthognathic Surgery: A FACE-Q- and 3D Facial Image-Based Comparative Study of Patient-, Clinician-, and Lay-Observer-Reported Outcomes

Journal of Clinical Medicine ◽

10.3390/jcm8060909 ◽

2019 ◽

Vol 8 (6) ◽

pp. 909 ◽

Cited By ~ 10

Author(s):

Rafael Denadai ◽

Pang-Yun Chou ◽

Yu-Ying Su ◽

Chi-Chin Lo ◽

Hsiu-Hsia Lin ◽

...

Keyword(s):

Orthognathic Surgery ◽

Outcome Measurement ◽

Three Dimensional ◽

Assessment Tools ◽

Multidisciplinary Teams ◽

Facial Appearance ◽

Patient Centered ◽

Centered Care ◽

The Face ◽

Decision Making Processes

Outcome measures reported by patients, clinicians, and lay-observers can help to tailor treatment plans to meet patients’ needs. This study evaluated orthognathic surgery (OGS) outcomes using pre- and post-OGS patients’ (n = 84) FACE-Q reports, and a three-dimensional facial photograph-based panel assessment of facial appearance and psychosocial parameters, with 96 blinded layperson and orthodontic and surgical professional raters, and verified whether there were correlations between these outcome measurement tools. Post-OGS FACE-Q and panel assessment measurements showed significant (p < 0.001) differences from pre-OGS measurements. Pre-OGS patients’ FACE-Q scores were significantly (p < 0.01) lower than normal, age-, gender-, and ethnicity-matched individuals’ (n = 54) FACE-Q scores, with no differences in post-OGS comparisons. The FACE-Q overall facial appearance scale had a low, statistically significant (p < 0.001) correlation to the facial-aesthetic-based panel assessment, but no correlation to the FACE-Q lower face and lips scales. No significant correlation was observed between the FACE-Q and panel assessment psychosocial-related scales. This study demonstrates that OGS treatment positively influences the facial appearance and psychosocial-related perceptions of patients, clinicians and lay observers, but that there is only a low, or no, correlation between the FACE-Q and panel assessment tools. Future investigations may consider the inclusion of both tools as OGS treatment endpoints for the improvement of patient-centered care, and guiding the health-system-related decision-making processes of multidisciplinary teams, policymakers, and other stakeholders.

Download Full-text

Face and Eye Detection

Automated Face Analysis ◽

10.4018/978-1-60566-216-9.ch002 ◽

2011 ◽

pp. 5-44 ◽

Cited By ~ 1

Author(s):

Daijin Kim ◽

Jaewon Sung

Keyword(s):

Face Detection ◽

Facial Expression Recognition ◽

Face Image ◽

Face Tracking ◽

Expression Recognition ◽

Human Face ◽

Eye Detection ◽

Face Image Analysis ◽

Face Images ◽

The Face

Face detection is the most fundamental step for the research on image-based automated face analysis such as face tracking, face recognition, face authentication, facial expression recognition and facial gesture recognition. When a novel face image is given we must know where the face is located, and how large the scale is to limit our concern to the face patch in the image and normalize the scale and orientation of the face patch. Usually, the face detection results are not stable; the scale of the detected face rectangle can be larger or smaller than that of the real face in the image. Therefore, many researchers use eye detectors to obtain stable normalized face images. Because the eyes have salient patterns in the human face image, they can be located stably and used for face image normalization. The eye detection becomes more important when we want to apply model-based face image analysis approaches.

Download Full-text

3D Facial Expression Recognition for Defining Users’ Inner Requirements—An Emotional Design Case Study

Applied Sciences ◽

10.3390/app9112218 ◽

2019 ◽

Vol 9 (11) ◽

pp. 2218 ◽

Cited By ~ 7

Author(s):

Maria Grazia Violante ◽

Federica Marcolin ◽

Enrico Vezzetti ◽

Luca Ulrich ◽

Gianluca Billia ◽

...

Keyword(s):

Facial Expression ◽

Facial Expressions ◽

Facial Expression Recognition ◽

Quality Function Deployment ◽

Three Dimensional ◽

Support Vector ◽

Expression Recognition ◽

Emotional Design ◽

3D Data

This study proposes a novel quality function deployment (QFD) design methodology based on customers’ emotions conveyed by facial expressions. The current advances in pattern recognition related to face recognition techniques have fostered the cross-fertilization and pollination between this context and other fields, such as product design and human-computer interaction. In particular, the current technologies for monitoring human emotions have supported the birth of advanced emotional design techniques, whose main focus is to convey users’ emotional feedback into the design of novel products. As quality functional deployment aims at transforming the voice of customers into engineering features of a product, it appears to be an appropriate and promising nest in which to embed users’ emotional feedback with new emotional design methodologies, such as facial expression recognition. This way, the present methodology consists in interviewing the user and acquiring his/her face with a depth camera (allowing three-dimensional (3D) data), clustering the face information into different emotions with a support vector machine classificator, and assigning customers’ needs weights relying on the detected facial expressions. The proposed method has been applied to a case study in the context of agriculture and validated by a consortium. The approach appears sound and capable of collecting the unconscious feedback of the interviewee.

Download Full-text

Three-Dimensional Analysis Techniques—Part 3: Color-Coded System for Three-Dimensional Measurement of Bone and Ratio of Soft Tissue to Bone: The Analysis

The Cleft Palate-Craniofacial Journal ◽

10.1597/1545-1569_1997_034_0052_tdatpc_2.3.co_2 ◽

1997 ◽

Vol 34 (1) ◽

pp. 52-57 ◽

Cited By ~ 10

Author(s):

Andrew M. Mccance ◽

James P. Moss ◽

W. Rick Fright ◽

Alf D. Linney

Keyword(s):

Soft Tissue ◽

Soft Tissues ◽

Three Dimensional ◽

Ct Scans ◽

Dimensional Changes ◽

Analysis Techniques ◽

Dimensional Measurement ◽

Three Dimensional Measurement ◽

The Face ◽

Bimaxillary Surgery

A new color-coded method of illustrating three-dimensional changes in the bone and the ratio of soft tissue to bone movement is described. The technique is illustrated by superimposing preoperative and 1-year postoperative CT scans of three patients following bimaxillary surgery. The method has proved to be a very simple, effective, and readily interpreted method of quantifying both bone and the ratio of movement of the overlying soft tissues across the face following surgery.

Download Full-text

A Facial Expression Recognition Method Using Improved Capsule Network Model

Scientific Programming ◽

10.1155/2020/8845176 ◽

2020 ◽

Vol 2020 ◽

pp. 1-12

Author(s):

Yifeng Zhao ◽

Deyun Chen

Keyword(s):

Facial Expression ◽

Network Model ◽

Facial Expression Recognition ◽

Recognition Rate ◽

Regression Tree ◽

Gaussian Process Regression ◽

Learning Technology ◽

Expression Recognition ◽

Recognition Method ◽

The Face

Aiming at the problem of facial expression recognition under unconstrained conditions, a facial expression recognition method based on an improved capsule network model is proposed. Firstly, the expression image is normalized by illumination based on the improved Weber face, and the key points of the face are detected by the Gaussian process regression tree. Then, the 3dmms model is introduced. The 3D face shape, which is consistent with the face in the image, is provided by iterative estimation so as to further improve the image quality of face pose standardization. In this paper, we consider that the convolution features used in facial expression recognition need to be trained from the beginning and add as many different samples as possible in the training process. Finally, this paper attempts to combine the traditional deep learning technology with capsule configuration, adds an attention layer after the primary capsule layer in the capsule network, and proposes an improved capsule structure model suitable for expression recognition. The experimental results on JAFFE and BU-3DFE datasets show that the recognition rate can reach 96.66% and 80.64%, respectively.

Download Full-text

A Study on Improved Real-Time Eye Region Detection Method for More Realistic Facial Expressions Recognition

Advanced Materials Research ◽

10.4028/www.scientific.net/amr.268-270.471 ◽

2011 ◽

Vol 268-270 ◽

pp. 471-475

Author(s):

Sungmo Jung ◽

Seoksoo Kim

Keyword(s):

Facial Expressions ◽

Facial Expression Recognition ◽

Detection Method ◽

Expression Recognition ◽

Improved Method ◽

Region Detection ◽

Recognition Of Facial Expressions ◽

The Face ◽

Facial Expressions Recognition ◽

Eye Region

Many 3D films use technologies of facial expression recognition. In order to use the existing technologies, a large number of markers shall be attached to a face, a camera is fixed in front of the face, and movements of the markers are calculated. However, the markers calculate only the changes in regions where the markers are attached, which makes difficult realistic recognition of facial expressions. Therefore, this study extracted a preliminary eye region in 320*240 by defining specific location values of the eye. And the final eye region was selected from the preliminary region. This study suggests an improved method of detecting an eye region, reducing errors arising from noise.

Download Full-text