scholarly journals Pairwise Ranking Distillation for Deep Face Recognition

2020 ◽  
pp. paper30-1-paper30-13
Author(s):  
Mikhail Nikitin ◽  
Vadim Konushin ◽  
Anton Konushin
Keyword(s):  
Face Recognition ◽  
Metric Learning ◽  
High Capacity ◽  
Recognition Task ◽  
Learning Task ◽  
Face Verification ◽  
Baseline Method ◽  
Task Knowledge ◽  
Model Compression ◽  
Knowledge Distillation

This work addresses the problem of knowledge distillation for deep face recognition task. Knowledge distillation technique is known to be an effective way of model compression, which implies transferring of the knowledge from high-capacity teacher to a lightweight student. The knowledge and the way how it is distilled can be defined in different ways depending on the problem where the technique is applied. Considering the fact that face recognition is a typical metric learning task, we propose to perform knowledge distillation on a score-level. Specifically, for any pair of matching scores computed by teacher, our method forces student to have the same order for the corresponding matching scores. We evaluate proposed pairwise ranking distillation (PWR) approach using several face recognition benchmarks for both face verification and face identification scenarios. Experimental results show that PWR not only can improve over the baseline method by a large margin, but also outperforms other score-level distillation approaches.

Discover the Effective Strategy for Face Recognition Model Compression by Improved Knowledge Distillation

2018 ◽  
Author(s):  
Mengjiao Wang ◽  
Rujie Liu ◽  
Narishige Abe ◽  
Hidetsugu Uchida ◽  
Tomoaki Matsunami ◽  
...  
Keyword(s):  
Face Recognition ◽  
Effective Strategy ◽  
Recognition Model ◽  
Model Compression ◽  
Knowledge Distillation

scholarly journals xCos: An Explainable Cosine Metric for Face Verification Task

2021 ◽  
Vol 17 (3s) ◽  
pp. 1-16
Author(s):  
Yu-Sheng Lin ◽  
Zhe-Yu Liu ◽  
Yu-An Chen ◽  
Yu-Siang Wang ◽  
Ya-Liang Chang ◽  
...  
Keyword(s):  
Neural Networks ◽  
Face Recognition ◽  
Recognition Task ◽  
High Accuracy ◽  
Face Verification ◽  
Verification Task ◽  
Deep Convolutional Neural Networks ◽  
Explainable Ai ◽  
The Face ◽  
Very High

We study the XAI (explainable AI) on the face recognition task, particularly the face verification. Face verification has become a crucial task in recent days and it has been deployed to plenty of applications, such as access control, surveillance, and automatic personal log-on for mobile devices. With the increasing amount of data, deep convolutional neural networks can achieve very high accuracy for the face verification task. Beyond exceptional performances, deep face verification models need more interpretability so that we can trust the results they generate. In this article, we propose a novel similarity metric, called explainable cosine ( xCos ), that comes with a learnable module that can be plugged into most of the verification models to provide meaningful explanations. With the help of xCos , we can see which parts of the two input faces are similar, where the model pays its attention to, and how the local similarities are weighted to form the output xCos score. We demonstrate the effectiveness of our proposed method on LFW and various competitive benchmarks, not only resulting in providing novel and desirable model interpretability for face verification but also ensuring the accuracy as plugging into existing face recognition models.

scholarly journals Knowledge Distillation Beyond Model Compression

2021 ◽  
Author(s):  
Fahad Sarfraz ◽  
Elahe Arani ◽  
Bahram Zonooz
Keyword(s):  
Model Compression ◽  
Knowledge Distillation

scholarly journals Low-Resolution Face Recognition in the Wild via Selective Knowledge Distillation

2019 ◽  
Vol 28 (4) ◽  
pp. 2051-2062 ◽  
Author(s):  
Shiming Ge ◽  
Shengwei Zhao ◽  
Chenyu Li ◽  
Jia Li
Keyword(s):  
Face Recognition ◽  
Low Resolution ◽  
In The Wild ◽  
Knowledge Distillation

scholarly journals Multi-Task Learning Using Task Dependencies for Face Attributes Prediction

2019 ◽  
Vol 9 (12) ◽  
pp. 2535
Author(s):  
Di Fan ◽  
Hyunwoo Kim ◽  
Jummo Kim ◽  
Yunhui Liu ◽  
Qiang Huang
Keyword(s):  
Human Computer Interaction ◽  
Video Surveillance ◽  
State Of The Art ◽  
Learning Task ◽  
Face Verification ◽  
Task Learning ◽  
The World ◽  
In The Wild ◽  
And Gender ◽  
Learning Architectures

Face attributes prediction has an increasing amount of applications in human–computer interaction, face verification and video surveillance. Various studies show that dependencies exist in face attributes. Multi-task learning architecture can build a synergy among the correlated tasks by parameter sharing in the shared layers. However, the dependencies between the tasks have been ignored in the task-specific layers of most multi-task learning architectures. Thus, how to further boost the performance of individual tasks by using task dependencies among face attributes is quite challenging. In this paper, we propose a multi-task learning using task dependencies architecture for face attributes prediction and evaluate the performance with the tasks of smile and gender prediction. The designed attention modules in task-specific layers of our proposed architecture are used for learning task-dependent disentangled representations. The experimental results demonstrate the effectiveness of our proposed network by comparing with the traditional multi-task learning architecture and the state-of-the-art methods on Faces of the world (FotW) and Labeled faces in the wild-a (LFWA) datasets.

scholarly journals Combine-Net: An Improved Filter Pruning Algorithm

Information ◽  
2021 ◽  
Vol 12 (7) ◽  
pp. 264
Author(s):  
Jinghan Wang ◽  
Guangyue Li ◽  
Wenzhao Zhang
Keyword(s):  
Structured Model ◽  
Compression Algorithms ◽  
Pruning Algorithm ◽  
Model Compression ◽  
The Neural Network ◽  
Empirical Determination ◽  
Knowledge Distillation ◽  
Resource Constrained Devices ◽  
Constrained Devices

The powerful performance of deep learning is evident to all. With the deepening of research, neural networks have become more complex and not easily generalized to resource-constrained devices. The emergence of a series of model compression algorithms makes artificial intelligence on edge possible. Among them, structured model pruning is widely utilized because of its versatility. Structured pruning prunes the neural network itself and discards some relatively unimportant structures to compress the model’s size. However, in the previous pruning work, problems such as evaluation errors of networks, empirical determination of pruning rate, and low retraining efficiency remain. Therefore, we propose an accurate, objective, and efficient pruning algorithm—Combine-Net, introducing Adaptive BN to eliminate evaluation errors, the Kneedle algorithm to determine the pruning rate objectively, and knowledge distillation to improve the efficiency of retraining. Results show that, without precision loss, Combine-Net achieves 95% parameter compression and 83% computation compression on VGG16 on CIFAR10, 71% of parameter compression and 41% computation compression on ResNet50 on CIFAR100. Experiments on different datasets and models have proved that Combine-Net can efficiently compress the neural network’s parameters and computation.

Data-Free Ensemble Knowledge Distillation for Privacy-conscious Multimedia Model Compression

2021 ◽  
Author(s):  
Zhiwei Hao ◽  
Yong Luo ◽  
Han Hu ◽  
Jianping An ◽  
Yonggang Wen
Keyword(s):  
Multimedia Model ◽  
Model Compression ◽  
Knowledge Distillation

Revisiting knowledge distillation for light-weight visual object detection

2021 ◽  
Vol 43 (13) ◽  
pp. 2888-2898
Author(s):  
Tianze Gao ◽  
Yunfeng Gao ◽  
Yu Li ◽  
Peiyuan Qin
Keyword(s):  
Object Detection ◽  
Essential Element ◽  
Detection Algorithm ◽  
Positive Sample ◽  
Detection Methods ◽  
Visual Object ◽  
Light Weight ◽  
Model Compression ◽  
Novel Approach ◽  
Knowledge Distillation

An essential element for intelligent perception in mechatronic and robotic systems (M&RS) is the visual object detection algorithm. With the ever-increasing advance of artificial neural networks (ANN), researchers have proposed numerous ANN-based visual object detection methods that have proven to be effective. However, networks with cumbersome structures do not befit the real-time scenarios in M&RS, necessitating the techniques of model compression. In the paper, a novel approach to training light-weight visual object detection networks is developed by revisiting knowledge distillation. Traditional knowledge distillation methods are oriented towards image classification is not compatible with object detection. Therefore, a variant of knowledge distillation is developed and adapted to a state-of-the-art keypoint-based visual detection method. Two strategies named as positive sample retaining and early distribution softening are employed to yield a natural adaption. The mutual consistency between teacher model and student model is further promoted through a hint-based distillation. By extensive controlled experiments, the proposed method is testified to be effective in enhancing the light-weight network’s performance by a large margin.

The effect of interpersonal multisensory stimulation on the self-face recognition in adults with autistic syndrome disorder

2016 ◽  
Vol 33 (S1) ◽  
pp. S367-S368
Author(s):  
N. Deltort ◽  
J.R. Cazalets ◽  
A. Amestoy ◽  
M. Bouvard
Keyword(s):  
Face Recognition ◽  
Mental Representation ◽  
Recognition Task ◽  
The Self ◽  
Control Group ◽  
Multimodal Integration ◽  
Multisensory Stimulation ◽  
Significant Difference ◽  
Unfamiliar Face ◽  
Multimodal Process

Studies on individuals without developmental disorder show that mental representation of self-face is subject to a multimodal process in the same way that the representation of the self-body is. People with autistic spectrum disorder (ASD) have a particular pattern of face processing and a multimodal integration deficit.The objectives of our study were to evaluate the self-face recognition and the effect of interpersonal multisensory stimulation (IMS) in individuals with ASD. We aimed to show a self-face recognition deficit and a lack of multimodal integration among this population.IMS consisted of the presentation of a movie displaying an unfamiliar face being touched intermittently, while the examiner applied the same stimulation synchronously or asynchronously on the participant. The effect resulting from IMS was measured on two groups with or without ASD by a self-face recognition task on morphing movies made from self-face and unfamiliar-face pictures.There was a significant difference between groups on self-recognition before stimulation. This result shows a self-face recognition deficit in individuals with ASD. Results for the control group showed a significant effect of IMS on self-face recognition in synchronous condition. This suggests the existence of an update of self-face mental representation by multimodal process. In contrast, there was no significant effect of IMS demonstrated in ASD group, suggesting a multimodal integration deficit for the constitution of self-representation in this population.Our results show the existence of a self-face recognition deficit in individuals with ASD, which may be linked to a lack of multimodal integration in the development of the self-face representation.Disclosure of interestThe authors have not supplied their declaration of competing interest.

Joint Bayesian guided metric learning for end-to-end face verification

2018 ◽  
Vol 275 ◽  
pp. 560-567 ◽  
Author(s):  
Di Chen ◽  
Chunyan Xu ◽  
Jian Yang ◽  
Jianjun Qian ◽  
Yuhui Zheng ◽  
...  
Keyword(s):  
Metric Learning ◽  
Face Verification ◽  
End To End
Sign in / Sign up

Export Citation Format

Share Document