Automatic Face Image Tagging in Large Collections

In this chapter, the authors present some issues related to automatic face image tagging techniques. Their main purpose in user applications is to support the organization (indexing) and retrieval (or easy browsing) of images or videos in large collections. Their core modules include algorithms and strategies for handling very large face databases, mostly acquired in real conditions. As a background for understanding how automatic face tagging works, an overview about face recognition techniques is given, including both traditional approaches and novel proposed techniques for face recognition in uncontrolled settings. Moreover, some applications and the way they work are summarized, in order to depict the state of the art in this area of face recognition research. Actually, many of them are used to tag faces and to organize photo albums with respect to the person(s) presented in annotated photos. This kind of activity has recently expanded from personal devices to social networks, and can also significantly support more demanding tasks, such as automatic handling of large editorial collections for magazine publishing and archiving. Finally, a number of approaches to large-scale face datasets as well as some automatic face image tagging techniques are presented and compared. The authors show that many approaches, both in commercial and research applications, still provide only a semi-automatic solution for this problem.

Local vs. Global

Face recognition in real world applications is a very difficult task because of image misalignments, pose and illumination variations, or occlusions. Many researchers in this field have investigated both face representation and classification techniques able to deal with these drawbacks. However, none of them is free from limitations. Early proposed algorithms were generally holistic, in the sense they consider the face object as a whole. Recently, challenging benchmarks demonstrated that they are not adequate to be applied in unconstrained environments, despite of their good performances in more controlled conditions. Therefore, the researchers' attention is now turning on local features that have been demonstrated to be more robust to a large set of non-monotonic distortions. Nevertheless, though local operators partially overcome some drawbacks, they are still opening new questions (e.g., Which criteria should be used to select the most representative features?). This is the reason why, among all the others, hybrid approaches are showing a high potential in terms of recognition accuracy when applied in uncontrolled settings, as they integrate complementary information from both local and global features. This chapter explores local, global, and hybrid approaches.

Human Face Region Detection Driving Activity Recognition in Video

Automatic recognition of human actions from video signals is probably one of the most salient research topics of computer vision with a tremendous impact for many applications. In this chapter, the authors introduce a new descriptor, the Human Constrained Pixel Change History (HC-PCH), which is based on PCH but focuses on the human body movements over time. They propose a modification of the conventional PCH that entails the calculation of two probabilistic maps based on human face and body detection, respectively. These HC-PCH features are used as input to an HMM-based classification framework, which exploits redundant information from multiple streams by employing sophisticated fusion methods, resulting in enhanced activity recognition rates.

Facial Action Recognition in 2D and 3D

Automatic Facial Expression Analysis systems have come a long way since the earliest approaches in the early 1970s. We are now at a point where the first systems are commercially applied, most notably smile detectors included in digital cameras. As one of the most comprehensive and objective ways to describe facial expressions, the Facial Action Coding System (FACS) has received significant and sustained attention within the field. Over the past 30 years, psychologists and neuroscientists have conducted extensive research on various aspects of human behaviour using facial expression analysis coded in terms of FACS. Automating FACS coding would make this research faster and more widely applicable, opening up new avenues to understanding how we communicate through facial expressions. Mainly due to the cost effectiveness of existing recording equipment, until recently almost all work conducted in this area involves 2D imagery, despite their inherent problems relating to pose and illumination variations. In order to deal with these problems, 3D recordings are increasingly used in expression analysis research. In this chapter, the authors give an overview of 2D and 3D FACS recognition, and summarise current challenges and opportunities.

Methods and Perspectives in Face Tracking Based on Human Perception

This chapter aims at analyzing the role of human early vision in image and video processing, with particular reference to face perception, recognition, and tracking. To this aim, the change of perspective in approaching image processing-based problems where the decoder (human eye) plays a central role is analysed and discussed. In particular, the main topics of this contribution are some important neurological results that have been successfully used in face detection and recognition, as well as those that seem to be promising in giving new and powerful tools for face tracking, which remains a less investigated topic from this new standpoint.

Fusion of Face Recognition Classifiers under Adverse Conditions

A face acquired by recognition systems is invariably subject to environmental and sensing conditions, which may change over time. This may have a significant negative impact on the accuracy of recognition algorithms. In the past, these problems have been tackled by building in invariance to the various changes, by adaptation, and by multiple expert systems. More recently, the possibility of enhancing the pattern classification system robustness by using auxiliary information has been explored. In particular, by measuring the extent of degradation, the resulting sensory data quality information can be used to combat the effect of the degradation phenomena. This can be achieved by using the auxiliary quality information as features in the fusion stage of a multiple classifier system, which uses the discriminant function values from the first stage as inputs. Data quality can be measured directly from the sensory data. Different architectures are suggested in this chapter for decision making using quality information. Examples of these architectures are presented and their relative merits discussed. The problems and benefits associated with the use of auxiliary information in sensory data analysis are illustrated on the problem of personal identity verification used in biometrics.

3D Face Recognition in the Presence of Partial Data

Performing face recognition under extreme poses and lighting conditions remains a challenging task for current state-of-the-art biometric algorithms. The recognition task is even more challenging when there is insufficient training data available in the gallery, or when the gallery dataset originates from one side of the face while the probe dataset originates from the other. The authors present a new method for computing the distance between two biometric signatures acquired under such challenging conditions. This method improves upon an existing Semi-Coupled Dictionary Learning method by computing a jointly-optimized solution that incorporates the reconstruction cost, the discrimination cost, and the semi-coupling cost. The use of a semi-coupling term allows the method to handle partial 3D face meshes where, for example, only the left side of the face is available for gallery and the right side of the face is available for probe. The method also extends to 2D signatures under varying poses and lighting changes by using 3D signatures as a coupling term. The experiments show that this method can improve recognition performance of existing state-of-the-art wavelet signatures used in 3D face recognition and provide excellent recognition results in the 3D-2D face recognition application.

Real World Applications

The analysis of face images has been extensively applied for the recognition of individuals in several application domains. Most notably, faces not only convey information about the identity of the subject, but also a number of ancillary information, which may be equally useful to anonymously determine the characteristics of an individual. Even though the first applications of face recognition have been related to security and access control, nowadays the analysis of human faces is related to several applications including law enforcement, man-machine interaction, and robotics, just to mention a few. This chapter explores the analysis of face images.

Face Recognition in Adverse Conditions

In this chapter, the authors discuss the main outcomes from both the most recent literature and the research activities summarized in this book. Of course, a complete review is not possible. It is evident that each issue related to face recognition in adverse conditions can be considered as a research topic in itself and would deserve a detailed survey of its own. However, it is interesting to provide a compass to orient one in the presently achieved results in order to identify open problems and promising research lines. In particular, the final chapter provides more detailed considerations about possible future developments.

Secure Face Recognition for Mobile Applications

Biometric systems are generally restricted to specialist deployments and require expensive equipment. However, the world has recently experienced the widespread rollout of cheap biometric devices in the form of smart phones and tablets. One of the main drivers for mainstream adoption of biometric technologies is the need to address continuing problems with authenticating to online systems. These mobile devices may now be suitable to provide biometric-based authentication to a wide user population. This chapter discusses the different ways that face recognition can be used on smart mobile devices. The authors highlight the online authentication problem and show how three-factor authentication can address many pressing issues. They also discuss the ways that such a system could be attacked, and focus on replay attacks which have yet to be seriously addressed in the literature. The authors conclude with a brief examination of the current research into addressing replay attacks.