scholarly journals RGBD Based Gaze Estimation via Multi-Task CNN

2019 ◽  
Vol 33 ◽  
pp. 2488-2495 ◽  
Author(s):  
Dongze Lian ◽  
Ziheng Zhang ◽  
Weixin Luo ◽  
Lina Hu ◽  
Minye Wu ◽  
...  
Keyword(s):  
Large Scale ◽  
Image Synthesis ◽  
Position Estimation ◽  
Point Estimation ◽  
Head Pose Estimation ◽  
Depth Image ◽  
Eye Position ◽  
Gaze Estimation ◽  
Head Pose ◽  
Gaze Tracking

This paper tackles RGBD based gaze estimation with Convolutional Neural Networks (CNNs). Specifically, we propose to decompose gaze point estimation into eyeball pose, head pose, and 3D eye position estimation. Compared with RGB image-based gaze tracking, having depth modality helps to facilitate head pose estimation and 3D eye position estimation. The captured depth image, however, usually contains noise and black holes which noticeably hamper gaze tracking. Thus we propose a CNN-based multi-task learning framework to simultaneously refine depth images and predict gaze points. We utilize a generator network for depth image generation with a Generative Neural Network (GAN), where the generator network is partially shared by both the gaze tracking network and GAN-based depth synthesizing. By optimizing the whole network simultaneously, depth image synthesis improves gaze point estimation and vice versa. Since the only existing RGBD dataset (EYEDIAP) is too small, we build a large-scale RGBD gaze tracking dataset for performance evaluation. As far as we know, it is the largest RGBD gaze dataset in terms of the number of participants. Comprehensive experiments demonstrate that our method outperforms existing methods by a large margin on both our dataset and the EYEDIAP dataset.

scholarly journals Real-time gaze estimation via pupil center tracking

2018 ◽  
Vol 9 (1) ◽  
pp. 6-18 ◽  
Author(s):  
Dario Cazzato ◽  
Fabio Dominio ◽  
Roberto Manduchi ◽  
Silvia M. Castro
Keyword(s):  
Real Time ◽  
Learning Algorithm ◽  
Natural Environments ◽  
Gaze Estimation ◽  
Head Pose ◽  
Data Set ◽  
Gaze Tracking ◽  
Illumination Changes ◽  
Wide Range ◽  
Estimation System

Abstract Automatic gaze estimation not based on commercial and expensive eye tracking hardware solutions can enable several applications in the fields of human computer interaction (HCI) and human behavior analysis. It is therefore not surprising that several related techniques and methods have been investigated in recent years. However, very few camera-based systems proposed in the literature are both real-time and robust. In this work, we propose a real-time user-calibration-free gaze estimation system that does not need person-dependent calibration, can deal with illumination changes and head pose variations, and can work with a wide range of distances from the camera. Our solution is based on a 3-D appearance-based method that processes the images from a built-in laptop camera. Real-time performance is obtained by combining head pose information with geometrical eye features to train a machine learning algorithm. Our method has been validated on a data set of images of users in natural environments, and shows promising results. The possibility of a real-time implementation, combined with the good quality of gaze tracking, make this system suitable for various HCI applications.

SCALE ROBUST HEAD POSE ESTIMATION BASED ON RELATIVE HOMOGRAPHY TRANSFORMATION

2014 ◽  
Vol 10 (01) ◽  
pp. 69-90 ◽  
Author(s):  
CHENGUANG LIU ◽  
HENGDA CHENG ◽  
ARAVIND DASU
Keyword(s):  
Pose Estimation ◽  
Large Scale ◽  
Region Of Interest ◽  
Head Pose Estimation ◽  
Head Pose ◽  
Virtual Camera ◽  
The Face ◽  
Large Scale Change ◽  
Chamfer Matching ◽  
Homography Transformation

Head pose estimation has been widely studied in recent decades due to many significant applications. Different from most of the current methods which utilize face models to estimate head position, we develop a relative homography transformation based algorithm which is robust to the large scale change of the head. In the proposed method, salient Harris corners are detected on a face, and local binary pattern features are extracted around each of the corners. And then, relative homography transformation is calculated by using RANSAC optimization algorithm, which applies homography to a region of interest (ROI) on an image and calculates the transformation of a planar object moving in the scene relative to a virtual camera. By doing so, the face center initialized in the first frame will be tracked frame by frame. Meanwhile, a head shoulder model based Chamfer matching method is proposed to estimate the head centroid. With the face center and the detected head centroid, the head pose is estimated. The experiments show the effectiveness and robustness of the proposed algorithm.

scholarly journals Highly Accurate and Fully Automatic 3–D Head Pose Estimation and Eye Gaze Estimation Using RGB-–D Sensors and 3D Morphable Models

2018 ◽  
Author(s):  
Reza Shoja Ghiass ◽  
Denis Laurendeau
Keyword(s):  
Pose Estimation ◽  
Eye Gaze ◽  
Head Pose Estimation ◽  
Head Model ◽  
Gaze Estimation ◽  
Head Pose ◽  
Depth Data ◽  
Manual Intervention ◽  
Fully Automatic ◽  
Eye Gaze Estimation

This work addresses the problem of automatic head pose estimation and its application in 3D gaze estimation using low quality RGB--D sensors without any subject cooperation or manual intervention. The previous works on 3D head pose estimation using RGB--D sensors require either an offline step for supervised learning or 3D head model construction which may require manual intervention or subject cooperation for complete head model reconstruction. In this paper, we propose a 3D pose estimator based on low quality depth data, which is not limited by any of the aforementioned steps. Instead, the proposed technique relies on modeling the subject's face in 3--D rather than the complete head, which in turn, relaxes all of the constraints with the previous works. The proposed method is robust, highly accurate and fully automatic. Moreover, it does not need any offline step. Unlike some of the previous works, the method only uses depth data for pose estimation. The experimental results on the Biwi head pose database confirm the efficiency of our algorithm in handling large pose variations and partial occlusion. We also evaluate the performance of our algorithm on IDIAP database for 3D head pose and eye gaze estimation.

Real-Time Head Pose Estimation and Face Modeling From a Depth Image

2019 ◽  
Vol 21 (10) ◽  
pp. 2473-2481 ◽  
Author(s):  
Changwei Luo ◽  
Juyong Zhang ◽  
Jun Yu ◽  
Chang Wen Chen ◽  
Shengjin Wang
Keyword(s):  
Real Time ◽  
Pose Estimation ◽  
Head Pose Estimation ◽  
Depth Image ◽  
Head Pose ◽  
Face Modeling

Generic and Robust Method for Head Pose Estimation

2016 ◽  
Vol 4 (2) ◽  
pp. 439
Author(s):  
Abdellatif Hajraoui ◽  
Mohamed Sabri
Keyword(s):  
Facial Expression ◽  
Pose Estimation ◽  
Head Pose Estimation ◽  
Svm Classifier ◽  
Robust Method ◽  
Head Pose ◽  
Gaze Tracking ◽  
Facial Motion ◽  
Facial Expression Analysis ◽  
The One

Head pose estimation has fascinated the research community due to its application in facial motion capture, human-computer interaction and video conferencing. It is a pre-requisite to gaze tracking, face recognition, and facial expression analysis. In this paper, we present a generic and robust method for model-based global 2D head pose estimation from single RGB Image. In our approach we use of the one part the Gabor filters to conceive a robust pose descriptor to illumination and facial expression variations, and that target the pose information. Moreover, we ensure the classification of these descriptors using a SVM classifier. The approach has proved effective view the rate for the correct pose estimations that we got.

Gaze Estimation From Eye Appearance: A Head Pose-Free Method via Eye Image Synthesis

2015 ◽  
Vol 24 (11) ◽  
pp. 3680-3693 ◽  
Author(s):  
Feng Lu ◽  
Yusuke Sugano ◽  
Takahiro Okabe ◽  
Yoichi Sato
Keyword(s):  
Image Synthesis ◽  
Gaze Estimation ◽  
Head Pose

scholarly journals ETH-XGaze: A Large Scale Dataset for Gaze Estimation Under Extreme Head Pose and Gaze Variation

2020 ◽  
pp. 365-381 ◽  
Author(s):  
Xucong Zhang ◽  
Seonwook Park ◽  
Thabo Beeler ◽  
Derek Bradley ◽  
Siyu Tang ◽  
...  
Keyword(s):  
Large Scale ◽  
Gaze Estimation ◽  
Head Pose ◽  
Large Scale Dataset

scholarly journals Low Cost Eye Tracking: The Current Panorama

2016 ◽  
Vol 2016 ◽  
pp. 1-14 ◽  
Author(s):  
Onur Ferhat ◽  
Fernando Vilariño
Keyword(s):  
Eye Tracking ◽  
Visible Light ◽  
Open Source ◽  
Low Cost ◽  
Gaze Estimation ◽  
Head Pose ◽  
Gaze Tracking ◽  
Estimation Techniques ◽  
Public Datasets ◽  
Interesting Alternative

Despite the availability of accurate, commercial gaze tracker devices working with infrared (IR) technology, visible light gaze tracking constitutes an interesting alternative by allowing scalability and removing hardware requirements. Over the last years, this field has seen examples of research showing performance comparable to the IR alternatives. In this work, we survey the previous work on remote, visible light gaze trackers and analyze the explored techniques from various perspectives such as calibration strategies, head pose invariance, and gaze estimation techniques. We also provide information on related aspects of research such as public datasets to test against, open source projects to build upon, and gaze tracking services to directly use in applications. With all this information, we aim to provide the contemporary and future researchers with a map detailing previously explored ideas and the required tools.

Sign in / Sign up

Export Citation Format

Share Document