Face alignment and eye tracking for interactive applications should be performed with very low latency or users will notice the delay. In this chapter, a face alignment method for real-time applications is introduced featuring a convolutional neural network architecture for face and pose alignment. The performance of the novel method is compared to a face alignment algorithm included in the freely available OpenFace toolkit, which also focuses on real-time applications. The approach exceeds OpenFace's performance on both our own and the 300W test sets in terms of accuracy and robustness but requires significant parallel processing power, currently provided by the GPU. For the eye tracking application, stereo cameras are used as input to determine the position of a user's eyes in three-dimensional space. It does not require synchronized recordings, which may contain redundant information, and instead prefers staggered recordings, which maximize the number of possible model updates.
Dense 3D face alignment from 2D video for real-time use