Eye detection is an important initial step in an automatic face recognition system. Though numerous eye detection methods have been proposed, many problems still exist, especially in the detection accuracy and efficiency under challenging image conditions. The authors present a novel eye detection method using color information, Haar features, and a new efficient Support Vector Machine (eSVM) in this chapter . In particular, this eye detection method consists of two stages: the eye candidate selection and validation. The selection stage picks up eye candidates over an image through color information, while the validation stage applies 2D Haar wavelet and the eSVM to detect the center of the eye among these candidates. The eSVM is defined on fewer support vectors than the standard SVM, which can achieve faster detection speed and higher or comparable detection accuracy. Experiments on Face Recognition Grand Challenge (FRGC) database show the improved performance over existing methods on both efficiency and accuracy.