The Eigenface-based Face Detection

1. Eigenface Model:

What is eigenface?

For a given set of sample faces, we apply the principle component analysis (PCA) on it, to calculate the eigenvectors of these sample faces. The obtained eigenvectors are called the eigenfaces of given sample faces.
 

Why eigenfaces?

These eigenfaces construct a feature space, which best encodes the distinguishing features of sample faces, such as different facial expressions, lighting conditions etc.

An example of eigenface set (frontal):

_______
    E0          E1          E2        E3           E4         E5          E6           E7

E0:         The mean template of sample faces.
E1-7:    The first seven eigenfaces calculated from sample faces.

Note: Different facial expressions and lighting conditions are extracted by eigenfaces.
 

What is distance-from-feature-space (DFFS)?

Images containing N pixels can be considered as vectors in a N diemonsinal space. Eigenfaces construct a feature subspace F,  which has a much lower dimensionality M equaling to the number of sample faces. The DFFS of a given image vector, is refered to the distance between itself and its projection into the feature space. This metric can be used as an effective indicator of similarity between given images and the eigenfaces set:

Multiple eigenface sets:

To tackle with the multiple poses & pose estimation problem, the multiple eigenface sets method is employed.  In this formulation, a separate set of eigenfaces is computed for each possible face pose. Face pose is identified by computing the eigenspace projection of the given image patch onto each eigenspace, and selecting the one with the minimum DFFS value. This scheme can be viewd as a multiple-locater system where separate eigenfaces sets are simultaneously competing in describing the given patch. Examples of multiple eigenface sets are shown below:

We have implemented three eigenfaces sets, namely the frontal and 45 degree side view(left, right), in the current system.  With these eigenfaces sets, we can identify whether the face is looking ahead, or turning left or right. More accurate pose estimation can be achieved by implementing more eigenface sets. For example, the figure in below suggests a way to estimate the face pose with 1 frontal, 5 sideview eigenfaces sets, and 3 in-plane-rotated eigenfaces sets:
 

2. Eigenface-based Face Detection:

For a given image patch, its DFFS value is calculated and used to estimate the likelihood of being faces. Among all image patches, the one with the minimum DFFS value is considered to be a face.

 

More results of face detection:

____

____

Features:

• Accurate location of faces (compared with the head detection).
• Variation of facial features, such as races, gender, expressions etc., are well encoded by eigenface sets.
• Different poses can be handled with multiple eigenface sets, and the pose is correctly estimated.

Return to homepage