Face Recognition Using Eigen Faces

This implementation of the face recognition using Eigenfaces technique was done with the Octave 4.0 tool. Octave is quite similar to Matlab, but freely distributed under GNU public license.

GNU Octave Logo

The sample images were taken from the AT&T face database. There are lot of other face databases that freely given for research work, so you can try out another or you can use your own.

So the implementation steps go as follows.

Step 1

First step was to create a suitable training image set.

For my work, I use 20 grayscale face images 92 pixels wide by 112 pixels tall from 20 different people. (10th index image set of the first 20 folders in the AT&T database)

My 20 Training Image set selected from the AT&T face database

Following code segment will setup our global variables

Step 2

All images need to be treated as one image vector. For this implementation, next step is to concatenate the rows of pixels in each image, so that all images of the training set are stored in a single matrix, where each column of the matrix is an image.

Following code segment will select our training image set and create this image vector. Code is written according to the AT&T folder structure.

Step 3

When the image vector is ready, next we normalize each image in the vector to reduce the negative effect due to lighting conditions and background. This step should be done before creating the image vector, if the images do not share a common pixel resolution and, eyes and mouths are not aligned across all images.As the AT&T face database images are created accordingly, we are doing this as the 3rd step.

The selected image set were normalized

Step 4

Next step is to create an average image from the normalized image vector we created.

Mean image of our image vector of 20 normalized images

Step 5

Next step is to subtract this mean image from each image in our normalized training-set and compute the covariance matrix.

Step 6

Then we have to obtain the EigenVectors of the covariance matrix. Each eigenvector has the same dimensionality (number of components) as the original images, and thus can itself be seen as an image. The eigenvectors of the covariance matrix are therefore called Eigenfaces. They are the directions in which the images differ from the mean image.

Eigenfaces

Step 7

As now we have our face space, Next step is to start the recognition process.

Once we have projected every sample image into the Eigenface subspace, we have a bunch of points in a 2 dimensional space. When given a new image (a new picture of someone in the training set), all we need do is project it into face space and find the smallest distance between the new point and each of the projections of the sample images (of the people pictured in the training set). This gives us the one that best matches the input picture.

So here we are inputting exactly the 13th image in our image set. This input is transformed into its Eigenface components. First we compare our input image with our mean image and multiply their difference with each eigenvector. Each value would represent a weight and would be saved on a vector, that we called as reconstructed image.

Input image and the reconstructed image

Step 8

We now determine which face class provides the best description for the input image. This is done by minimizing the Euclidean distance. The graphs shown depict the distance of each input image to each person on the training set (along the x-axis, 1-20). The input image belongs to the person with the minimum shortest distance. As shown it gives the index 13 with the minimum distance value (12060.971) as we expected.

Euclidean Distance

Issues & Resolutions

Looking only for the shortest distance is not enough. This method will mislead the results when inputting an image not in the dataset. So checking for both maximum and minimum distance with double thresholding is a good way to resolve this issue.

If the minimum distance value is below 12,100 then the face of the input image, surely in our training set. If it is above 12,100 we don’t have the input image on our training set. But the same face in a different angle could be. Therefore we’re check whether the minimum value somehow below 15,000 and maximum value somehow below 16,000. If these conditions satisfy we may find a good result, but not always. Anyway we could arrive at this point that the image is a face, not any other object. If the values do not satisfy these conditions, we can surely say the input image is not a face.

One threshold on a single variable is not enough to arrive at a good conclusion, so double thresholding using multiple variables is a good way to optimize.

P.S

I was first started to use OpenCV with Java to build this thing as I was familiar with that thing only. Then by I get into know still the Eigenface functionality is not implemented in OpenCV Java wrapper but for other languages. Rather than learning another language such like python or C with OpenCV, I thought to move into Matlab. But Matlab is a heavyweight commercial application. So the only option was Octave, which is free and lighter than Matlab, and here you are...