Improving KNN-Based Face Recognition with PCA: The Olivetti Dataset

Olivetti Dataset

Brief information about Olivetti Dataset:

• Face images taken between April 1992 and April 1994.
• There are ten different image of each of 40 distinct people
• There are 400 face images in the dataset
• Face images were taken at different times, variying ligthing, facial express and facial detail
• All face images have black background
• The images are gray level
• Size of each image is 64x64
• Image pixel values were scaled to [0, 1] interval
• Names of 40 people were encoded to an integer from 0 to 39

link:https://www.kaggle.com/datasets/imrandude/olivetti

import numpy as np
data=np.load("/content/drive/MyDrive/faces_dataset/olivetti_faces.npy")
target=np.load("/content/drive/MyDrive/faces_dataset/olivetti_faces_target.npy")

print("There are {} images in the dataset".format(len(data)))
print("There are {} unique targets in the dataset".format(len(np.unique(target))))
print("Size of each image is {}x{}".format(data.shape[1],data.shape[2]))
print("Pixel values were scaled to [0,1] interval. e.g:{}".format(data[0][0,:4]))

There are 400 images in the dataset

There are 40 unique targets in the dataset Size of each image is 64x64

Pixel values were scaled to [0,1] interval. e.g:[0.30991736 0.3677686 0.41735536 0.44214877]

print("unique target number:",np.unique(target))

unique target number: [ 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39]

def show_10_faces_of_n_subject(images, subject_ids):
    cols=10# each subject has 10 distinct face images
    rows=(len(subject_ids)*10)/cols #
    rows=int(rows)

    fig, axarr=plt.subplots(nrows=rows, ncols=cols, figsize=(18,9))
    #axarr=axarr.flatten()

    for i, subject_id in enumerate(subject_ids):
        for j in range(cols):
            image_index=subject_id*10 + j
            axarr[i,j].imshow(images[image_index], cmap="gray")
            axarr[i,j].set_xticks([])
            axarr[i,j].set_yticks([])
            axarr[i,j].set_title("face id:{}".format(subject_id))

#You can playaround subject_ids to see other people faces
show_10_faces_of_n_subject(images=data, subject_ids=[1,4, 25, 20, 39])

#We reshape images for machine learnig  model
X=data.reshape((data.shape[0],data.shape[1]*data.shape[2]))
print("X shape:",X.shape) # X shape: (400, 4096)

from sklearn.model_selection import train_test_split

# Assuming you have X (features) and target (labels)

# Split the data into training and temporary sets (70% training, 30% temp)
X_train_temp, X_test_val, y_train_temp, y_test_val = train_test_split(X, target, test_size=0.3, stratify=target, random_state=0)

# Further split the temporary set into testing and validation sets (50% testing, 50% validation)
X_test, X_val, y_test, y_val = train_test_split(X_test_val, y_test_val, test_size=0.5, stratify=y_test_val, random_state=0)

print("X_train shape:", X_train_temp.shape) #X_train shape: (280, 4096)
print("y_train shape:", y_train_temp.shape) #y_train shape: (280,)
print("X_test shape:", X_test.shape) #X_test shape: (60, 4096)
print("y_test shape:", y_test.shape) #y_test shape: (60,)
print("X_val shape:", X_val.shape) #X_val shape: (60, 4096)
print("y_val shape:", y_val.shape) #y_val shape: (60,)

from sklearn.neighbors import KNeighborsClassifier
from sklearn.metrics import accuracy_score
clf=KNeighborsClassifier()
clf.fit(X_train_temp,y_train_temp)
accuracy_score(y_test_val,clf.predict(X_test_val))
print("Kneighbors score:",accuracy_score(y_test_val,clf.predict(X_test_val)))
#Kneighbors score: 0.7916666666666666

Principle Component Analysis

Machine learning methods are divided into two: supervised learning and unsupervised learning. In supervised learning, the data set is divided into two main parts: ‘data’ and ‘output’. The data holds the values of the sample in the data set, while the ‘output’ holds the class (for classification) or the target value (for regression). In unsupervised learning, the data set consists of only the data section.

Non-supervised learning is generally divided into two: data transformation and clustering. In this study, the transformation of the data will be carried out using unsupervised learning. Unsupervised transformation methods allow for easier interpretation of data by computers and people.

The most common unsupervised transformation applications is to reduce data size. In the size reduction process, the dimension of the data reduced.

Principle Component Analysis (PCA) is a method that allows data to be represented in a lesser size. According to this method, the data is transformed to new components and the size of the data is reduced by selecting the most important components.

import mglearn
mglearn.plots.plot_pca_illustration()

The above illustration shows a simple example on a synthetic two-dimensional data set. The first drawing shows the original data points colored to distinguish points. The algorithm first proceeds by finding the direction of the maximum variance labeled “Component 1”. This refers to the direction in which most of the data is associated, or in other words, the properties that are most related to each other.

Then, when the algorithm is orthogonal (at right angle), it finds the direction that contains the most information in the first direction. There are only one possible orientation in two dimensions at a right angle, but there will be many orthogonal directions (infinite) in high dimensional spaces.

number_of_people=10
index_range=number_of_people*10
fig=plt.figure(figsize=(10,8))
ax=fig.add_subplot(1,1,1)
scatter=ax.scatter(x_train_pca[:index_range,0],
            x_train_pca[:index_range,1], 
            c=target[:index_range],
            s=10,
           cmap=plt.get_cmap('jet', number_of_people)
          )

ax.set_xlabel("First Principle Component")
ax.set_ylabel("Second Principle Component")
ax.set_title("PCA projection of {} people".format(number_of_people))

fig.colorbar(scatter)

fig,ax=plt.subplots(1,1,figsize=(8,8))
ax.imshow(pca.mean_.reshape((64,64)), cmap="gray")
ax.set_xticks([])
ax.set_yticks([])
ax.set_title('Average Face')

Text(0.5, 1.0, ‘Average Face’)

number_of_eigenfaces=len(pca.components_)
eigen_faces=pca.components_.reshape((number_of_eigenfaces, data.shape[1], data.shape[2]))

cols=10
rows=int(number_of_eigenfaces/cols)
fig, axarr=plt.subplots(nrows=rows, ncols=cols, figsize=(15,15))
axarr=axarr.flatten()
for i in range(number_of_eigenfaces):
    axarr[i].imshow(eigen_faces[i],cmap="gray")
    axarr[i].set_xticks([])
    axarr[i].set_yticks([])
    axarr[i].set_title("eigen id:{}".format(i))
plt.suptitle("All Eigen Faces".format(10*"=", 10*"="))

Text(0.5,0.98,’All Eigen Faces’)

from sklearn.decomposition import PCA
pca=PCA(n_components=50)
pca.fit(X_train_temp)
x_train_pca=pca.transform(X_train_temp)
x_test_pca=pca.transform(X_test_val)
clf=KNeighborsClassifier()
clf.fit(x_train_pca,y_train_temp)
accuracy_score(y_test_val,clf.predict(x_test_pca))
print("PCA score:",accuracy_score(y_test_val,clf.predict(x_test_pca)))
# PCA score: 0.8166666666666667

References

[embed]olivetti Kaggle is the world's largest data science community with powerful tools and resources to help you achieve your data…www.kaggle.com

https://www.kaggle.com/code/serkanpeldek/face-recognition-on-olivetti-dataset