The objective of this project was to develop a method for identifying faces in situations where the face is angled away from the camera or partially obscured. Current state-of-the-art face recognition systems perform very poorly in these circumstances and only work well when the subject faces the camera, unobscured, with good lighting. Faces obtained from a publicly available dataset used in a graduate-level CMU machine learning class were: (1) Put through a series of image processing and filtering steps, (2) Analyzed for custom landmarks that generated useful feature vectors for characterizing each face, and (3) Classified using a version of the K-Nearest-Neighbors algorithm with a feature vector involving both the feature already extracted, as well as pixels from the image themselves. Tens of thousands of automated tests were conducted using different combinations of the facial features and filters as steps to generate models using KNN with varying accuracies. The optimal model yielded an accuracy of 90.4% for the dataset as a whole, and an accuracy of 55% when only the subset of images with angled and obscured faces were selected. This algorithm performs similarly to conventional approaches for the dataset as a whole, but performs incredibly well (over 50% accurate) when only trained and tested on images that had partially obscured and angled faces in them. Face recognition techniques today rarely perform any better than random guessing when presented with this problem. The applications of this project thus have wide implications to national security because real life videos and photographs very rarely capture people facing the camera directly with their full face uncovered.
Third Award of $1,000