(Here’s a link to the data I used! Details of the contents are at the end of the article.)
This morning I went through Mark Zuckerberg’s Facebook photos and cropped 176 headshots of various Marks from the last fourteen years. Then I used a TensorFlow implementation of FaceNet to generate a feature vector (a sort of “facial fingerprint” made up of 512 numbers) for each one.
I calculated the midpoint of all of these feature vectors in an attempt to find the platonic ideal of Mark Zuckerberg’s face. Then I sorted all of the faces I’d collected by how far they deviated from the norm, and stitched them into this image:
It’s hard to tell whether the faces towards the bottom are there because they were taken in unusual light or by bad cameras, or because Mark’s head is turned (all of which I’d want algorithms to correct for), or if it’s picking up on more interesting features like how suntanned he is, whether he’s pulling an usual facial expression, how old he is, or whether he’d been hitting the gym in the months before the photo was taken.
I used MTCNN to crop and rotate his face, but judging by the fact that the two most distant faces are ones where Mark is turning his head, I could probably do with more sophisticated “face alignment” before I generate the feature vectors (e.g. running the face through an algorithm that builds a 3D model, and “turns” the face til its facing the front).
Another thing I’d like to try is doing dimensionality reduction on the feature vectors, and then seeing which faces of his are outliers along different dimensions. Would I be able to tell from visual inspection what those dimensions represent?
Other directions I could take:
- Clustering the Marks
- Calculating feature vectors for faces in big datasets (like the one with hundreds of thousands of faces scraped from IMBD) and seeing which other humans look the most like Mark
- Using a GAN to generate new Zuckerbergs, or to transform existing faces so that they look more like him
Here’s a zip file containing a pickled Python object that maps image filenames to feature vectors, along with all of the images. Let me know if you make find out anything cool!