Lesson Learned: Face Detection is Hard
A while back I thought it would be a good idea to implement face detection in Gallery2, based on a similar feature in Facebook. I downloaded the OpenCV computer vision toolkit and ran the facedetect sample app against a collection of 60 photos from my gallery, some with no faces, others with a single face, still others with multiple faces, faces in profile, etc. No matter which training file I used, the face detection was horribly unreliable. I would estimate 10% of actual faces were detected, and of all the faces reported by the tool, maybe 5% were actual faces.
This was pretty disappointing, so I uploaded the same corpus of photos to Facebook to see how well it did with them. Imagine my surprise when it didn’t detect any! The face detection I was seeing on my sister’s profile was due to manual tagging of the photos from within the upload applet, and was not driven by computer vision algorithms. That explains that.
I still think a similar feature in Gallery would be nice, but with the CV aspect removed it doesn’t seem quite so fun.
Project Idea: Face detection in Gallery2
This past weekend my little sister and I were going through the Facebook profiles of various cousins, and I noticed something about Facebook’s photo support that I somehow missed before: it automatically detects the presence of faces in each photo, and allows users to tag each face with the identity of its owner. Already-tagged faces have the owner’s name superimposed over the image.
That’s an awesome feature, and reminds me of the stuff Riya was working on a few years back (although FB doesn’t do facial recognition (yet, anyway), so you still have to tag everyone yourself). I was immediately jealous that my photo hosting software of choice, Gallery, didn’t have this feature.
I investigated this a bit, and I found that Intel’s OpenCV library includes open-source face detection code. Using Intel’s sample face detect app, I found it to be both quick (~150ms per photo) and accurate. I wonder how much work would be required to create a Gallery module that used opencv to detect faces in photos, and provided an AJAX UI for tagging the photos. It would certainly be cool.