Future of Facial Recognition
The Boston Bombings reaffirmed how far facial recognition needs to develop to be an effective surveillance tool. Despite the PR spin jobs, facial recognition failed to make any difference. Proponents, of course, are quick to talk about how fast improvements are coming and how all this will change in the future (ironically, just like they said 5 years ago). However, one Carnegie Mellon professor had insightful comments about the issues and needs for facial recognition. In this note, we break them down and analyze their likelihood of development.
People Look A Like
That people look a like is underestimated in every day life as most regularly interact with few people. However, a facial recognition system is 'looking at' lots and lots of people. Think of a supermarket. They easily have hundreds of visitors a day, tens of thousands a month. With that many people, quite a number are going to look similar. For a good visual example, see a series of portraits of people who look alike but are not related.
As the professor notes:
"You still have the problem of false positives. When you start working with databases that contain millions of faces, you start to realize that many people look similar to each other."
This problem is complicated further when comparing photos of people with slightly different poses, different lighting conditions, that might be a little fuzzy, etc. Because of this, even when you have just thousands of faces, you can still have problems. Having more pixels does help but it does not overcome the pose and lighting conditions problems.
The Solution?
The professor has an interesting concept to solve this problem, noting what "makes people so good at distinguishing similar faces is that we humans don’t simply do facial recognition when we recognize other people. We do a holistic recognition of the entirety of the person, based on how they are dressed, how they are walking, and on where they are geographically and physically."
The solution he recommends then is:
"Facial recognition will start to put images together with metadata such as how tall you are, what is your favorite way of dressing, where were you most likely to be in the last 24 hours. From a social network, for example, I can see the IP address on which you connected three hours ago, and I can see, because there are so many photos of you, how tall you are, whether you have gained weight or not, your dressing style."
Those who worry about Big Brother will be extremely worried but those who want to better matching will embrace this.
Ultimately, though, while this could help, implementing this will still be difficult. Measuring someone's height from cameras is not possible without manual calibration. Real criminals are unlikely to be posting up to date photos on facebook with showing their new hair style or how much weight they gained. Dress can be easily changed to camouflage appearance, etc.
The Future
The most powerful improvement for facial recognition will likely be high definition cameras, which finally delivers enough detail to capture quality faces across non trivial field of view widths. This, at least, makes it feasible to index faces. Unfortunately, no silver bullet exists for matching, from the unresolved issues in correcting pose to the similarities of people looking like each other.