FAILED: Facial Recognition & Boston Bombing

By John Honovich, Published Apr 20, 2013, 08:00pm EDT (Info+)

Facial recognition vendors have been tripping over themselves to take credit, insert or claim the wonders of their offerings to identify the Boston bombers. However, an excellent Washington Post investigative report cites Boston's Police Commissioner, confirming that:

"Facial-recognition software did not identify the men in the ball caps. The technology came up empty even though both Tsarnaevs’ images exist in official databases."

** **** ****** *** ****? * lot **:

"**** [****] *** *********** *** ****-*******: One ***** ******* *** **** ******* of ***** *** *****."

****, * ******** *** **** * victim *** *** **** *** ****** face ** ****:

"*** ****** ******* **** ** ******’* bedside. * *** ** ********** *** black ******** *** *** ****** ***** up ** ***, ****** * ***** backpack ** *** ****** *** ******* away, ****** **********.

*** *** ****** * ******** ****, according ** *** *********** *********."

* ***** **** ****** ** ***** the ****** *** ***** **** ** human ************* *** *** ******* ** a *** ***** **** **** **** off *** **** ****** **** * promotional ******** *** ********** *******.

Technical ******** ** ****** ***********

*** ****** ******** ** *** ******, both **** ************ ******* *** ***********, were *** **** ****** ** ******* high ******** ********. ****** *** ***** from *** **** & ******'* ************ cameras, ******* *****:

*****'* *** **** ******** **** *** ******** are ******* ** **** *** ***** way, **** *** ***** ******* ** too *** (******** ********** ****** **** a ~** **** **** ***). *** obviously *** ***** ******* (#*) *** wearing * ******* ****** *** *** sunglasses, ******* **** **** ********.

****, **** **** *** **** *** cameras ******** **** *********** '***'. **** are ********** *** ******* ********** *** not *** *** ********* **** ********* needs ** ****** ***********.

**** '*********' **** ******* ****** *********** could *** ****.

**********, *** ****** **** *********** **** of ****** *********** *** ****** *********** than ************ *******. ****'* ** ******* from *** ** *** **** ****:

**** ** ********* **** * **** wider *** *** *** ********* ***** density ** *** *** ** ******* accurate ****** ********. * ***** ***** of * **** *** ** '****' for * ***** ** ****** ******* a ******** ******/***** *** *** * computer ** ** **** ******* ******** of ****** *** ** ******* ** exercise ** ********.

Comments (18)

I have facial recognition software on my laptop and I have been testing the software. I've found that a controlled enviornment is in credibly important. (1) you have the pixel between the eyes needing 30 minimum to potentially 150 plus accross the face. (2) the pitch angle needs to be very shallow and not a top down shot. (3) the lighting is critical...controlled lighting as in a teller line or retail line where day or night the lighting is the same. This is important for face face capture and even MORE for searching. There probably lots of other super high end face cameras and software out there that i have not tested. However from what I have experienced the set up is an "art" and has to be tested to the up most detail.

Jon, absolutely correct about the testing for any type of video analytics, but especially for facial recognition.

For proof of concept testing on video analytics, I recommend creating a selection of recorded video clips containing the situations they want to detect or the quality of facial images they want to be able to identify, and then provide these to the candidate analytics vendors.

When the technology does not seem to work well enough with existing video samples, a next step could be identifying the conditions under which detection or recognition isn't working, and seeing if those conditions can be improved. A year or so later, with improved video images (better lighting, WDR, relocated cameras, higher resolution images, etc.) and improved analytics, chances of success are better.

But getting back to the title of this discussion, I wish more vendors would PROVE their claimed results rather than HYPE them.

I wish camera manufacturers would publish the test conditions and camera settings under which their lux ratings were measured.

I wish camera makers would include info in the data sheets about feature combinations that tax the processor and won't work together, such as with both a high frame rate and a high resolution. (Note that I have seen a few data sheet footnotes along this line and so on this topic I'm encouraged.)

I can see how Facial Recognition would work in a 'Controlled Environment' like an entry point where the camera is placed directly in front of the face capturing a clear crisp image. Though it would still be required to have the face in the databse. I have never see it work in real world scenarios (City Surveillance, etc). It is just a problem with expectation mismanagement. Hollywood has spoilt us silly.

BTW, how is the development of 3D FR coming by?

Lets "face" it, facial recognition only works in a very controlled environment. Proper lighting - proper angle of incidence - enough pixels to satisfy the requirements. Choke points are where this might work where all the above are present. Standard surveillance camera mounting locations most likely will never do the trick. Most cameras you see out there are drop tile ceiling mounted (for ease of placement presumably) or outside up high enough to deter easy defeat by a 9 iron through the housing or a quick spray paint to blind it. None of this is profound and has really been mentioned above.

What may be trouble for future IP Video sales (for the analog holdout at least) is the fact? that the famous sequences played over and over on the news of the Lord & Taylor Standard definition clips were "good enough" for ID. Good enough?? They look like Shite to me for an evenly lit bright day. I wonder what others think of that? 3fps - maybe. Ghosting. I dont recall a still coming out of that sequence of those two walking around the corner accept the one John embedded in this post.

We got better stills from 5MP Iphones or 8MP Samsung phones.....from a distance!

Its Monday. One week removed. I can already hear the marketing wheels spinning...

In reply to Michael Larkin, I completely agree with your comments especially the second paragraph and after.

I often tell people that you frequently get exactly what you don't pay for. If the customer does not know what they are buying and there is no detailed professional specification, then they get bupkus, just like what Lord & Taylor's cameras showed.

Don't want to pay for a "Good Quality" camera because you don't know what the difference is? Got WDR cameras? Huh? What's that Megapixel thing? Too confusing...

Don't understand that you simply can't just keep adding cameras to a 1 TB DVR or NVR and still have 30 days of recording unless you drop frame rate and resolution to unacceptable levels? Then you should not be allowed to purchase equipment!

Does your security vendor spend more time wining and dining you than explaining why their system is a better choice due to technical capability and then show you actual video? No? Technical presentations are too tedious for you...

Then you just got exactly what you did not pay for...a system that can barely be called a video surveillance system sold by an irresponsible trunk slammer only concerned about the next sale and not your security.

John,

Each technology has limitations and requirements to conditions of usage. Will you use a bus for pursuit? Yes, it's possible (Holywood prooves it :) ) but its main purpose is to transport people.

"FAILED: Facial Recognition"
There were not conditions for efficient usage of it. It is easy to say that it is useless but I can say it for ANY product or technology. It just depends on the conditions and purpose.

"Facial recognition vendors have been tripping over themselves to take credit, insert or claim the wonders of their offerings to identify the Boston bombers."

You need to use more correct wording. If one or another company make stupid claims it does not mean that ALL vendors are the same.

P.S. Is there any alternative to a face recognition technology for usage in crowded areas?

Konstantin, here are the face recognition vendors who have been touting the technology in stories about the bombing: 3VR, Animetrics, FaceFirst, Morphotrac, etc. That's 'facial recognition vendors', not just one or two companies, especially significant in such a niche market like facial recognition.

I don't see any video analytics technology, whether abandoned baggage or face recognition, working in crowded areas.

The Boston investigations revealed not so much a failure of facial recognition as a confirmation that camera position (height and vertical offset in relation to subject) and resolution parameters need to be satisfied for FR to be successful.

Let's say each private business went ahead and made those changes (lower height, better angles, higher resolution, narrower FoVs). I don't think this will happen because of the cost of doing so but, even if it did, you still have the issue with one subject wearing sunglasses and a ballcap and the other one being so young that his face might have materially changed since his mug shut photo.

That said, I do agree the probabilities would have been much improved if these businesses designed around the tighter requirements of facial recognition (though I question how many would do so).

John, the question was about an alternative. In general we need to identify people and we can do it in different ways but some of them are not acceptable (implanted RFID chip) or too expensive (security or police officer on each corner). At the same time there is FR technology which is limited but it is a real way to try to get some additional value from CCTV.

Generalization is a bad practice. If you can name companies then just list them.

I reread this article today and the discussions following. It's interesting to note that the title, "FAILED: Facial Recognition & Boston Bombing", does not really try to make the case that facial recognition failed, but actually pointed out the fact that vendors were taking completely unwarranted credit for the identification and resulting apprehension.

Donald Soper's comment made me take a step back and look at this again.

Strictly speaking, there was no facial recognition system in place or camera system in place that was intended to support street criminal facial recognition. So there can't be a failure of it.

However, the lack of facial recognition ID success doesn't mean the various video clips were of no value, as once an identification was made one way or another, the individuals could be tracked in various video clips based upon other elements than facial ID.

Vendors need to be very aware of how their marketing impacts public perception, as false assertions wrongly hurt the valid and fully feasible applications that do exist. There is enough trouble with false technology expectations based upon CSI shows and various movies that depict technically impossible results from computerized images. Of course I'm preaching to the choir and repeating assertions that have been made elsewhere, including other IPVM articles.

I've seen facial recognition work very well coupled with access card or keypad code access control, because the image to be validated is taken in a controlled fashion as was the stored image that is used for comparison.

So I don't want to decry all facial recognition just because target-in-a-crowd-anywhere facial recognition isn't highly feasible (and I know that is not the intention of the article).

Here's another article: CNN opinion piece mostly negative about surveillance expansion.

Another "scary story"... I wonder do they try to think before writing such kind of articles? Why people believe that government is interested in it?

It will be much cheaper and easier just to implant tags and make them mandatory for financial transactions, access to anything, traveling etc. than try to use FR.

Some thoughts on government interest in surveillance...

---------------------

If you believe the impossible stunts that TV and movies show related to video, why wouldn't you believe that other things in movies (like widespread government surveillance) could become real?

---------------------

See what Qatar is doing.

In 2011, Qatar, created a Security Systems Department to regulate and oversee all security operations throughout the country. The regulations apply to these businesses:

  • Banks and Exchange Shops
  • Clubs, Sports Venues and Cultural Centers
  • Warehouses and Hazmat Storage Areas
  • Hospitals and Clinics
  • Hotels, Apartments, and Residential Buildings
  • Shopping and Entertainment Centers

The country also sees it as a national defense initiative, even requiring some cameras and control rooms to be accessible through the National Command Center.

---------------------

You have to consider these things in the context of what tomorrow's technology will be capable of, not just today's tech. Things that required what was considered "massive computing power" just three decades ago can be done today on one's home computer or online. When it becomes easy and inexpensive, it can be done relatively quickly and quietly behind the scenes.

---------------------

An example of what can happen behind the scenes is presented by Malte Spitz in his 10-minute TED talk: Your phone company is watching. He asserts that the Berlin Wall would likely not have come down had today's cell phone technology been in use and today's cell phone regulations been in play in Germany at that time.

---------------------

I'm still thinking through these issues myself, but I think a better online article about surveillance is this one: Civil society, not Big Brother, is the American way.

John, I am working on a facial rec system now and I am finding that a best practice should be to employ is a pilot and/or a proof of concept to ensure the system performs as expected. To the naked eye faces seem to be able to be detected, however lighting, traffic patterns, FOV, pixel count, camera compatibility, CPU processing needs and installation with the right height to distance, make all the difference in a successful system.

One of the difficult items that I am experiencing is finding dome cameras that have long range lenses. Most dome cameras have max 12mm lenses internally, Axis, Arecont, Sony...etc. As well, some dome cameras when mounted to the ceiling can't view out at 90 degrees, due to the now large "auto focus, auto zoom" lenses bodies.

All in all, I recommend that all facial rec systems be 100% fully engineered so that expectations and problems are found in a "low risk" environment. From what I have found it is equally valuable to know where facial recognitian does NOT WORK along with where it does.

Jon, it would be crazy NOT to do a pilot. Indeed, it should be a rigorous one that, as much as possible, emulates real world load and scale, as issues may not be uncovered with just a few people.

I have learned more in a month than my entire 4 year mini-career. What you do for video and giving the community real world testing is priceless. Keep up the good work.

Login to read this IPVM report.
Why do I need to log in?
IPVM conducts reporting, tutorials and software funded by subscriber's payments enabling us to offer the most independent, accurate and in-depth information.
Loading Related Reports