Boston Bombings - Outrageous Manufacturer Claims

As details emerge on the Boston bombing, the impact and use of surveillance technologies is coming into view. [Now, updated below with a list of manufacturer claims on how their products would have (ha ha) prevented or solved the bombings.]

The FBI says that one of the bombs was contained in a black backpack that looked like so:

Presumably, since no one suggests this is a suicide bombing, the bomber must have abandoned the bag outside.

Immediately, this raises the question of detecting the abandoned object, either through a person calling it in or an analytic system automatically detecting it. However, give the dense crowds and numbers of moving people, this would be very hard to do via computer.

The next question is the role of recorded video. Historically, this would have been whatever surveillance cameras were in place. However, a huge shift has occurred in the last 5 years. Now, literally everyone has cameras integrated into their phones and many people were recording images and videos before and during the incident. Given the sheer number of those cameras, their superior resolution to most deployed cameras, and the variety of angles they were taken at, those personal cameras could prove even more critical than public cameras.

There's been a debate about the 'crowdsourcing' of the investigation where people on the Internet are going through shared videos and images, looking for suspects. This image has gained the most attention:

This could simply be too guys hanging out, falsely accused on the Internet. Or it might be right, or there might be another lead identified by people on the Internet scanning through personal videos and images.

Questions: Any analytics that can help here? Either to detect beforehand or search? What about the role of traditional surveillance cameras vs. personal camera/phones?

The best way an analytic might be able to help is only if you already have a person or item of interest. And if you were able to collect all video and pictures of the area in one place (exported video from surveillance system, YouTube and picture postings made on the Internet, any other video turned in by bystanders), like a folder on a computer, and have that analytic search all that video and pictures in the folder for any other instances of that person or item, you might be able to piecemeal together the person's movements and possibly find clues as to who they were and maybe even if anyone else was involved.

That would be an "offline" use per say, and would not require any integration with a system.

'Could analytics help?' is a good question. These before and after pictures look just likes scenes in "object left behind/removed" demos.

Wow, what a pain it would be to get all that video all in one place, no?

Carlton, is that TV station claiming the highlighted object is the bomb?

"Any analytics that can help here?"

I still believe there is NOTHING that could have done anything preventative in this (or similar) situations. Not enough camera coverage, too much crowd density, too many things going on.

For the post-facto investigation, this is something that Briefcam addresses. They can gather up lots of video, process it, and then provide options to control playback speed and object density.

These situations still require a lot of human intelligence. You're dealing with fuzzy shots, uneven whitebalanc and exposure across different cameras, time stamp issues, and so forth.

The TV station was saying it was a suspicious item in the same spot where a bomb exploded.

I just caught the tail end of Fox News interviewing the ObjectVideo CEO on analytics and Boston. I have no idea what he was saying because I only saw the last few seconds but if I can find it online, and it's anything interesting, I'll post it on here.

Maybe he was saying if anyone was attempting to use analytics on any of the videos they may be facing a lawsuit from Object Video.

That's funny!

Here's a compilation of pictures and videos being analyzed by people over the Internet. It will be interesting to see if any, or how many, identified the actual bombers.

Given the false reports so far, take this with some skepticism but the NYPost is reporting that they have obtained an email saying that the two guys in the photo at the top of this discussion are now wanted suspects.

If they turn out to be the bombers, this would be a pretty amazing milestone in the use of personal cameras and Internet investigations to identify criminals.

[UPDATE: This has been verified as FALSE.]

And now, people on Reddit are saying these claims are false: "It's a high school athlete and his track coach. He went to talk to the police an hour ago."

Here's the FBI's video releasing images of suspect. From what I can see, it looks like most images are from surveillance cameras.

And here's a screencap of one of the suspects from the video:

Here's the FBI's 'high resolution' images of the suspects though they are all relatively fuzzy.

Video on Fox News: Facial recognition and reconstructing Boston attack.

Anybody know of this company?

Animetrics is a niche player specializing in face recognition. They've been marketing 2d to 3d conversion for nearly a decade.

As for the claims in the video, let's say you could take a subject with a hat, at an angle and convert it. Best case scenario that would help searching (not live alerting) against mugshuts, not against regular surveillance video (which also will overwhelmingly have shots of people with the same issue).

The former President of Oncam Global says analytics could have prevented this. Video below:

The NY Times has an article citing 3VR's CEO:

"Even without a name, [3VR's CEO] said, investigators could program multiple cameras at airports and elsewhere with the suspects’ images so the cameras would send an alert to them if someone resembling a suspect passed by."

And Bloomberg has a video with FaceFirst's CEO making all sorts of amazing claims:

Slate magazine/website calls for more surveillance cameras, citing, among other things, BRS Lab's promotional material :)

Ann Coulter calls for more surveillance cameras.

I heard an interview with 3VR's CEO on NPR on way to work this morning. He was saying talking about using analytics to sift through video for clues, but to his credit he also talked about the limitations of analytics and actually said analytics have overpromised and under-delivered over the years. They did not say directly, but it seemed to imply 3VR analytics were used to sift through video.

I'll refrain on commenting on what Ann Coulter thinks.

Luis, thanks for the feedback. I am sure 3VR analytics are used to sift through video. What video and what use it is are the real questions?

Here's an article from MIT Technology Review. Quote from Michigan State professor:

"Attempting facial recognition on images like those released by the FBI Thursday is out of the question, [the professor] says. However, there may be other images and videos available that contain a better view that could be high quality enough, he says. “You could search all the other images based on clothing,” he says, “[and then] you could locate the same person and collect multiple images.”

I agree with the former, on it being out of the question for face recognition with those images. However, while you could do a search for color, there are an immense number of people wearing those colors, so it might help a little.

Kudos to the 3VR marketing department who got CNET to run what's little more than an advertorial for them:

The video concludes with a dismissal of a core limitation, noting 'while a hat or sunglasses can stump some cameras, engineers are already working to counteract" that. Lulz. They maybe working on it but I doubt they are solving it.

Here's USA Today on surveillance technology with basically another ad for 3VR. Money quote:

If the FBI has a clear video or photo image of a suspect, it's possible to quickly search through surveillance tapes from businesses that already use searchable video systems, which contain a ready-made catalogue of all individuals video taped, indexed by their facial characteristics. Search requests will turn up a list of close matches, leaving it up to the human investigator to make the final call, [3VR's CEO] says.

Two things:

  • Even if this works perfectly (it's nowhere close but let's assume), less than 1% of deployments have 'searchable video systems' for the type of metadata that 3VR is suggesting. So unless the plan is to standardize Boston on 3VR, it's not much of a practical recommendation
  • The camera positioning restrictions would make 95% of cameras ineligible for this type of analytic anyway (see our review of 3VR's gender/demographic analytics).

This one is kind of interesting, because the quotes appear to be from a US Park Police member.

It's also the first mention of Briefcam I've seen in print, which is maybe the only video analysis program that actually could be of help, simply to make searching faster. Even still, operating in such crowded scenes is hard, at best, and impossible (and irresponsible to claim) at worst.

All of these articles use lots of sentences describing what various software "could" do, not what it "did".

Briefcam still seems to me like the only real practical option here, and even that is no magic bullet.

What would a BRS system do in this case? This day was essentially a complete anomoly from start to finish (as it would have been even if it went perfectly normal). I find it hard to believe that any truly useful data would have come out of a video analytics platform meant to spot rogue activity (rules based or not). I also don't see this changing much in the near future. It's a really really hard problem to solve, and statistically speaking we don't see enough of these events (thankfully) to cause people to want to budget for a solution, which discourages anyone from attempting to solve this problem.

It would certainly be nice if some of what we saw on TV (or what bored reporters dreamed up) was true, but I think this is going to be human-analytics, assisted at times by tools like Briefcam, invetigating these kinds of events. The computers aren't going to find the red shirts and black backpacks for us.

SRI has an IR based system called VerifIR that I saw demonstrated at last years ISC West.

It tries to give the viewer a picture of what's under clothing and I even saw a video of it showing a shallow buried device under dirt. The rep at the show walked in front of the camera and I could see his threat resistant vest under his shirt.

So might this have helped as a preventive measure? Some caveats.

... How close would a person have to be to the camera to make out an object?

... Would require active viewing, I think. The picture doesn't look clear enough to really use an analytic to detect threats, and in this case, would a pressure cooker have been obvious enough to raise an alarm?

Luis, thanks, that's interesting. I embedded a video for the VerifIR in your comment that shows demonstrations of it. That seems more useful than any video analytic we have discussed so far.

Here's an AP article, republished by SIW, that has this gem:

"David Antar of New York-based IPVideo Corporation says video surveillance can be set up to trigger warnings if bags are left unattended or suspicious activity takes place before or during a large-scale event."

I thought everyone abandoned this idiotic idea years ago?

Hey he said it can be, not that it should be or that it'll work if you try!

The Wall Street Journal has a new survey article on the debate for more surveillance coming from the bombings. It rehashes standard issues but adds nothing terribly new, good or bad.