Boston Bombings - Outrageous Manufacturer Claims

The best way an analytic might be able to help is only if you already have a person or item of interest. And if you were able to collect all video and pictures of the area in one place (exported video from surveillance system, YouTube and picture postings made on the Internet, any other video turned in by bystanders), like a folder on a computer, and have that analytic search all that video and pictures in the folder for any other instances of that person or item, you might be able to piecemeal together the person's movements and possibly find clues as to who they were and maybe even if anyone else was involved.

That would be an "offline" use per say, and would not require any integration with a system.

'Could analytics help?' is a good question. These before and after pictures look just likes scenes in "object left behind/removed" demos.

Wow, what a pain it would be to get all that video all in one place, no?

Carlton, is that TV station claiming the highlighted object is the bomb?

"Any analytics that can help here?"

I still believe there is NOTHING that could have done anything preventative in this (or similar) situations. Not enough camera coverage, too much crowd density, too many things going on.

For the post-facto investigation, this is something that Briefcam addresses. They can gather up lots of video, process it, and then provide options to control playback speed and object density.

These situations still require a lot of human intelligence. You're dealing with fuzzy shots, uneven whitebalanc and exposure across different cameras, time stamp issues, and so forth.

The TV station was saying it was a suspicious item in the same spot where a bomb exploded.

I just caught the tail end of Fox News interviewing the ObjectVideo CEO on analytics and Boston. I have no idea what he was saying because I only saw the last few seconds but if I can find it online, and it's anything interesting, I'll post it on here.

Maybe he was saying if anyone was attempting to use analytics on any of the videos they may be facing a lawsuit from Object Video.

That's funny!

Here's a compilation of pictures and videos being analyzed by people over the Internet. It will be interesting to see if any, or how many, identified the actual bombers.

Given the false reports so far, take this with some skepticism but the NYPost is reporting that they have obtained an email saying that the two guys in the photo at the top of this discussion are now wanted suspects.

If they turn out to be the bombers, this would be a pretty amazing milestone in the use of personal cameras and Internet investigations to identify criminals.

[UPDATE: This has been verified as FALSE.]

And now, people on Reddit are saying these claims are false: "It's a high school athlete and his track coach. He went to talk to the police an hour ago."

Here's the FBI's video releasing images of suspect. From what I can see, it looks like most images are from surveillance cameras.

And here's a screencap of one of the suspects from the video:

Here's the FBI's 'high resolution' images of the suspects though they are all relatively fuzzy.

Video on Fox News: Facial recognition and reconstructing Boston attack.

Anybody know of this company?

The former President of Oncam Global says analytics could have prevented this. Video below:

The NY Times has an article citing 3VR's CEO:

"Even without a name, [3VR's CEO] said, investigators could program multiple cameras at airports and elsewhere with the suspects’ images so the cameras would send an alert to them if someone resembling a suspect passed by."

And Bloomberg has a video with FaceFirst's CEO making all sorts of amazing claims:

Slate magazine/website calls for more surveillance cameras, citing, among other things, BRS Lab's promotional material :)

Ann Coulter calls for more surveillance cameras.

I heard an interview with 3VR's CEO on NPR on way to work this morning. He was saying talking about using analytics to sift through video for clues, but to his credit he also talked about the limitations of analytics and actually said analytics have overpromised and under-delivered over the years. They did not say directly, but it seemed to imply 3VR analytics were used to sift through video.

I'll refrain on commenting on what Ann Coulter thinks.

Luis, thanks for the feedback. I am sure 3VR analytics are used to sift through video. What video and what use it is are the real questions?

Here's an article from MIT Technology Review. Quote from Michigan State professor:

"Attempting facial recognition on images like those released by the FBI Thursday is out of the question, [the professor] says. However, there may be other images and videos available that contain a better view that could be high quality enough, he says. “You could search all the other images based on clothing,” he says, “[and then] you could locate the same person and collect multiple images.”

I agree with the former, on it being out of the question for face recognition with those images. However, while you could do a search for color, there are an immense number of people wearing those colors, so it might help a little.

Kudos to the 3VR marketing department who got CNET to run what's little more than an advertorial for them:

The video concludes with a dismissal of a core limitation, noting 'while a hat or sunglasses can stump some cameras, engineers are already working to counteract" that. Lulz. They maybe working on it but I doubt they are solving it.

Here's USA Today on surveillance technology with basically another ad for 3VR. Money quote:

If the FBI has a clear video or photo image of a suspect, it's possible to quickly search through surveillance tapes from businesses that already use searchable video systems, which contain a ready-made catalogue of all individuals video taped, indexed by their facial characteristics. Search requests will turn up a list of close matches, leaving it up to the human investigator to make the final call, [3VR's CEO] says.

Two things:

• Even if this works perfectly (it's nowhere close but let's assume), less than 1% of deployments have 'searchable video systems' for the type of metadata that 3VR is suggesting. So unless the plan is to standardize Boston on 3VR, it's not much of a practical recommendation
• The camera positioning restrictions would make 95% of cameras ineligible for this type of analytic anyway (see our review of 3VR's gender/demographic analytics).

This one is kind of interesting, because the quotes appear to be from a US Park Police member.

It's also the first mention of Briefcam I've seen in print, which is maybe the only video analysis program that actually could be of help, simply to make searching faster. Even still, operating in such crowded scenes is hard, at best, and impossible (and irresponsible to claim) at worst.

All of these articles use lots of sentences describing what various software "could" do, not what it "did".

Briefcam still seems to me like the only real practical option here, and even that is no magic bullet.

What would a BRS system do in this case? This day was essentially a complete anomoly from start to finish (as it would have been even if it went perfectly normal). I find it hard to believe that any truly useful data would have come out of a video analytics platform meant to spot rogue activity (rules based or not). I also don't see this changing much in the near future. It's a really really hard problem to solve, and statistically speaking we don't see enough of these events (thankfully) to cause people to want to budget for a solution, which discourages anyone from attempting to solve this problem.

It would certainly be nice if some of what we saw on TV (or what bored reporters dreamed up) was true, but I think this is going to be human-analytics, assisted at times by tools like Briefcam, invetigating these kinds of events. The computers aren't going to find the red shirts and black backpacks for us.

SRI has an IR based system called VerifIR that I saw demonstrated at last years ISC West.

It tries to give the viewer a picture of what's under clothing and I even saw a video of it showing a shallow buried device under dirt. The rep at the show walked in front of the camera and I could see his threat resistant vest under his shirt.

So might this have helped as a preventive measure? Some caveats.

... How close would a person have to be to the camera to make out an object?

... Would require active viewing, I think. The picture doesn't look clear enough to really use an analytic to detect threats, and in this case, would a pressure cooker have been obvious enough to raise an alarm?

Luis, thanks, that's interesting. I embedded a video for the VerifIR in your comment that shows demonstrations of it. That seems more useful than any video analytic we have discussed so far.

Here's an AP article, republished by SIW, that has this gem:

"David Antar of New York-based IPVideo Corporation says video surveillance can be set up to trigger warnings if bags are left unattended or suspicious activity takes place before or during a large-scale event."

I thought everyone abandoned this idiotic idea years ago?

The Wall Street Journal has a new survey article on the debate for more surveillance coming from the bombings. It rehashes standard issues but adds nothing terribly new, good or bad.

Paraphrasing a role model of mine, nowhere did he we use the word 'actual'...

[VerifIR] looks cool, but it looks like you need relatively tight shots to get much value out of it. It would seem nearly impossible to effectively scan dense crowds dispersed over a large area.

Technorati claimed, "Facial recognition technology software like that made by the company 3VR has been useful in helping police and the FBI track down the two suspects in the Boston Marathon bombing."

They offer no evidence how it was used, or if 3VR was used. Amazing, though, again, what a job by 3VR to get the press to give them credit. But holy...

Reuters has a contrarian take, claiming that, "Search for Boston bombers likely relied on eyes, not software,"

Money quote: "They probably just applied a lot of eyeballs to this problem," says Todd Waits, digital investigation and intelligence director at Carnegie Mellon University's Software Engineering Institute and a specialist in forensic video techniques.

Oh brother. Here's a face recognition vendor claiming in a Boston Bombing article, "“Facial recognition is as good as a fingerprint.... Technology itself has improved significantly so that the false acceptance is less than 0.1 percent in an ideal situation.”

If only the world was an 'ideal situation'...

A UK newspaper cites a biometrics manufacturer making a particularly good point about the impact of the brother's youth:

"People's faces change a lot," Mr Carr-Archer said, especially between early adolescence and the mid-twenties. If one of the suspects had applied for a driving licence aged 15 (as is legal in the US), his face may have changed considerably from the image portrayed on it, he said. "If you have a ten-year-old photo on a passport, you can look very different."

The Mercury News, in Silicon Valley, has a fairly balanced article on face recognition and the Boston manhunt. Interestingly, they do not make any claims or insinuations that it is being used in Boston. Rather they discuss the future of it and potential issues, noting:

"The products promise to one day revolutionize crime fighting. Although experts say police won't fully embrace the software until it becomes more sophisticated, they also say that day is not far away."

In fairness, the same exact claims were made in 2005. I remember...

One thing that will be interesting to analyze carefully (Carlton) is what percentage of the video evidence came from surveillance vs from smartphones.

The original photos were clearly from surveillance cameras but they had very bad angles as they were from store surveillance cameras. It appears that the most detailed / straight on pictures were from smartphones, like:

And

And

Here's a question: How would the investigation be impacted if smartphone evidence was not available?

Tragically, the best evidence is still likely to come from first hand sources:

"Minutes before the bombs blew up, Jeff Bauman looked into the face of one of the suspects, his brother Chris Bauman told Bloomberg News. Two and half minutes after the man dropped a bag at his feet, the bag exploded, tearing Jeff Bauman’s legs apart.

“He woke up under so much drugs, asked for a paper and pen and wrote, ‘bag, saw the guy, looked right at me,’” Chris Bauman said yesterday in an interview. While still in intensive care, Jeff Bauman gave the FBI a description of the man he saw, his brother said."

With such a large crowd, having an eyewitness account most likely did far more to narrow the potential leads than any surveillance. While our industry continues to harp on how important surveillance was, let's remember the role this victim played in identifying the bombers.

Ugh, evidently ObjectVideo can prevent future bombings. Featured on FoxNews (at the 5:30 mark of this video), their Chairman says, "Technology is available today where cameras can automatically detect someone leaving a backpack behind" while showing simplistic scenes totally unlike the Boston bombing, e.g.,

Just terrible...

Boston Globe editorial today - spinning yarns:

"Given the horrors at the Marathon finish, Boston officials should be keeping close tabs on new surveillance software that puts emphasis on prevention as well as detection. New systems have been designed to alert officials of unusual behavior. Such a program, for example, might have flagged the Marathon terrorists when they set down their packs and walked away."

Good find, Marty. This shows that manufacturer 'investment' in 'emergency PR' has benefits (to themselves at least).

"might have flagged the Marathon terrorists"

Yeah, it "might" have, just like it also might have suggested a good wine pairing for a pleasant souffle or spontaneously created a bridal registry list.

The inflated claims by these attention-whoring video analytics vendors are bad for the entire industry. Their inability to deliver on the fantanstic claims will collateralize everyone in the business when they do not stop anything the next time this happens. (God forbid.)

Completely agree with Brian (both of them).

While it sure seems that the analytics PR folks are working overtime after the Boston Marathon bombing, do a simple google news search for the word 'surveillance'. EVERY story regarding municipal systems since April 16th references the Boston Marathon bombing as the reasoning behind the need for more cameras....

The flawed logic driving this wave of disinformation may boost short-term sales in our industry, but as Mr. Rhodes points out, "fantanstic claims will collateralize everyone in the business when they do not stop anything the next time this happens."

Here's a panomera of the scene from an iPhone 90 minutes before the bombing. It's incredibly detailed, and might be 28MPs.

Wired has a story on how this was used in the investigation.

Chopper born thermal cameras were used to help track down Dzhokar Tsarvaev hiding in boat:

Track down? I thought the guy who owned the boat went out for a smoke and saw blood smears and a tear in his boat cover and he looked inside and saw something. Thermal imaging confirmed his presence maybe..... :)

So it wasn't cell phone triangulation, tracking dogs, facial recognition or any other modern technology that caught suspect #2..... it was smoking.

^^^ good point, I changed my language! :)

Bloomberg rhetoric mirrors Globe editorial - almost verbatim:

"..the network now has the ability to 'alert police to abnormalities it detects on the street', such as an abandoned package that is left on a corner."

BRS Labs marketing & PR people do cartwheels every time they read someone singing the lyrics they wrote... :)

Boston's police commissioner confirms that facial recognition failed to identify the boston bombing suspects. It's a good thing PR people don't let facts get in the way of their promotions!

It was fascinating following a thread on the whole thing on one forum, where guys were constantly posting info/reports/speculation from various news sources they were following... many conflicting. I ended up just following @Boston_Police on twitter as being the only accurate source of information.

You know, the thing that gets me about all the analytics claims, about being able to spot a dropped backpack etc. besides the fact that the sample videos like above occur in nearly-empty areas... is that there's no mention of how the system would discriminate between a "suspicious backpack", and say, a non-suspicious diaper bag... or someone's forgotten shopping... or any number of hundreds if not thousands of perfectly innocuous backpacks one might see with that many people out and about. Not to mention, you're near the finish line of the race where there are bound to be LOTS of gym bags with changes of clothes in them. Can you imagine all the false alarms going off and the bomb squad detonating someone's sweaty runners?

Of course, no sooner will they have everything programmed to watch for suspicious backpacks, than the next bombers will use diaper bags... or shopping bags... or a baby stroller... or any of a thousand other everyday objects that one wouldn't be surprised to see sitting around behind the scenes.

On a side note... the Vancouver Sun Run, which annually draws 50,000-60,000 runners (twice the size of the Boston Marathon, although the Sun Run is only a 10km race), happens today. There was a lot of discussion last Sunday about whether the race would go ahead, but the organizers were adamant that aside from some extra vigilance, there were no plans to alter or cancel the run. And for the rest of the week... daily registrations doubled over last year, with people signing up to show support for Boston, to the point where they're saying they'll have a record number of runners this year.

"But Say with a Cray Computer with billions of processes ( Algorithyms) going on at a time, you could run processes that would allow to account for these scenios and make high level dicisions to differ between True & False scenieros."

I highly doubt that.

Yet another BRS reference - Minneapolis Star Tribune

"It then alerts officials when something strays from normalcy — people who are loitering instead of getting about their business, people who are jumping turnstiles, and folks who drop a package and then walk away."

In the marathon scenario, there is no 'normalcy', everyone is loitering and there are backpacks and sport bags everywhere. Definitely would've caught any turnstile jumpers though.

Their marketing machine is well-oiled.... that is quite a few major metro papers (and leaders) chanting their marketing talking points in their editorial/opinion pages.

How many people have lost their mind?

An executive at the NRF claimed that, essentially, Hollywood science fiction enhancement has become a reality:

“Now the ability to enhance the video, and as we have all seen by first images of the Boston bombing, those images were better and more clearer as the hours passed. So the first images of individual faces, that gets enhanced over a computer system,” he said. “Those fuzzy black and white images of people for which we cannot make an identification now have been enhanced to the point they keep refining the image so it’s so clear that they can make an identification.”

I am dumbfounded. I have not heard anyone else make this claim. The closest thing I can think is that he is confusing it with other images being found from other cameras that have a narrower FoV, better angle or higher resolution.

It was an interesting week on the radio talk circuit here, with a lot of calls for the re-implementation of the thousands of surveillance cameras that were installed for the Olympics, then removed when it was all done. Naturally, there were the typical detractors as well, ranging from the "they're not a deterrent anyway" types to the "Big Brother" and "police state" crowd. In the end, no extra security was added, and the Sun Run went off without a single hitch, with 48,196 participants.

I expect there's a lot of push in most centers to start putting up more cameras - even aside from the marketing claims we're seeing, people still realize that surveillance DID play SOME role in tracking down the bombers, and if nothing else, made it a lot easier for Joe Average to see how the investigation was progressing. It will be interesting, going forward, to see what kind of longevity the calls for more cameras have - whether they simply peter out in some areas, or are actually realized in others after prolonged public pressure... or shot down by public pressure, for that matter.

The WSJ with a cringeworthy column that shows even the biggest name publications can get their 'information' from little more than rehashed PR.

He calls video analytics 'new technology', which is amazing since it's been commercially offered for at least a decade and widely derided for the last 5 years.

And the top providers?

"The technology, from companies with names like IPVideo Corp. and ObjectVideo, is still new."

Even OV's lawsuits are no longer 'new'! And how the hell is IPvideo Corp being cited as a leading video analytics provider? The only connection is that these two companies have done a good job inserting themselves into the recent media coverage.

Alas, this is how the public is being 'informed'.

Here's another article on CNBC from Jim Albers, senior vice president of MorphoTrust. In it, he claims:

"The technology has come a long way in just the last five years and now there are features that will allow users to clean up even low-resolution photos.

For example, if law enforcement only had the profile of an individual in a picture, they could use MorphoTrust's FR software to create an image of what the individual would look like if his whole face was shown in the picture. The software does this by basically copying the visible side and recreating it on the other side. It won't be an exact match, but it will be close to what the individual actually looks like, Albers said."

That's more of the 2D to 3D conversion claims, which would be great if people had perfectly symmetrical faces. Also, that's just for the angle problem, not low resolution.