Healthy Skepticism for Deep Learning Is Prudent

Intellectual property is a good question.

For example, I never though 'metadata' or the concept of a 'tripwire' could be patented but then Object Video did it.

I do not know the answer but we will ask around for what different players are claiming (e.g., is deep learning video surveillance 'covered' by AV/OV's patents?). @Karas, see what you find.

It is a complex topic, and we are at the early stages.

Where many video analytics products struggled (or outright failed) was in doing accurate object classification. Many products tried various end-around approaches to get to a reliable method of separating human from non-human (pixel blob sizes, blob movement tracking, onboard tilt-sensors coupled with mounting height data to determine rough horizon,  etc.)

If DNN-based approaches can output reliable data on what an object is, then other things can be built on top of that in a simpler/easier/more reliable manner.

In some ways, DNN could be the embedded linux of analytics. Before linux, it was costly to build devices with an embedded operating system, as options were limited, complex, and expensive. Linux solved a stage 1 type of problem for embedded devices, letting companies concentrate on the device/functionality part more than the OS (and no point getting into side-tracks here of cyber security or other things embedded device companies still get wrong).

Applying DNN to analytics could get the industry over a major hurdle, but you still will need strong developers (and marketing/sales) for the downstream stuff that comes after you have figured out what an object in an image is.

Currently, this is more of a theoretical risk than an actual one, particularly for security applications. This could change if DNN-based analytics become far more mainstream and accessible, as explained below.

I can't/won't comment on the "fool a Tesla" examples because I do not know enough about how autonomous vehicle analytics work at a low level.

For security applications, the adversarial attack would typically be some form of digital camouflage, so that a given object (most likely a person) could evade detection by the system. This could be to make the human register as "cat", or to simply evade detection altogether (probably a better scenario to be invisible vs. just mis-categorized).

There are a few things that make it difficult/less rewarding to do an adversarial attack for surveillance:

1. Security analytics are generally not open source/downloadable/easily accessed for hacking around on. Manufacturers develop their algorithms in-house, with closed teams. The channel control within security, where most of these products cannot be easily bought online also makes it harder to attackers to get access to a system for R&D/testing purposes. Not saying it is impossible, but it also less likely a casual hacker has the ability to test against many of these products.

2. Better analytics take several samples of an object before actually giving it a classification. Security systems have the benefit of generally only caring about moving objects. Also, actually analyzing an image is computationally intensive, so it helps to pre-filter things out of the image and only analyze portions that meet some base criteria. This is generally a process of: look for blob motion->analyze for rough size/shape/aspect ratio criteria->analyze for "predictable motion"->then finally run the blob through the DNN for classification. This means you may need to do more than just inject singular static images into the system to affect it.

3. Every system behaves differently, so developing an attack for one would likely not work on another, unless they were trained with the same dataset (possible, and likely to be slightly more common in the future if other trends of selling/licensing labeled data sets progress), but overall low odds.

Therefore, to fool a halfway decent security-based DNN analytics system, you would need to construct some form of visible camo that provided continuous evasion, from multiple angles and while in motion (which would include lighting variations in most scenes). If the system could properly classify the disguised person for even 1 second, it would generate an alarm.

Froom an external perspective, it is very hard to tell which, if any, analytics are in use (I guess this is also a slight detraction for Avigilon and their prominently-labelled cameras). Carrying out a successful attack with this method would require some degree of inside knowledge and a very highly motivated attacker. In general these are the kinds of applications for which video analytics, of the sort on the market today, are probably not appropriate for the level of risk, those situations are usually the sort that use hyper-active shoot-first-ask-questions-later armed guards.