Axis Live Privacy Shield Analytics Tested

By Sean Patton, Published Jun 25, 2019, 10:51am EDT

Privacy is becoming a bigger factor in video surveillance, driven both by increased public awareness and by GDPR.

Now, Axis has released Live Privacy Shield, a video analytic that Axis claims is "reliable and cost-effective" and is "the only edge-based solution that dependably masks moving people and objects in real time at up to full frame rate".

We tested Live Privacy Shield in multiple scenarios to see how it performs, including:

  • How well does it consistently mask people
  • Does it mask multiple subjects simultaneously
  • How it performs outside
  • How it performs in low light
  • How easy is it to configure
  • Pricing
  • Comparison to Genetec Privacy Protector

*******

** *** *****, **** Privacy ****** *** *** perform ****:

  • *** ******* ** ********** applied ** *** **** than ****** *******, ***** on ********* ***** ******, lighting *******, *** *****. Setting *** *********** ******* to *** ****** ******** setting **** ********* *** time ****** * *********** portion ** *** ***** of **** *** ******.
  • **** ********** ********* *** which ********* ***** ******* and ********* *********.
  • ** ** *********** ******** for ******* ******* ** it ********** ***** **** of *** ***** *** to *** ********* ** ignore / ************* ****, light, ******, ***********, ***.

************, **** ******* ****** is *********, $*** **** per ******, ***** ** the **** ********* ******* *********, ***** ********* **** in *** *****, *** also ***** *********** *** usage.

**** *** ******** ******* can ** ********** **** a ****** *** ******* if *** *** ************ '**** *****', ***** ******** * (or ****) *********** ******* per ****** ********** (*.*. 1 ****** ******, * unmasked ******). ** **** Areas *** *** *********, 2 ****** ******** ***** be ******** *** ******** viewing *** *********. ************, we ***** ** *** that ********** * ****** toggle ******* *** *** streams.

***** *** ************* **** easy, ** ***** *** minimal *******, *** **** ACAP ***** **** ** less **** * *******. It *** **** ** installed ***** **** ****** Manager ****** **** ******* at **** (**** ****** ******* ********).

Live ******* ****** ********

**** ******* ****** ***** any ****** ******* ** obstruct *********** ******* ** protect ******'* *******. **** Privacy ****** ** *** a ************ **** *** does *** ***** ********* sizes ** ***** ** masks, **** * ******/**** mask **** ********** ******** the ****** ******. ******* to ******* *********, ** blurs *** ****** ******, not **** *** ****.

**** ************* ********* ** "***** *****" has *** ********* **********:

  • ** *** * ******** light ******
  • ** ** ******* **** no ******* ***** ****** daylight *** ***** ************

**** ******* * "*****" scene **:

  • ** ** ********
  • ** *** * *** of ********** ********
  • ***** *** * *** of ******** *******
  • ** *** ******* ******* objects (*****)

** ***** **** ** default ********, **** ** "ideal" **********, ****-******* *** common, *** ****** ******** changes *** ******* ******* false **** **********.

Poor - *********** **** ********

*** ** *** ******* issues ** ***** ** testing *** ******** ******** light *** *** ***** conditions ****** ** * mask ******* ** *** majority ** *** ***** of ****. ****'* ***** scene ********** *********** ** a ******** ***** ****** and ** ******* *** restrictive ** *** ***** of *** ***** ******.

Poor - ****** ******* *** ********* *******

******* ***** ** ***** was ********* ******* ** the ***** ** **** (e.g. **** **** *****) would ** "******" *** semi-permanently ******. ***** ** a ******* *** "********** merge ****" *** ****** settings ******* **** ********** masking. *******, ** ************ with * *** ** constant ******** (*.*. *********, terminals, ****** ********), ******* settings *** *********** ** avoid ***** ***** ** artifacts:

Poor - ******* ***********

**** ******* ****** ********* poorly ** ******* ************, masking ******* ******, *** reflections ** ***********. **** when ********** ** *** lowest *********** *****, ******* was ********* ** ***** as ******* ***** ****** detection ** *** **** conditions. ******* ***** **** used ** ****** ***** of *** ***** ** view (*.*. *****, ***), identified ** *** ****** trapezoid *******:

Masking ****** *********** *********

*******, **** ******* ****** effectively ****** ****** ******* the ***** ** **** without ******* * ******, whether *******, ******* ** partially ********** ****** ***** or ********. *******, **** video ****** ********* *************, Live ******* ****** ** not *** *********** ********* method, *** ******** ** changing ***** *** ****** in **** *****.

** **** **** ** adjust *********** *** ********** merge ******** ** *********** to *** **** ******** results **** **** ********, however, **** *** ******* were **** ** ********, the **** ****** *** limitations **** *****.

Dual ****** ********* ******** *** *********

** ***** ** *** capability ** ****** ******* masked *** ******** *****, which ***** ** *** most *************** ***** (*.*., Genetec *** ********* **** it **** ***).

**** **** ******* ****** is *******, *** ******* views **** *** ****** will ** ******. **** View ***** *** **** for *******/********* *** ******** views. **** **** ** would **** ** *** VMS **** ******** **** View *****. ** ***** this ****** ******* *** special ************* ** ******* and ********* *** *** with ******* *****.

Camera ******* *******

*** ******** ** **** available ** * ******* number ** **** *******, generally ***** * *** Q ****** **** **** ARTEC-6 ** ******:

Region ************ ******* ** ****** - ******** ****** ****

**** ******* ****** ** only ********* ** ******** Europe ** ******, **** said:

** ******* ** ****** the ******** ****** ** region. ** ******** ** Northern ******, **** ** Europe **** ** *** next *****. *** **** for *** ** *** decided ***.

Comments (11)

IPVM criticizes video analytics regularly including China-developed one.

This is worse. It's bad and they charge a premium for it. Axis, come on.

Agree: 4
Disagree
Informative
Unhelpful
Funny

It's bad

As I was reading through this report, I was thinking "these sound like the base problems video analytics had 10 years ago". Problems which are (mostly) solved with some decent applications of DNNs these days. Kind of confusing why this couldn't be substantially better.

Do object classification

Refine segmentation mask

Obfuscate pixels of classified objects

It would probably still be a bit of a greedy match, "shielding" some things that weren't meant to be shielded, but should at least be able to ignore shadows, etc.

Or, I'm just totally missing what they're trying to accomplish here.

Agree: 4
Disagree
Informative
Unhelpful
Funny

I actually mistakenly assumed they were using deep learning for this (it is 2019) but Axis confirmed they are not. I don't know enough about the Axis ARTPEC chips to understand what they can do but I am assuming they cannot do object classification or else they would.

I still don't understand why they would release this, as is. The one thing I think is that, for privacy protection, it's better to err on the side of blurring out more than less but, with this setup, it blurs our way too much. I could see them offering this for free, given the issues.

Agree
Disagree
Informative: 1
Unhelpful
Funny

If they cannot do basic DNN analytics on the ARTPEC chip, it seems like they should be buying chips from Ambarella instead of trying to roll their own.

 

Agree
Disagree
Informative
Unhelpful
Funny

Axis responded saying they can do objection detection but do not because of concerns about missing a person and exposing them:

we could do object classification but for this use case (Live) it would not be reliable even if using a powerful server. Sooner or later the classification will be missed and reveal someone’s identity.

If they can do such classifications, just set the threshold low (to reduce false negatives) and I would suspect that would still be better than this simplistic VMD approach.

Axis also emphasized that they make clear the limitations in the documentation, e.g.:

For indoor areas with good, stable lighting

As well as the online manual, though our testing still showed problems with our indoor stable lighting scene. On the other hand, if users are ok with extra random masked areas, so be it.

Axis also expanded on potential applications:

In many countries it is not allowed to do any video surveillance at some places due union regulations (and GDPR is off course also to be considered). For example in the production use case the main interest is to monitor the production process (machines or conveyor belt) and not the employees.

Here is the example of the conveyor belt that was excluded from masking:

My concern would be that if a person walked in front of the excluded zone than they would be shown. It's a tough thing with such low intelligence analytics being used.

Agree
Disagree
Informative: 2
Unhelpful
Funny

Axis responded saying they can do objection detection but do not because of concerns about missing a person and exposing them:

Are they guaranteeing to NOT ever miss anything in the current version?

I have a hard time believing that you would run a significant risk of being able to ID a person that was missed with a halfway decent modern DNN analytic that was set somewhat sensitive. I do believe that such an analytic could successfully ignore the majority of the problems your report found, while not increasing the risk of missing a clearly identifiable person relative to the current version.

Here is the example of the conveyor belt that was excluded from masking:

At this point, why not just apply a privacy mask over the rest of the video and be done?

My concern would be that if a person walked in front of the excluded zone than they would be shown. It's a tough thing with such low intelligence analytics being used.

I agree. I just don't see how this sloppy privacy mask app is making things "better". It's way too sensitive, does not seem to be guaranteed to catch ALL people, and unreliable in its operation. On top of that, it costs money.

If Axis is struggling for revenue, they should have just put up a GoFundMe page or something.

Agree: 1
Disagree
Informative
Unhelpful
Funny: 3

What application is something like this useful for? It seems like it would make the system almost useless blurring out the entire person. I would think that blurring out only someones face so you can at least figure out what a person is doing. 

Agree: 5
Disagree
Informative
Unhelpful
Funny

European way of thinking....  Completely useless.  Why even waste developer time on this.  Just one more reason we have stopped promoting Axis.  Maybe they will create a doorbell that doesn't show the person's face....

Agree: 5
Disagree: 1
Informative: 2
Unhelpful: 3
Funny: 7

To paraphrase an old TV show line, the video is masked to protect the innocent.

There are a number of situations where the motion privacy feature is useful.  

Medical Facilities want to be able to see and have a record of people entering and leaving their facility, while maintaining patient privacy rights. 

Prisons have the challenge of needing to monitor areas like showers and other areas where they want to see activity while maintaining inmate privacy and sensitivity to both male and female guards who may be monitoring the area with video. 

Shipping and receiving areas where companies want to document incoming or outgoing shipments while maintaining employee privacy.

I am sure there are other examples, but these are some of the more common areas we get requests for.

With a properly configured and reliable motion privacy feature activity can be monitored while maintaining privacy.  If an event occurs that needs to be reviewed to determine what actually occurred and who the subjects were that were involved in an incident the video can be "unmasked" by either dual authentication or administrator credentials.

Geutebruck (who I work for) has had the motion privacy feature for a number of years and it works quite well in the situations described above.  To be sure, outdoor areas and shadows are challenging situations.  Hopefully, with advances in AI those situations can be mitigated if not resolved in the future.

I am not that familiar with Genetec's or Axis' "Privacy Shield". Are you able "unmask" their video in situations where you need to see what clearly occurred if there is an incident that calls for it?  

Agree
Disagree
Informative: 2
Unhelpful
Funny

Hi Mark,

to my knowledge, I think what you get is two streams - one with the privacy mask and another "normal" stream.

So, the one with the privacy shield, you have that overlay codified in the stream. So if you need to checkout the full image, you have to access the second stream.

Agree
Disagree
Informative
Unhelpful
Funny

I was referring to the Axis Privacy Shield...

Agree
Disagree
Informative: 1
Unhelpful
Funny
Read this IPVM report for free.

This article is part of IPVM's 7,114 reports and 943 tests and is only available to members. To get a one-time preview of our work, enter your work email to access the full article.

Already a member? Login here | Join now
Loading Related Reports