Axis Perimeter Defender Video Analytics Tested

By IPVM Team, Published Jul 12, 2018, 10:00am EDT (Research)

**** '**** ********' ***** ********* ******** is ********* ********, ***** / ********* **** ******* ********.

*** *** **** ** ********* ********? And *** **** ** ******* ****** ************ ***** *********?

** ****** ********* ******** ** ****** and ******* ****** ** **** ***, using *** *******:

 

** ****** ** ******** ****** ******** and *******, *********:

  • ************* *** ***********
  • ***** ***** ***********
  • ********* *****
  • ********* *********
  • ********** ** **** **** ******** ***** *********
  • ****** *************
  • *** ***********

******* ****** **** *** *** ***** updated ****** ********* *****, ********* **** ***** *****, **** ****, ***** *** ***** ********* *** ******+ VMD, ****** ******* * *********, ********* ********* *********, *** ***** ********* *********.

*******

**** ********* ******** **** ******* **** problematic *** '**** ********'. ***** ** performed ****** **** *** **** **** and *** ~$** ***** *****, ** still *** *********** ***** ****** ********. Given *** ******* ***** *** **** 'high ********' ****'* ************ *** ****-**** false ******, ********* ******** ***** ***** problematic.

****, *************, ** *** ******** ******** will ****, ***** *** * ***** vendors **** ******* ****** ********** ********* for ***** **** **** **** ********, low ***** ***** *********.

** *** *** *********:

  • ******* ******* ***** ******:********* ******** *** *** ******* ** many ****** ******* ** ***** ****** outdoors, **** ** ***** *******, ******* lights, ** *******.
  • **** ********* *****:********* ********'* ********* ***** *** ****** than **** *********** ********* ******, **** detection ** **** ~*-* ***, ***** most ********* ******** **+ *** ******** detection.
  • ****/******** ***********:********* *********** ****** ******* *** ********** in *** *****, **** *****-***** ******* within * *** ****** ** ****** size, *** ****** ***** ******** *******, and *** ******* ********** *********.

*******, ***** **** *** *********:

  • ********** ***** ****** ***********: **** ********* ***** ** ****** ***** in **** ****** *****, ********* ******** frequently ********* ***** ******. ****** **** reduced ** **** **** *** *** day **** ******** ***** ***** **** the ********* ****.
  • **** ***** ******:******* ** **** *********** ****** *********, Perimeter ******** ********* ********** ** **** dripping **** *** ******'* ****, **** multiple ****** **** ****** ******** ****** over *** ****** ** ******* (**** two *****).
  • ******* ***** **** **** *****: **** ******* *** ********* ******* ** additional ******** ********* ** ********** ** order ** ********* ******** ** *** VMS, ** ********** **** *** ******** when ***** ***** ** ** *******, nor ******** ** **** ***** ****** analytics ****** (***** ***, ****, ******, etc.). 

****: ** **** * ****** **** of *** **** ********* ** ** Axis ******* ******, ****** *** ************* higher ***** ***** ** ******* **** limit *************, ********** ** *******.

Vs. ***** *****/****

********* ******** *********** ****' ***** ********* (VMD4/Motion *****) ** **** ***** ***** resistance *** ********* *****. **** *** Guard ***** **** **** *********** ** blowing *****, *******, *** ********** ***** animals. ********, *** ****** *** ******** at **-** ***, ***** ********* ******** needed **** *-* *** *** ******** detection.

Camera *************

********* ******** ** ************** **** ** ****' ******* * and * ****** ****** *** ****** M ****** ******. **** **** ****' compatible ******* ******* ** *** ** to ****, *** ** ******** ************* with **** ******* ********** ****** *******. Additionally, **** **** **** *** ****** performance ** ***** ****** ***** ** customer ******* *** ** **** ** recommend ***** ******* **** ** *** desired ***** ** *** ******.

**** ****** ******* ** *** ***** as ***** ** ***** ***** ***** or **** ************, ***** ******* ****** all ******* ******, *** ****** **** mid ** **** *** ****** ***** are ****** ** ** **** ** these ************.

*******

********* ******** ******** **** *** ~$*** USD *** ******. ****** ******** *** ten ***** *** *********. **** ******* is ****** **** ****' *** ***** Suite (~$** *** ***********). **** *********** include ***** ********* ** ** ********** charge **** *** ******** ** ***** cameras.

Separate Application *** *************

********* ******** ** ********** ***** * separate ***********, *** ** *** *******' web *********, ******* ** ***** *** and ********'* ** *********. **** *********** does *** **** ** *** **** configured, **** ****** *** ******** **** directly **** *** ******* ** ********** VMS.

Automatic ***********

********* ******** **** ** ********* *********** method ***** ******** ***** ** * subject ******* ********** *** ***** ** view ** ********* *** *****. ***** calibration, ***** *** ********* **** ** image *** ************, ******* *** ******* at ******* ******, *** ******* *** horizon (***** ** ****), ***** **** a "*********" ****** ***** ********* *** camera's ********** ** **** ***********. ***** calibration, ***** *** **** **** *** recorded **** **** *** *********** **** the ******** ** **** ********* ** these ******.

** *** *****, ********* *********** ****** properly *** ********** ***** **** ** was ********* *******, **** ******* **** very ***** ** *** ******. ** smaller ****** ******, ****** *********** *** required, ** *** ******** ***** *** properly interpret *** *****.

** ****** *** *********** ******* ** this *****:

Improved ***** ***** **********

****** ****' ***** *********, ********* ******** *** not ****** **** ***** ****** ** many ** *** ****** ******* *******. For *******, ******* *** *** ***** on ***** ******* ** *** **** field ** ****, ****** ** **** and ***** *****.

****** ******* *** *** ******* *** alarms ** *** *****, ******, ***** below.

*******, ********* ******** *** *********** ****** with ***** ****** ****** ** ******* brush, **** ******** ****** ** **** lightly ***** **** (***** *****). **** when ******** ***** ***** **** *** detection ****, **** ****** **** ***** triggered (~* *** ***) *** ** brush ******.

************, **** ***** ********* ****** ***** alerts, **** ** *** ******* ****** down *** **** *****, **** ******* false ****** ****** ******** ****** ** the ****** ** *******.

Solid ****** ***********

******** ****** ******** ** ****** ******* areas, ********* ******** *** **** ** used ******* (**** ****** ***********).

*******, ********* ******** *** *** ***** on *** ******* ***** ***** *******, such ** ****** ******* ** *** off, ***** *****:

 

************, ** ****** **** ********* **** monitors ****** **/***, * ****** ****** false ***** ****** ** ***** *****:

*** **** ****** ** ******** ***** ******* shadows/reflections:

Detection *********

********* ******** ************ ******* ** *** subject ** ~*-* *** ****** *** and ***** ***** ******** ** *** following ***** ** **** ******/*********. ***** ranges **** **** ** *** ******* seen ** *** *****, ****** **** most ***** *********, **** *** ********* of ***** (~*-* *** ** **** scenes). ******* ****** ********* ****** **** ~10 *** ** **** **. 

  • *****-*** (***/*:*): ~***' ****/~***' ********
  • *****-*** ** ** (***/**:*): ~***' ****/~***' ********

Detection *********

********* ******** ******** **** ********* *********:

  • *********: ******** **** ** ****** ****** a ****
  • **** ********: ******** **** **** ******* cross **** *** ******** **** **** another
  • ***********: ******** **** ******* ***** * zone ******* ***** ******** *******
  • *********: ******** ***** * ******** ****** of ****

** ****** ******** ******** ** **** video:

** *** *****, ***** **** ***** rules ****** ******* *****, **** ****** and ***** ***** *********** ******* ** intrusion. *** *******, ** *** **** crossing ***** *****, *** ****** **** alarms **** *** ******* ***** **** the ********* **** ** *** **** closer ** *** ********. 

VMS ***********

********* ******** *********** ****** ********* ** *** VMS, **** ******* *** ********* ********** both ****** *** ******** *****, ***** Exacq *** ******* ***** ******* **** events, **** ** ******** ***** *********. Avigilon ******* ****** **** *** ******* either.

Milestone/Genetec ***********

** ********* *** *******, ******* *** overlaid **** **** *** ******** *** information, ** **** ** * *****/*** alarm ********* ** *** *** ***** corner, ***** *** **** ** ********'* attention ****** **** ******* ******** ***** in *****-****** *****.

****** *** **** ** **** ** create ******, ***** ****:

***** ************ ******* ********** ******** ********* on *** *** ******, ****** *** Perimeter ******** ******, ***** ********* ******** into *** ****** ****** *** **** VMS. **** **** **** ** * lightweight ***********, **** ******** ********, *** is *** ********* *********.

** ******** ** *** ******, ******* users **** **** *** ** **** SDK ******** ********* ** ***** ********* machine ** ***** ** ******* ********* Defender ******, * ~$*** **** *******.

Exacq/Network ***** ***********

** ***** *** ******* *****, ********* Defender ** **** ********** ***** ******, with ** ******** ***** *********. ***** integrations ** *** ******* ********** ******** and **** *** ** *** ***. In **** *****, ***** **** **** select ********* ******** ** ** ***** trigger ** ***** *********, **** *************, trigger ******, ***., ***** ** ***** here:

Versions ****

*** ********* ******** *** ******** ******** were **** ****** *******:

  • **** *****-***: *.**.*
  • **** *****-*** ****: *.**.*
  • ********* ********: *.*.*.*****

 

Comments (15)

Undisclosed Integrator #1

07/12/18 02:54pm

We use this - (well what it really is - Safezone edge from Digital barriers ;) ). Its usually spot on. The bush detection shouldn't have been an issue as if the zone was set correct and drawn around to avoid the area IMHO.

You can easily do a "lite" install to most VMS's by sending TCP notifications in to the VMS. The more complicated full version is more work but also quite useful on larger installations.

The only issue we get is reflections, so if you have a large vehicle near a road the anayltics will see a mirror of this and trigger, same for people walking past. Something any analytic will suffer - poor positioning.

Agree
Disagree
Informative: 1
Unhelpful
Funny

John Honovich

IPVM | In reply to Undisclosed Integrator #1 | 07/12/18 03:01pm

The bush detection shouldn't have been an issue as if the zone was set correct and drawn around to avoid the area IMHO.

I'll let Ethan / Rob talk about the specific test results with 'bush detection' but I want to emphasize a fundamental point.

If an analytic system cannot tell the difference between leaves and a human, that's a major constraint. There are simply too many areas and applications where leaves and trees will be an issue. It also shows the analytics 'intelligence' is quite rudimentary.

Agree: 3
Disagree
Informative
Unhelpful
Funny

Undisclosed Integrator #1

In reply to John Honovich | 07/12/18 03:17pm

bush or human - that is a fair point! lol :)

Agree: 1
Disagree
Informative
Unhelpful
Funny
Avatar

Ethan Ace

IPVM | In reply to Undisclosed Integrator #1 | 07/12/18 03:03pm

We removed the brush from the zone during testing and it improved, but did not eliminate false alerts. We still ended up with ~1 per day as the bush encroached too near the zone drawn strictly on the ground.

Yes, it's true that you can draw it further still from the brush, but in real world scenes, this is not always possible, as it may narrow a zone too much and risk missing detections, or brush may simply be in the middle of a zone.

Moreover, others such as Bosch IVA and Avigilon (test coming) did not trigger on these same areas.

Agree
Disagree
Informative: 3
Unhelpful
Funny

Undisclosed Manufacturer #2

07/13/18 12:05pm

Hi Ethan,

i have a question regarding the methodology of the testing. In full disclosure I am a field guy with one of the manufacturers tested.

In all of the tests so far, the cameras have been set to a wide field of view.  I suspect that towards the back of the field of view the stated capabilities of some or all of the analytics are exceeded.  My guess would be that you are attempting to gauge the ultimate detection performance (ppf) on a level playing field?  I’m wondering how you weigh false detections outside of the design envelope of the analytics? I don’t know if there were any occurrences with any of the manufacturers, but I was wondering if it was a consideration.

Thank you

Agree
Disagree
Informative: 2
Unhelpful
Funny
Avatar

Ethan Ace

IPVM | In reply to Undisclosed Manufacturer #2 | 07/13/18 05:09pm

The false alerts we have seen in this as well as other tests on brush have almost all been triggered by brush in the near field of view, not at long ranges. Additionally, we have tested all the analytics in this series in both wide and narrow fields of view to check differences, and performance has been fairly consistent in both wide and narrow AOVs in terms of false alerts. Meaning, if a camera has issues with brush using wide fields of view, it also had issues in narrower fields of view.

Also worth noting, we have discussed our findings with manufacturers as we've been testing these analytics, both tech support and product management where possible. We have adjusted settings and retested based on that feedback, as well. So manufacturers have had opportunity to point out whether we were testing outside of their capabilities.

Agree
Disagree
Informative: 1
Unhelpful
Funny
Avatar

Rafael Oneda

07/13/18 01:03pm

"Axis Perimeter Defender test results were problematic for 'high security'"... I think any IVA will be problematic for outdoor applications if were used visual cameras, because there are smoke, dust, illumination issues, rain, etc.. a lot of potential troubles that will inevitably produce false alerts. Besides that, if a simple bug lend at the lenses, your detection system will fail... In my oppinion, a system with these fragility can not be used in an 'high security' application.

In my projects, I avoid to use IVA in outdoor and usually use auxiliar systems to detect acuratelly.

Agree
Disagree: 2
Informative
Unhelpful
Funny

Undisclosed Integrator #3

In reply to Rafael Oneda | 07/16/18 12:16pm

Good point Rafael, in addition we have seen Axis AVMD 4 completely miss a person walking across a parking lot on a few occasions.  We tested it side by side in a real environment (office parking, docks) and it missed too many people to use effectively although it did remove a ton of non-critical alerts.  Missing the few real people is a big deal, especially if you're not doing a secondary full time recording.

Agree: 1
Disagree
Informative: 1
Unhelpful
Funny
Avatar

Rafael Oneda

In reply to Undisclosed Integrator #3 | 07/16/18 12:39pm

Let me make clear my comment. I was no criticizing Axis specifically, but, the technology of Video Analysis (does not matter the manufacturer). I dont think it is a good technology for outdoor applications in 'high security' systems using visual cameras. There are several non visual systems much more reliable for this.

 

 

Agree: 1
Disagree: 1
Informative
Unhelpful
Funny
Avatar

Geoffrey Leader

In reply to Rafael Oneda | 07/16/18 01:52pm

 I dont think it is a good technology for outdoor applications in 'high security' systems using visual cameras.

I agree entirely.  The problem is that software creators, vendors, and even trade magazines are all over how wonderful video analytics are.  This means that when I tell our director and GSOC leadership that I do not recommend analytics because they are not sufficient to remove live monitoring and onsite security for critical locations I get a very aggitated "so I guess xyz vendor, videos online, and the news articles I read are all lying about its effectiveness?"

Agree
Disagree
Informative
Unhelpful
Funny: 1
Avatar

Rafael Oneda

In reply to Geoffrey Leader | 07/16/18 03:03pm

As I writed in a comment of this article, the most embarrassing situation is when I need to desagree about customer's video analytics wrong perception of effectiveness. The exaggerated marketing (sometimes almost unreal), creates in the customer's mind a unattainable level of accurance, and the installer needs to achieve it... so, it is a frustrated customers industry.
I'm sure that IVA technology has a huge potential for video surveillance applicantions, and, there are some great applications right now, mainlly in retail (that can tolerates a bit less accurance) and indoor controlled environments. But, I dont fell fine in use it for outdoor security applicantions, yet.
I think when deep learning video analysis, like Camio and Briefcam, who can really 'understand' what is happening, not just see a mass of pixels crossing a line or entering in a forbiden area, start to be viable to real time applincations, we will experience a revolution in the video surveillance market, but, while it does not happen, I keep using non visual systems to detect in outdoor.

Agree
Disagree
Informative
Unhelpful
Funny
Avatar

Sergio Guzman

Pine Crest School
IPVMU Certified | 08/23/19 09:58pm

I have a 50 acres campus I manage. We're currently thinking of using Perimeter Defender. I was wondering how well it performs today a year later?

Agree
Disagree
Informative
Unhelpful
Funny

Undisclosed Manufacturer #4

09/04/19 07:49pm

How about the other available perimeter VCA options on Axis ACAP? There are quite a few of them besides Perimeter Defender. For example Agent Vi. Is there any experience with these other options, how well they perform against some of the criteria in these tests?

Agree
Disagree
Informative
Unhelpful
Funny
Avatar

Rob Kilpatrick

IPVM | IPVMU Certified | 09/04/19 08:25pm

UM4, we have not tested Agent Vi yet but I will queue it up internally for testing.

The only other options we have tested are VMD4 and Guard Suite, which did not perform as well as Perimeter defender, which we noted in the report.

Agree
Disagree
Informative
Unhelpful
Funny

Undisclosed Manufacturer #4

09/05/19 08:42am

That sounds great, thanks Rob!

If I may, here's a wish-list

AgentVi, savVi (listed on the the AgentVi website, but no longer in the ACAP portal, wonder why)

Digital Barriers, SafeZone-edge (what Perimeter Defender is based on, is it better in original form?)

Jemez Technology, Eagle-i-Edge (I heard Axis was testing this on display in one of their experience centers a while back)

Two other criteria I might submit are

1. VCA + Thermal camera performance (a common combination at perimeters)

2. The option for “burning in” the metadata bounding boxes into the image itself before leaving the camera. This is bad practice in some circles, but does ensure that the alarm and bounding boxes show up at the monitoring station (regardless of PSIM) and they they are in sync – this is still limited.

Agree
Disagree
Informative
Unhelpful
Funny
Subscribe to IPVM Research to read the full report.
Why do I need to subscribe?
The IPVM Research Service includes products tests and shootouts plus competitive and financial analysis, helping decision-makers better evaluate purchasing, partnering, developing, and/or competing against companies in physical security.
Already have an account?
Loading Related Reports