Axis Perimeter Defender Improves, Yet Worse Than Dahua and Wyze

By Rob Kilpatrick, Published Sep 19, 2019, 10:32am EDT

While Axis Perimeter Defender analytics improved from our 2018 testing, the market has improved much faster, with much less expensive offerings from Wyze (tested) and Dahua (Analytics+ tested) outperforming the expensive Axis offering.

free image2

Inside this report, we examine what has Axis improved, what problems still remain and where and how it is significantly worse than Axis much less expensive competitors.

Improved: ******* ******* ***** ****** **** *******/**** ********

********* ******** *** *** alarm ** ******* ** rain ** *** ** a **** ** *******. By ********, ***** *** sources **** *********** *** the **** ******** ** false ****** ** ********* Defender ** **** ***** so ********/*********** ***** ****** improves ** ****' *********** versus ******* ******** *** Bosch.

Not ********: ******* ***** ********** ** ***** ******* *******, *** **********, *** "*******" *******

*******, ** ***** **** due ** ******** ***** alerts ** ******* ******* (e.g. *****) ** *** headlights ** *** ***** of ****, ***** ****** did *** ****** ****. Additionally, *-* ***** *** night, ** "*******" ****** type *** ********, ********* on ******* **** ** a ****** **** *******/******* and ***** ***** ** no ******* ****** ** the *****. ** ******** changes ******* ***** ****** totally.

Vs. ***** *********+

******** *******'* *********+ ** ******, **** ********* ********'* false ***** ********* ** worse. *** ***** ****** rejected ***** ****** **** foliage, *******, *******, ****, etc., ******* ** ********* Defender, *** ** *** not ********* ** ******* objects ** ********** ***** triggered ****' *********. ************, Analytics+ ******** ****** *************, calibration, ** ************, *********** just ******* * **** and ******* ** ***** and/or ******* **************.

Vs. **** *** / ****

****'* **** ** ****** detection*********** **** ********* ********, ignoring *******, *******, ******, shadows, *** ***** *******. However, **** *** **** non-analytic *********** ***** **** it ********** *** ********** use *** ** ***** integration ** ***** ***** VMS.

Reduced ***** ****** ** ****** ******* *******

** *** *****, ******* false ****** **** ******* reduced ******** ** *********** ********* ******** *******. ****** ******* **** as ****** ******* *** shadows **** *** *******, previously * *********** ***** for ********* ********.

No-False-Alerts-On-Foliage-And-Shadows-Outdoors

************, **** *** ******** on *** ****** *** not ***** ***** ****** in **** *** ******* of ********* ********, ***** they ********** ********* ******** false ******.

Swaying *****/*** **********/"*******" *************** ***** *****

********* ******** ********** ******* on * **** ******* in *** ********** ** one **** *****, *********** below. ********* *********** ** this ***** ******* ****** but *** *** ********** eliminate ****.

[optimize output image]

*********, ********* ******** ********** this ** * *** and * ****** **************, and **** * ******** box ****** **.

Flag Classified As Person And Vehicle Simultaneously

******* ****** ** ***** alerts **** ****** *** headlights ** *** ********, shown ** *** **** back ** *** ***** below. ***** ****** **** often ********** ** "***** of ******" ** "*******." Settings ******* *** *** mitigate ***** ******.

[optimize output image]

*******, ****** ****** ********** as "*******" **** ******* on, ********* ** ***** once *** ***. *****, exactly **** ********* *** alert *** ********* ** discern. ***** ***** ** was **** *****, **** as *** ****** **** shown ******* *****, ~***' from *** ******.

Opening Garage Door Classified As Unknown

Other ******* ***** ****** *******

******* ** *** **** tests, ***** ******* **** as *** *** *** cat ** *** ***** below *** *** ******* false ******.

No False Alert On Animals

Crouching/Crawling ****** ********* *********

**** ********* ******** *** not ****** ******** ******** unless * ******** ******/***** setting *** *******.

*******, ***** **** ******* allowed ******** ******** ** be ********, ** **** resulted ** ***** ****** on ******* ** *** scene, ***** ****:

[optimize output image]

Running ******** ******** **********

** *** *****, ****** were *** ****** ******* upright/running ******* *** ***** within ~***'/~*-* ***.

[optimize output image]

Solid ***** ***** ********* ** ****** ****** *******

****** ****** ******* ** false ****** **** ********** ignored ***** ******** ************, which ********* ********* ** just ********** *** "******* presence ** ****" ** ~1 ******, **** ******** in ********* ** ****** false ***** ******* ******* such ** ******* ***** changes *** *******, ***** below.

VMS ***********

********* ******** ********** ****** and ******** ***** ** Exacq, *******, *** *********. Network ***** ******** **** event ***********, **** ** boundoing *** *********, ***** Avigilon ******* ****** **** not ******* ******.

VMS Integration

ExacqVision *********** ********

**** ** **** ****** Perimeter ********, *********** **** Exacq **** ********* ******, but *** ******** ***** or ********. ***, ******** metadata ** ********* ** the ***, ***** **** bounding *****, ***** *****:

Bounding-Box-_-Metadata-Now-Integrated-To-Exacq

Genetec *** ********* ************

******* *** ********* *********** remains ********* ******** ** our ******** ********* ******** test.

** ******* *** *********, cameras *** ******** **** zone *** ******** *** information, ** **** ** a *****/*** ***** ********* in *** *** ***** corner, ***** *** **** an ********'* ********* ****** than ******* ******** ***** in *****-****** *****.

****** *** **** ** used ** ****** ******, shown ****:

***** ************ ******* ********** software ********* ** *** VMS ******, ****** *** Perimeter ******** ******, ***** translate ******** **** *** proper ****** *** **** VMS. **** **** **** is * *********** ***********, only ******** ********, *** is *** ********* *********.

** ******** ** *** bridge, ******* ***** **** have *** ** **** SDK ******** ********* ** their ********* ******* ** order ** ******* ********* Defender ******, * ~$*** MSRP *******.

Camera *************

********* ******** ** ************** **** ** ****' current * *** * series ****** *** ****** M ****** ******. ************, Axis **** **** *** verify *********** ** ***** models ***** ** ******** request *** ** **** we ********* ***** ******* Axis ** *** ******* model ** *** ******.

**** ****** ******* ** not ***** ** ***** as ***** ***** ***** or **** ************, ***** support ****** *** ******* models, *** ****** **** mid ** **** *** models ***** *** ****** to ** **** ** these ************.

*******

********* ******** ******** **** for ~$*** *** *** camera. ****** ******** *** ten ***** *** *********. This ******* ** ****** than ****' *** ***** Suite (~$** *** ***********). Many *********** ******* ***** analytics ** ** ********** charge **** *** ******** of ***** *******.

Separate *********** *** *************

********* ******** ** ********** using * ******** ***********, not ** *** *******' web *********, ******* ** Bosch *** *** ********'* H4/H5 *********. **** *********** does *** **** ** run **** **********, **** events *** ******** **** directly **** *** ******* to ********** ***.

Automatic ***********

********* ******** **** ** automatic *********** ****** ***** analyzes ***** ** * subject ******* ********** *** field ** **** ** interpret *** *****. ***** calibration, ***** *** ********* with ** ***** *** verification, ******* *** ******* at ******* ******, *** marking *** ******* (***** in ****), ***** **** a "*********" ****** ***** indicates *** ******'* ********** in **** ***********. ***** calibration, ***** *** **** feed *** ******** **** used *** *********** **** the ******** ** **** detection ** ***** ******.

** *** *****, ********* calibration ****** ******** *** accurately ***** **** ** was ********* *******, **** subject **** **** ***** at *** ******. ** smaller ****** ******, ****** calibration *** ********, ** the ******** ***** *** properly ********* *** *****.

** ****** *** *********** process ** **** *****:

Versions ****

*** ********* ******** **** used ****** *******:

  • **** ********* ********:*.*.*
  • **** *****-*:*.**.*.*

Comments (18)

Thanks for the report and fantastic job as usual. Thanks.

Regarding bounding boxes integration into the various VMSs, is the video+bounding boxes also able to be passed on intact to a monitoring center PSIM platform? In other words, would an operator at a monitoring center using a PSIM platform like Immix see both the video and the bounding boxes.

Agree: 2
Disagree
Informative
Unhelpful
Funny

Axis responded:

It is possible to include the bounding boxes (as well as the detection zones and alert status if required) in the video stream using the feature called "burnt-in metadata". All clients using video at the configured resolution will be able to see the alarm overlays

Here is an up to date list of the cameras that support burnt-in metadata and how to enable:

Axis Perimeter Defender Burnt-In Metadata Compatibility

Agree
Disagree
Informative
Unhelpful
Funny

Dears,

That's interesting, good job !

One thing that I miss in the usual tests you run guys is range of detection test while having a good filtering of false events, as it is always a problem when you have to choose between good detection AND low false alarms rate. Especially one should re-run detection tests after applying "optimizations" so as to get rid of false detection, to make sure you still can detect properly, right ?

Also, imho a good deal of the market has switched to thermal sensors and that would be great to test solutions in that environment, that supposedly would help filtering light sources. Here it's not that easy as you have to get a thermal camera out of Axis, and that's not a cheap model to put your hand on.

Cheers,

Laurent

Agree
Disagree
Informative
Unhelpful
Funny

Laurent, good feedback. On this:

imho a good deal of the market has switched to thermal sensors and that would be great to test solutions in that environment, that supposedly would help filtering light sources.

For sure, thermal helps compensate for poor algorithms but analytics are significantly improving. That is our point contrasting this to Dahua and Wyze.

Axis suggested the same about thermal. However, our push back is simple. So Axis needs a $2,000+ camera to compensate for its inferior analytics vs a $20 Wyze cam?

Axis flat out needs to get better here. Going thermal is no longer the answer. Getting real deep learning analytics is. With deep learning, it's fairly easy to distinguish between shadows and a human, without having to use thermal.

Agree
Disagree
Informative
Unhelpful
Funny

Dear John,

I totally understand your point, I'm not saying at all that thermal should be chosen to compensate for deficient analytics: keep in mind also that thermal introduces issues that are not present on classic cameras, such as fast heating grass and wind on variable weather that can cause false alarms also, and invisible individuals on walls in the summer for example.

Thermal cameras have to be used for what they were designed for: allowing proper visibility when there is little or no light, or adverse conditions such as fog or heavy rain, as in these condition there is no traditional camera able to provide a usable image.

I'm particularly addressing trespassing issues on industrial sites that for many reasons (environmentally, economically, need to avoid attracting attention) do not have/want lighting or that need to guarantee proper contrast during bad weather. These situations represents at least half of my business on intrusion even though I can provide the same level of performance on a good classic image.

Hope that helps,

Laurent

Agree
Disagree
Informative
Unhelpful
Funny

Laurent, very good feedback and makes perfect sense for your application. Moreover, we believe Axis that their analytics will perform significantly better on thermal. So, if for image quality reasons, you are using thermal regardless of analytics, it is a good choice.

Agree
Disagree
Informative
Unhelpful
Funny

What's the non-Chinese solution for camera side analytics?

Agree
Disagree
Informative
Unhelpful
Funny

Sergio, in our testing, Avigilon and Bosch are the two that stand out. I expect the list to expand, though, Hanwha announced AI cameras, I even think Axis will have true AI cameras next year, the tech is maturing overall.

Agree
Disagree
Informative: 1
Unhelpful
Funny

thanks for the response. I really appreciate it. My concern is that I have some new Q-series doing a superb job (not doing analytics), but I hope they continue it on the older Q lines the use of Perimeter defender, and not discontinue Perimeter Defender on them.

Agree
Disagree
Informative
Unhelpful
Funny

That's my global concern about edge analytics, the manufacturer may decide any moment to let you out of an upgrade because your camera is "too old". Keep in mind that the analytics price is marginal in comparison to the camera price, so the manufacturer's interest is quite clear to incite you purchasing the latest model (sometimes giving the analytics for free on the new model).

Just to state things clearly as my opinion might be biased, I'm a provider of non edge analytics, even though I've been approached several times by some of these cameras providers, I totally understand the apparent simplicity of edge detection, but it comes at a price as you see.

Laurent

Agree: 1
Disagree
Informative
Unhelpful
Funny

Great report, as usual. Seems like some of those outdoor issues can be reduced by having the bounding boxes avoid things like bushes and trees whenever possible. That, with the fixes you mentioned, make this sound like an incredibly good value.

Agree
Disagree
Informative
Unhelpful
Funny

That, with the fixes you mentioned, make this sound like an incredibly good value.

Incredibly good value?”

i’m not sure we’re reading the same report.

regarding value:

Perimeter Defender licenses sell for ~$260 USD per camera. Single licenses and ten packs are available. This pricing is higher than Axis' own Guard Suite (~$60 per application). Many competitors include their analytics at no additional charge with the purchase of their cameras.

we examine...where and how it is significantly worse than Axis much less expensive competitors.

regarding fixes:

Additionally, 1-2 times per night, an "unknown" object type was detected... No settings changes removed these issues totally.

Perimeter Defender frequently alerted on a flag swaying in the background of one test scene, highlighted below...Adjusting sensitivity in this scene reduced alerts but did not completely eliminate them.

Another source of false alerts were mostly car headlights in the distance...Settings changes did not mitigate these alerts.

Agree
Disagree
Informative
Unhelpful
Funny

I'm assuming you're a competitor since you're commenting undisclosed...

My comment was for my situation, which I can't imagine you're familiar with.

Agree
Disagree
Informative
Unhelpful
Funny

My comment was for my situation, which I can't imagine you're familiar with.

ok, you got me, i admit i’m not overly familiar with your “situation”. honestly, i didn’t even know you had a “situation” til you mentioned it.

fwiw, i’ve actually had a “situation” or two myself, so i know what you’re going thru, (in a general, non-specific sense only). chin up!

I'm assuming you're a competitor since you're commenting undisclosed...

nah, not a competitor, just a curmudgeon :)

Agree
Disagree
Informative
Unhelpful: 1
Funny: 2

@U#2, besides Avigilon and Bosch, are there any other Edge perimeter VCA solutions you recommend looking at? For example other brands entirely, or other edge VCA apps on ACAP, instead of APD.. Genuinely curious, you apparently have a high bar on this and it is still slim pickings. A lot of options out there that are poised for greatness, as it were, but still complications by pricing, architecture, integration, feature set, etc.

Agree
Disagree
Informative: 1
Unhelpful
Funny

Genuinely curious, you apparently have a high bar on this and it is still slim pickings.

no, i haven’t had enough experience with edge VCA to feel comfortable raising a bar. in fact, most of my knowledge comes from IPVM testing. i am going to personally test the Dahua, because it’s cheap, and sounds decent enough. after that i’ll probably look into setting up at least a low bar for edge VCA.

btw, i didn’t object to Michaels post based on any VCA bar; i only objected to the “Incredible value” characterization.

and although he may well be able to justify why it’s such an outstanding value to him, the way the post is actually written, it provides no such context, and thus gives a false impression of the report’s conclusion, imho.

read the comment again and imagine you had only skimmed the article beforehand:

Great report, as usual. Seems like some of those outdoor issues can be reduced by having the bounding boxes avoid things like bushes and trees whenever possible. That, with the fixes you mentioned, make this sound like an incredibly good value.

Agree
Disagree
Informative
Unhelpful
Funny

Bounding boxes can be integrated with any VMS via "burnt-in metadata in the RTSP stream" ...

From Axis datasheet: To enable the feature, open AXIS Perimeter Defender, click on Output in the menu, and press on the keyboard Ctrl+Alt+E. Each product can behave differently in terms of performance, depending on the selected resolution in combination with the burnt-in metadata overlay feature.

Agree
Disagree
Informative
Unhelpful
Funny

Forgive the ignorance, but just starting to look at AI to support our National SOC:

Is the licence per camera transferable or is it a license bound to the camera hardware itself? Concern is that if a camera dies, do you have to buy the hardware and another license?

What are the pros/cons of AI on the camera, verses if you have a Milestone/Genetec system, going VMS server based AI (Kiwi etc)? Understand benefits of pushing processing to the edge, but are there other strengths or weaknesses with either approach?

Would also assume that going server based AI means you could apply it on cameras that can’t naturally support at the edge, link multiple cameras together in the AI, and retain settings/config if the camera needs to be replaced.

NOTICE: This comment has been moved to its own discussion: Questions - Starting To Look At AI To Support Our National SOC

Agree
Disagree
Informative
Unhelpful
Funny
Read this IPVM report for free.

This article is part of IPVM's 7,276 reports and 969 tests and is only available to subscribers. To get a one-time preview of our work, enter your work email to access the full article.

Already a subscriber? Login here | Join now
Loading Related Reports