Axis Video Analytics Are Weak

By: Ethan Ace, Published on Oct 05, 2015

For more than a decade, video analytics has frustrated and disappointed users.

Now, Axis has released their own "series of robust video analytics applications" that they call Guard Suite [link no longer available]. Unfortunately, IPVM's testing shows that Axis' video analytics are weak. We tested both the Q1615 and the P1428E with the motion and fence applications ($149 MSRP each but free 30-day trial applications are available here [link no longer available]).

*** **** **** * decade, ***** ********* *** frustrated *** ************ *****.

***, **** *** ******** ***** own "****** ** ****** ***** analytics ************" **** **** call ***** ***** [**** no ****** *********]. *************, IPVM's ******* ***** **** Axis' ***** ********* *** weak. ** ****** **** the ***** *** *** P1428E **** *** ****** and ***** ************ ($*** MSRP **** *** **** 30-day ***** ************ *** available **** [**** ** longer *********]).

** ** ************* *** surprising ***** **** ******* quality ***** *** ****** R&D ******.

[***************]

*********

******** ** ****' ******** effort,*** *, **** ***** ***** sources **** **** ******* or **********. ** *** tests, ***** ***** *** not ******** ******* ******, birds, ** **** ** *******, occasional ******** ** *** 3. 

** **** ***, ******** size, **********-****, **** *** weather, ********** ******** ****** slowly **** ***** ***** should work ********** *** ********.

*********

**** ***** ***** ** dumb.

***** ******* *** ** the ******, *** **** Guard ***** ********* ** reliably ****** ***-***** **********. **** modest **** ***** **********, false ********* *** ***** negatives ****** *********** **** Guard ***** ********** ** recognize **** ** * real ***** ** ******, wires, *******, ***. ***** problems *** *** **** severe ************ (******)******** ******* (******).

*********** *** **** ** intelligence is * ********** ***** process **** ***** ******** controls ** ******* **** might **** ********** *** this, ****** ********, ***** largely ******* **** *************, and ***** ******* **** provides ************* ***** *** calibration ** ******* ***********.

Market ****** / ***************

** *** *** **** Guard ***** ****** *** use ** ** ****** indoor ****** ** ****** in ** ***** *** week ** ******* ** the ***** ******** *** plan ** *** **. Despite ****' ****** ******, Guard ***** ** *** 'robust'.

** ****** *********** ***** this ** ** ********** by *** **** *********** and *** ********* ** overcome **. ** ** not *** **** ***** widely ****, ******* **** to ******** ** ***** on *** ***** ******** apps.

Performance ******

** **** ******* ** ***** down *** *** ****** impacting **** ***** ***** performance, ********* ***** ***** issues, ****** ***********, *** more, ** **** *****:


***** *********** 

****' *********, **** *** 3 ****** **, ****** from ******** ***** *********** from *****, ********, *******, and ***** **********. ** the ***** *****, *** brush ** *** ***** side ** *** *** is **** ** ******* by *** ******, ******* calibrating *** **** *** perspective. **** ******** ** far ** ** ************ cause ***** *********** ** the ******** **** ** the *****. 

***** *********, **** ** Avigilon *** ******* *** not ********** ***** ******** activations.

****** *********** 

** *** *******, ******** crossing * ******** ***** Axis ***** ***** **** often *** ********. ** the ***** *****, *** camera ****** **** *** detect *** ******* ** an ****** ***** ** is **** *** ****. Avigilon's *** ****** *** 100% ******** ***** **** same *****.

** ******* ******, **** was **** *****, **** Axis ******* ******* **** than **% ** *** time, ***** ******** *** successful >**%. ** *** example *****, ******* *** subject ***** **** ** an ****** ***** ** running, *** ****** ***** to ****** *** **** he ****** ******.

**** **** ** ***** FOVs, **** **** **** to ****** *** ******* as * ***** ******, tripwire *********** ****** **** not ** ****** ** *** examples *****. *******, **** still *** ******* ********* running ********, ******* **** about **% ** *** time.

** *****, ****** ********** at ****** ****** **** frequent, ** ****, **** with *** ******* ** well ***** ****** ******** areas ** *** *****, reviewed ** **** *****:

********* *****

***** * ~**° ********** field ** ****, ***** Suite ********* **** ******** at * ******* ***** of ***** ***' ***** *** 1080p ***** ****** *** day. **** ** ***** **% less ******** **** *** Avigilon *** ******, ***** covered ~***' ****.

**** **** **** ** the ******* ***** ******** detection *** ******** (>**%). Beyond ****, ******** ** normal ******* ****** *** **** frequently ******, ****** ********* spotty.

Setup *** ***********

**** ***** ***** ********** perspective *********** *** ***** ** their ******** ************, **** as *** *, ******** to ****** ********* ******** and ****** **** ** measuring ******* ** *** FOV.

**** ******** ******* ******** on **** *******, **** instructing ***** ** *** measurements "** ********* ********* from *** ******."

******** ** *****

******** ** *****, ***** also ******** *********** (******** does ***), ***** *** several *** *********** ** this *******:

  • ** ******* ****/***** ***** window:****' *********** ************ *** performed ** * ***** window, **** ** ****** to ********* **** ** enlarge *** ****** ** more ********** ******* ******** as ** ***** (**** A ** *** ***** below). **** *** **** it ********* ** ****** measurements ** *** ***** top *** ****** ** an ******, ******** ********.
  • ***** ********* ** ********:**** **** * **** to ******* *******, **** a ***** *** ** either *** ***** ********* size, ***** *** ******* the **** ** *** object ***** ********. ***** uses * ******* * shape (**** * *****) to ******* *** ****** scroll ***** ********** ** size, ****** ** ****** to ****** ** *** top *** ****** ** an ******.
  • ** ******** ***********:**** ******** **** ****** of *******, *** ******** calibration ***** ** ***** (and *****'* ****** *** releases). (**** * ** image *****)
  • ** ******* ****:***** ******** * ******* marker ***** ** **** to ****** ****** *********** of *** ******, ** well ** ******* ******* in *** ***. **** does ***.

************, **** ******** ** visual ************ ***** ** check *** ******** ** measurements, **** ** ****** or ********, ** ****** measurements *** ********. ***** and ***** **** ******* this *******.

*******, **** ******** ** maximum **** ******** ** vehicle **. ***** **************. All ******* *** ********, regardless ** ****/*****.

*********** *************

** *** ***** ***** we *********** *** **** calibration ******* *** *** issues ***** *****:

VMS ***********

*********, **** ***** ***** is ******* ** ****** integration *** *** ****** or ***** ***** ****. It ** ****** **** integrations **** ** ******** in *** **** *** months, **** **** *******:

"**** ***** ***** **** be ********* ** **** ****** ******* *** ******* *** ********* Genetec *** ******* ******* through **** *******’ ***** stream, ***** ********* ******* will ******* ****** ***********. Support ** ******* ****., Exacq ************ *** ***** is ******* ** ** available ** *** *** of ****."

** ** ***, ***** may ******* ***** ****** by ********** **** ** the ***** *** ****** in *** ******, ** seen ** *** *** of *** ****** *****. Users *** *** * motion **** ** ********* motion **** ** *** area ***** **** **** appears, *********** ******** ** the ******** ****:

***** *** **** **** TCP ****** ************* **** the ****** ** ***** which ****** ****. **** is ************ ** *** video ***** ***** ***** the *********** ** ***** Suite **** *****, **** event **** ********** *** ********** overlaid **** **** *****.

Comments (9)

Use digital barriers on a Axis - UK iLids approved. IMHO it just works :)

The analytics developed by Axis was informed to target a low-end segment of VCA, although a step up from their VMD and simple cross-line it was not to become a high-end VCA (yet at least). The argument was not to compromise their open platform by competing with the 3rd party higher-end VCA solutions that can run on their cameras.

What I'm a little bit surprised about is the price, I had anticipated that they were going to offer this for a much lower cost, more in line with their previous Cross Line Detection.

I think it is dangerous to offer analytics for a 'low-end segment of VCA'. It is very hard to determine how poorly the analytics will work in different scenarios, which makes it likely that they are deployed in conditions they cannot handle well, which makes customers frustrated and causes integrators to lose lots of money troubleshooting / trying to optimize.

Axis lead marketing claims for Guard Suite is "Robust and reliable indoor and outdoor detection" that "Protect and detect around the clock". If they really wanted it to be 'low-end', then they should have avoided such claims. As is, they have set expectations that they cannot meet.

Yes, unfortunately marketing talk will almost always over-sell products.

As for targeting the low-end, that's what they did with the Cross Line (in my opinion). It seemed to perform rather well compared to its competitors in the tests you performed. But I guess their marketing language wasn't as bold then.

http://ipvm.com/report/analytics-rain-axis-bosch-sony

http://ipvm.com/report/camera-analytics-tested

I would personally be extremely reluctant to use the Axis Cross Line in any outdoor environment since the consequences of missed alarms and a high false alarm rate would be devastating to our costs and brand value.

It will however be interesting to see how well the new Axis analytics perform. Hopefully it is a step up from the old Cross Line and you seemed to think that it at least reduced the false-alarm rate somewhat compared to VMD 3.0. Perhaps it can perform good enough in ideal, sterile environments (as you mentioned in the test).

If you have time for it, it would be interesting to see how the latest Bosch-analytics compares with the new Axis as the latest Bosch should be much better than the old.

Similarly to the Bosch analytics (as well as just about any other VCA on the market that is actively developed), I think we'll see improved performance over time with the new Axis solution, the question is when (or if) it will be a good enough to use in non-ideal scenarios as well and whether it will be enough to be competitive towards the other camera manufacturers. Only time will tell...

Axis Crossline definitely had more false alert issues than you see in Guard Suite. I recall headlights, shadows, and blowing leaves being bigger issues than even what you see here. Guard Suite is at least far less susceptible to the headlights/shadows.

That being said, Crossline also didn't seem to have the same rate of missed objects that Guard Suite does. That's likely because it detected everything as an object, like debris/headlights/shadows, so it was less choosy and less likely to drop an object once detected.

I would love to see an on-board analytic shoot between Pelco, Axis, Samsung, Bosch etc..

The placement of bars for calibration in you tests (see the "Axis Perspective Calibration" Figure) is certainly wrong because bars should be normal to the ground plane. Otherwise the data essential for calibration will be inaccurate. This is best seen with bars 1 and 2 drawn over the two cars. The bars should go along the front door vertical edge (which is not at all "vertical" in the picture).

I don't know if Axis has made a note of this fact in its manual but no calibration is possible with bars drawn at random.

The bars are not drawn at random. The bars you see on the cars were drawn along the door line from the roof to the ground as you point out. However, Axis snaps the lines to vertical after they're drawn, but they remain the same length. I measured the cars those lines were drawn on, as well, to be accurate.

The cars themselves don't appear level due to the angle of view. The FOV overall is level, though, as you can see the poles in the background are vertical.

Also, this scene was later recalibrated when I moved cameras back to this point for further testing.

For reference, Axis gives little guidance as to what is right/wrong in this calibration, nor any best practice in their manual.

Thanks for your clarification. Really strange that Axis "snaps" the bars to vertical... No way to double check the bars had been drawn correctly.

Login to read this IPVM report.
Why do I need to log in?
IPVM conducts unique testing and research funded by member's payments enabling us to offer the most independent, accurate and in-depth information.

Related Reports on Video Analytics

Bosch Budget 3000i Cameras Tested on Dec 05, 2019
Bosch has long had a hole in its lineup for, as it describes, "competitively-priced cameras". Now, Bosch has released its 3000i series cameras...
Wyze AI Analytics Tested - Beats Axis and Hikvision, Now Improved, Now Being Terminated on Nov 25, 2019
When we originally tested Wyze's free person detection deep learning analytics in July 2019, they performed well, with few false alarms, but a long...
Top 2020 Trend - AI Analytics on Nov 22, 2019
170+ Integrators answered: What do you think will be the top industry trend in 2020? Why? For the 4th year in a row, AI/video analytics was...
Rhombus Cameras, VMS and Analytics Tested on Nov 06, 2019
Rhombus boasts they have created "the new standard in Enterprise, cloud-managed video security" and told IPVM in January 2019 they offer twice the...
100+ Companies Profile Directory on Nov 06, 2019
While IPVM covers the largest companies in the industry regularly (like Axis, Dahua, Hikvision, etc.), IPVM strives to do a profile post on each...
nFlux AI Startup Profile on Oct 28, 2019
nFlux, an Amazon, Google, and Microsoft funded AI startup told IPVM their mission is to build the most intelligent video analytics platform in the...
RealNetworks SAFR Facial Recognition Profile on Sep 25, 2019
RealNetworks entered the surveillance market by giving away their analytics to schools for free, and is now targeting large commercial users with...
Hikvision Acusense Analytics Tested on Sep 23, 2019
Hikvision touts "The Magic Behind It All" in their new Acusense line are 'deep learning algorithms' inside these cameras and recorders. But how...
Directory of 70 Video Surveillance Startups on Sep 18, 2019
This directory provides a list of video surveillance startups to help you see and research what companies are new or not yet broadly known. 2019...
Vivotek "Neural Network-Powered Detection Engine" Analytics Tested on Sep 17, 2019
Vivotek has released "a neural network-powered detection engine", named Smart Motion Detection, claiming that "swaying vegetation, vehicles passing...

Most Recent Industry Reports

Disruptor Wyze Releases Undisruptive Smartlock on Dec 06, 2019
While Wyze has disrupted the consumer IP camera market with ~$20 cameras, its entrance into smart locks is entirely undisruptive. We have...
Bosch Budget 3000i Cameras Tested on Dec 05, 2019
Bosch has long had a hole in its lineup for, as it describes, "competitively-priced cameras". Now, Bosch has released its 3000i series cameras...
Anixter Resisting Takeover From Competitor on Dec 05, 2019
Mega distributor Anixter is going to be acquired but by whom? Initially, Anixter planned to go private, being bought by a private equity firm....
Security Sales Course 2020 - Last Chance Save $50 on Dec 05, 2019
This sales course is customized for the current needs and challenges specific to professionals selling video surveillance and access control...
Ireland National Children's Hospital Chooses Hikvision End-to-End With Facial Recognition on Dec 05, 2019
The world's most expensive hospital project ever, the New Children's Hospital in Ireland, has chosen an all-Hikvision surveillance system including...
AVTech ~$70 IP Cameras Tested Vs Dahua and Hikvision on Dec 04, 2019
Taiwanese manufacturer Avtech is taking direct aim at low cost leaders Dahua and Hikvision with ~$70 starlight and white light illuminator...
Ultinous European Analytics Startup Company Profile on Dec 04, 2019
European analytics-startup Ultinous pitches customers to "Have your own video analysis service!" We spoke to Ultinous to better understand their...
Access Startup Multi-Mount Aims To Streamline Reader Installs on Dec 03, 2019
Startup Multi-Mount claims it makes installing access readers 'Fast', 'Secure,' and fit 'any size frame.' The company states its bracket 'fits most...
Resideo CEO To Step Down on Dec 03, 2019
Resideo's CEO, Mike Nefkins, is stepping down, just 18 months after being brought in to lead the now plagued spin-out. Inside this note, we...
Arcules CEO Retracts False GDPR Claim + Dahua and Milestone Claims Examined on Dec 03, 2019
Arcules CEO has retracted a false claim about his organization being a "fully compliant GDPR company" after IPVM reporting (Arcules CEO Threatens...