Axis Video Analytics Are Weak

Author: Ethan Ace, Published on Oct 05, 2015

For more than a decade, video analytics has frustrated and disappointed users.

Now, Axis has released their own "series of robust video analytics applications" that they call Guard Suite. Unfortunately, IPVM's testing shows that Axis' video analytics are weak. We tested both the Q1615 and the P1428E with the motion and fence applications ($149 MSRP each but free 30-day trial applications are available here).

It is disappointing and surprising given Axis overall quality level and strong R&D effort.

*** **** **** * ******, ***** ********* *** ********** *** disappointed *****.

***, **** *** ******** ***** *** "****** ** ****** ***** ********* ************" **** **** ********* *****. *************, ****'* ******* ***** **** ****' ***** ********* *** weak. ** ****** **** *** ***** *** *** ****** **** the ****** *** ***** ************ ($*** **** **** *** **** 30-day ***** ************ ************ ****).

** ** ************* *** ********** ***** **** ******* ******* ***** and ****** *&* ******.

[***************]

*********

******** ** ****' ******** ******,*** *, **** ***** ***** ******* **** **** ******* ** **********. In *** *****, ***** ***** *** *** ******** ******* ******, birds, ** **** ** *******, ********** ******** ** *** *. 

** **** ***, ******** ****, **********-****, **** *** *******, ********** subjects ****** ****** **** ***** ***** ****** **** ********** *** ********.

*********

**** ***** ***** ** ****.

***** ******* *** ** *** ******, *** **** ***** ***** struggles ** ******** ****** ***-***** **********. **** ****** **** ***** **********, false ********* *** ***** ********* ****** *********** **** ***** ***** struggling ** ********* **** ** * **** ***** ** ******, wires, *******, ***. ***** ******** *** *** **** ****** ************ (******)******** ******* (******).

*********** *** **** ** ************ ** * ********** ***** ******* **** lacks ******** ******** ** ******* **** ***** **** ********** *** this, ****** ********, ***** ******* ******* **** *************, *** ***** ioimage **** ******** ************* ***** *** *********** ** ******* ***********.

Market ****** / ***************

** *** *** **** ***** ***** ****** *** *** ** in ****** ****** ****** ** ****** ** ** ***** *** week ** ******* ** *** ***** ******** *** **** ** use **. ******* ****' ****** ******, ***** ***** ** *** 'robust'.

** ****** *********** ***** **** ** ** ********** ** *** poor *********** *** *** ********* ** ******** **. ** ** not *** **** ***** ****** ****, ******* **** ** ******** to ***** ** *** ***** ******** ****.

Performance ******

** **** ******* ** ***** **** *** *** ****** ********* **** Guard ***** ***********, ********* ***** ***** ******, ****** ***********, *** more, ** **** *****:


***** *********** 

****' *********, **** *** * ****** **, ****** **** ******** false *********** **** *****, ********, *******, *** ***** **********. ** the ***** *****, *** ***** ** *** ***** **** ** the *** ** **** ** ******* ** *** ******, ******* calibrating *** **** *** ***********. **** ******** ** *** ** to ************ ***** ***** *********** ** *** ******** **** ** the *****. 

***** *********, **** ** ******** *** ******* *** *** ********** these ******** ***********.

****** *********** 

** *** *******, ******** ******** * ******** ***** **** ***** Guard **** ***** *** ********. ** *** ***** *****, *** camera ****** **** *** ****** *** ******* ** ** ****** until ** ** **** *** ****. ********'* *** ****** *** 100% ******** ***** **** **** *****.

** ******* ******, **** *** **** *****, **** **** ******* runners **** **** **% ** *** ****, ***** ******** *** successful >**%. ** *** ******* *****, ******* *** ******* ***** seen ** ** ****** ***** ** *******, *** ****** ***** to ****** *** **** ** ****** ******.

**** **** ** ***** ****, **** **** **** ** ****** the ******* ** * ***** ******, ******** *********** ****** **** not ** ****** ** *** ******** *****. *******, **** ***** *** trouble ********* ******* ********, ******* **** ***** **% ** *** time.

** *****, ****** ********** ** ****** ****** **** ********, ** well, **** **** *** ******* ** **** ***** ****** ******** areas ** *** *****, ******** ** **** *****:

********* *****

***** * ~**° ********** ***** ** ****, ***** ***** ********* were ******** ** * ******* ***** ** ***** ***' ***** *** 1080p ***** ****** *** ***. **** ** ***** **% **** ******** than *** ******** *** ******, ***** ******* ~***' ****.

**** **** **** ** *** ******* ***** ******** ********* *** possible (>**%). ****** ****, ******** ** ****** ******* ****** *** **** frequently ******, ****** ********* ******.

Setup *** ***********

**** ***** ***** ********** *********** *********** *** ***** ** ***** ******** applications, **** ** *** *, ******** ** ****** ********* ******** and ****** **** ** ********* ******* ** *** ***.

**** ******** ******* ******** ** **** *******, **** *********** ***** to *** ************ "** ********* ********* **** *** ******."

******** ** *****

******** ** *****, ***** **** ******** *********** (******** **** ***), there *** ******* *** *********** ** **** *******:

  • ** ******* ****/***** ***** ******:****' *********** ************ *** ********* ** * ***** ******, **** no ****** ** ********* **** ** ******* *** ****** ** more ********** ******* ******** ** ** ***** (**** * ** the ***** *****). **** *** **** ** ********* ** ****** measurements ** *** ***** *** *** ****** ** ** ******, reducing ********.
  • ***** ********* ** ********:**** **** * **** ** ******* *******, **** * ***** dot ** ****** *** ***** ********* ****, ***** *** ******* the **** ** *** ****** ***** ********. ***** **** * capital * ***** (**** * *****) ** ******* *** ****** scroll ***** ********** ** ****, ****** ** ****** ** ****** to *** *** *** ****** ** ** ******.
  • ** ******** ***********:**** ******** **** ****** ** *******, *** ******** *********** ***** in ***** (*** *****'* ****** *** ********). (**** * ** image *****)
  • ** ******* ****:***** ******** * ******* ****** ***** ** **** ** ****** adjust *********** ** *** ******, ** **** ** ******* ******* in *** ***. **** **** ***.

************, **** ******** ** ****** ************ ***** ** ***** *** accuracy ** ************, **** ** ****** ** ********, ** ****** measurements *** ********. ***** *** ***** **** ******* **** *******.

*******, **** ******** ** ******* **** ******** ** ******* **. human **************. *** ******* *** ********, ********** ** ****/*****.

*********** *************

** *** ***** ***** ** *********** *** **** *********** ******* and *** ****** ***** *****:

VMS ***********

*********, **** ***** ***** ** ******* ** ****** *********** *** TCP ****** ** ***** ***** ****. ** ** ****** **** integrations **** ** ******** ** *** **** *** ******, **** Axis *******:

"**** ***** ***** **** ** ********* ** **** ****** ******* *** ******* *** ********* ******* *** ******* ******* ******* **** cameras’ ***** ******, ***** ********* ******* **** ******* ****** ***********. Support ** ******* ****., ***** ************ *** ***** ** ******* to ** ********* ** *** *** ** ****."

** ** ***, ***** *** ******* ***** ****** ** ********** text ** *** ***** *** ****** ** *** ******, ** seen ** *** *** ** *** ****** *****. ***** *** set * ****** **** ** ********* ****** **** ** *** area ***** **** **** *******, *********** ******** ** *** ******** rule:

***** *** **** **** *** ****** ************* **** *** ****** to ***** ***** ****** ****. **** ** ************ ** *** video ***** ***** ***** *** *********** ** ***** ***** **** Exacq, **** ***** **** ********** *** ********** ******** **** **** *****.

Comments (9)

Use digital barriers on a Axis - UK iLids approved. IMHO it just works :)

The analytics developed by Axis was informed to target a low-end segment of VCA, although a step up from their VMD and simple cross-line it was not to become a high-end VCA (yet at least). The argument was not to compromise their open platform by competing with the 3rd party higher-end VCA solutions that can run on their cameras.

What I'm a little bit surprised about is the price, I had anticipated that they were going to offer this for a much lower cost, more in line with their previous Cross Line Detection.

I think it is dangerous to offer analytics for a 'low-end segment of VCA'. It is very hard to determine how poorly the analytics will work in different scenarios, which makes it likely that they are deployed in conditions they cannot handle well, which makes customers frustrated and causes integrators to lose lots of money troubleshooting / trying to optimize.

Axis lead marketing claims for Guard Suite is "Robust and reliable indoor and outdoor detection" that "Protect and detect around the clock". If they really wanted it to be 'low-end', then they should have avoided such claims. As is, they have set expectations that they cannot meet.

Yes, unfortunately marketing talk will almost always over-sell products.

As for targeting the low-end, that's what they did with the Cross Line (in my opinion). It seemed to perform rather well compared to its competitors in the tests you performed. But I guess their marketing language wasn't as bold then.

http://ipvm.com/report/analytics-rain-axis-bosch-sony

http://ipvm.com/report/camera-analytics-tested

I would personally be extremely reluctant to use the Axis Cross Line in any outdoor environment since the consequences of missed alarms and a high false alarm rate would be devastating to our costs and brand value.

It will however be interesting to see how well the new Axis analytics perform. Hopefully it is a step up from the old Cross Line and you seemed to think that it at least reduced the false-alarm rate somewhat compared to VMD 3.0. Perhaps it can perform good enough in ideal, sterile environments (as you mentioned in the test).

If you have time for it, it would be interesting to see how the latest Bosch-analytics compares with the new Axis as the latest Bosch should be much better than the old.

Similarly to the Bosch analytics (as well as just about any other VCA on the market that is actively developed), I think we'll see improved performance over time with the new Axis solution, the question is when (or if) it will be a good enough to use in non-ideal scenarios as well and whether it will be enough to be competitive towards the other camera manufacturers. Only time will tell...

Axis Crossline definitely had more false alert issues than you see in Guard Suite. I recall headlights, shadows, and blowing leaves being bigger issues than even what you see here. Guard Suite is at least far less susceptible to the headlights/shadows.

That being said, Crossline also didn't seem to have the same rate of missed objects that Guard Suite does. That's likely because it detected everything as an object, like debris/headlights/shadows, so it was less choosy and less likely to drop an object once detected.

I would love to see an on-board analytic shoot between Pelco, Axis, Samsung, Bosch etc..

The placement of bars for calibration in you tests (see the "Axis Perspective Calibration" Figure) is certainly wrong because bars should be normal to the ground plane. Otherwise the data essential for calibration will be inaccurate. This is best seen with bars 1 and 2 drawn over the two cars. The bars should go along the front door vertical edge (which is not at all "vertical" in the picture).

I don't know if Axis has made a note of this fact in its manual but no calibration is possible with bars drawn at random.

The bars are not drawn at random. The bars you see on the cars were drawn along the door line from the roof to the ground as you point out. However, Axis snaps the lines to vertical after they're drawn, but they remain the same length. I measured the cars those lines were drawn on, as well, to be accurate.

The cars themselves don't appear level due to the angle of view. The FOV overall is level, though, as you can see the poles in the background are vertical.

Also, this scene was later recalibrated when I moved cameras back to this point for further testing.

For reference, Axis gives little guidance as to what is right/wrong in this calibration, nor any best practice in their manual.

Thanks for your clarification. Really strange that Axis "snaps" the bars to vertical... No way to double check the bars had been drawn correctly.

Login to read this IPVM report.
Why do I need to log in?
IPVM conducts unique testing and research funded by member's payments enabling us to offer the most independent, accurate and in-depth information.

Related Reports on Video Analytics

IFSEC 2018 Final Show Report on Jun 20, 2018
IPVM attended the IFSEC show for the first time this year. The Chinese took over the show, centered on Hikvision, flanked by Dahua, Huawei and a...
ReconaSense - The AI / Access Control / Analytics / IoT / Video Company Profile on Jun 12, 2018
One company's ISC West booth stood out for displaying a light-up tower of buzzwords. The company, ReconaSense, pledged to be 'making sense of it...
Bosch IVA Video Analytics And Motion+ VMD Tested on Jun 06, 2018
Bosch's video analytics now ship on nearly every model, from indoor domes to high-end 5MP starlight cameras.  In this test, we evaluate Bosch's...
Hanwha Wisenet X Analytics and VMD Test on May 24, 2018
Continuing our updated testing of camera analytics, we tested Hanwha's Wisenet X analytics for over two weeks in multiple scenes, indoors and out,...
The Hikvision Smart Classroom Behavior Management System on May 22, 2018
Hikvision's rapidly growing offering of analytics, which we most recently examined with Hikvision's ethnic minority analytics, is now going into...
Dahua Intrusion Analytics And VMD Tested (Poorly) on May 21, 2018
Dahua ships basic analytics on practically all their cameras, ranging from low cost to high end. To see how these analytics work in real world...
Stealth / UCIT - Remote Video Monitoring Provider Profile on May 18, 2018
Can 2 remote video monitoring companies, Stealth Monitoring from the US and UCIT from Canada combine to impact the market and compete in a changing...
Nortek Acquires IntelliVision on May 08, 2018
One of the largest intrusion manufacturers, Nortek, has acquired the self-proclaimed 'Leader in AI-Based Video Analytics Software',...
Hikvision's Minority Analytics on May 08, 2018
Hikvision has developed and is deploying minority analytics inside China, automatically identifying and categorizing individuals as ethnic...
Hikvision VMD And Intrusion Analytics Tested on May 04, 2018
VMD worked poorly on Hikvision cameras, new IPVM testing found. However, Hikvision ships analytics on practically every IP camera, with intrusion...

Most Recent Industry Reports

July 2018 IP Networking Course on Jun 22, 2018
  This is the only networking course designed specifically for video surveillance professionals. Register now. Lots of network training exists...
Installation Hardware for Video Surveillance - Indoor Fasteners on Jun 22, 2018
As part of our Installation for Video Surveillance series, in this note, we cover drywall anchors. A key part of installing security hardware is...
Hikvision ColorVu Integrated Visible Light Cameras Examined on Jun 22, 2018
When it comes to low light, infrared light has become the defacto standard in surveillance. But IR is limited to monochrome images, making colors...
'Secure Channel' OSDP Access Control Examined on Jun 21, 2018
Despite claiming to be better than Wiegand, OSDP's initial releases did not address the lack of encryption between reader and controller, leaving...
Most Wanted Improvements In Manufacturer Technical Support (Statistics) on Jun 21, 2018
5 key areas of improvement and 1 clear wanted support feature were voiced by 140+ integrator responses to: What improvement in manufacturer...
GDPR / ICO Complaint Filed Against IFSEC Show Facial Recognition on Jun 20, 2018
IPVM has filed a complaint against IFSEC’s parent company UBM based on our concern that the conference violates core GDPR principles on...
IFSEC 2018 Final Show Report on Jun 20, 2018
IPVM attended the IFSEC show for the first time this year. The Chinese took over the show, centered on Hikvision, flanked by Dahua, Huawei and a...
Mobotix Releases 'Move' Into 21st Century on Jun 20, 2018
For years, Mobotix stood resolutely against, well, every other manufacturer, selling it as a virtue: MOBOTIX equipment is designed with no...
Cybersecurity Startup VDOO Disclosing 10 Manufacturer Vulnerabilities Starting With Axis And Foscam on Jun 20, 2018
Cybersecurity startup VDOO has uncovered significant vulnerabilities in Axis cameras along with many others not yet disclosed. In this report, we...
Axis Guardian - Cloud VMS And Alarm Monitoring - Released on Jun 19, 2018
Axis has struggled to deliver a cloud-based managed service video platform. Video service providers have utilized AVHS for over a decade, and have...

The world's leading video surveillance information source, IPVM provides the best reporting, testing and training for 10,000+ members globally. Dedicated to independent and objective information, we uniquely refuse any and all advertisements, sponsorship and consulting from manufacturers.

About | FAQ | Contact