Axis Video Analytics Are Weak

Author: Ethan Ace, Published on Oct 05, 2015

For more than a decade, video analytics has frustrated and disappointed users.

Now, Axis has released their own "series of robust video analytics applications" that they call Guard Suite. Unfortunately, IPVM's testing shows that Axis' video analytics are weak. We tested both the Q1615 and the P1428E with the motion and fence applications ($149 MSRP each but free 30-day trial applications are available here).

It is disappointing and surprising given Axis overall quality level and strong R&D effort.

*** **** **** * ******, ***** ********* *** ********** *** disappointed *****.

***, **** *** ******** ***** *** "****** ** ****** ***** ********* ************" **** **** ********* *****. *************, ****'* ******* ***** **** ****' ***** ********* *** weak. ** ****** **** *** ***** *** *** ****** **** the ****** *** ***** ************ ($*** **** **** *** **** 30-day ***** ************ ************ ****).

** ** ************* *** ********** ***** **** ******* ******* ***** and ****** *&* ******.

[***************]

*********

******** ** ****' ******** ******,*** *, **** ***** ***** ******* **** **** ******* ** **********. In *** *****, ***** ***** *** *** ******** ******* ******, birds, ** **** ** *******, ********** ******** ** *** *. 

** **** ***, ******** ****, **********-****, **** *** *******, ********** subjects ****** ****** **** ***** ***** ****** **** ********** *** ********.

*********

**** ***** ***** ** ****.

***** ******* *** ** *** ******, *** **** ***** ***** struggles ** ******** ****** ***-***** **********. **** ****** **** ***** **********, false ********* *** ***** ********* ****** *********** **** ***** ***** struggling ** ********* **** ** * **** ***** ** ******, wires, *******, ***. ***** ******** *** *** **** ****** ************ (******)******** ******* (******).

*********** *** **** ** ************ ** * ********** ***** ******* **** lacks ******** ******** ** ******* **** ***** **** ********** *** this, ****** ********, ***** ******* ******* **** *************, *** ***** ioimage **** ******** ************* ***** *** *********** ** ******* ***********.

Market ****** / ***************

** *** *** **** ***** ***** ****** *** *** ** in ****** ****** ****** ** ****** ** ** ***** *** week ** ******* ** *** ***** ******** *** **** ** use **. ******* ****' ****** ******, ***** ***** ** *** 'robust'.

** ****** *********** ***** **** ** ** ********** ** *** poor *********** *** *** ********* ** ******** **. ** ** not *** **** ***** ****** ****, ******* **** ** ******** to ***** ** *** ***** ******** ****.

Performance ******

** **** ******* ** ***** **** *** *** ****** ********* **** Guard ***** ***********, ********* ***** ***** ******, ****** ***********, *** more, ** **** *****:


***** *********** 

****' *********, **** *** * ****** **, ****** **** ******** false *********** **** *****, ********, *******, *** ***** **********. ** the ***** *****, *** ***** ** *** ***** **** ** the *** ** **** ** ******* ** *** ******, ******* calibrating *** **** *** ***********. **** ******** ** *** ** to ************ ***** ***** *********** ** *** ******** **** ** the *****. 

***** *********, **** ** ******** *** ******* *** *** ********** these ******** ***********.

****** *********** 

** *** *******, ******** ******** * ******** ***** **** ***** Guard **** ***** *** ********. ** *** ***** *****, *** camera ****** **** *** ****** *** ******* ** ** ****** until ** ** **** *** ****. ********'* *** ****** *** 100% ******** ***** **** **** *****.

** ******* ******, **** *** **** *****, **** **** ******* runners **** **** **% ** *** ****, ***** ******** *** successful >**%. ** *** ******* *****, ******* *** ******* ***** seen ** ** ****** ***** ** *******, *** ****** ***** to ****** *** **** ** ****** ******.

**** **** ** ***** ****, **** **** **** ** ****** the ******* ** * ***** ******, ******** *********** ****** **** not ** ****** ** *** ******** *****. *******, **** ***** *** trouble ********* ******* ********, ******* **** ***** **% ** *** time.

** *****, ****** ********** ** ****** ****** **** ********, ** well, **** **** *** ******* ** **** ***** ****** ******** areas ** *** *****, ******** ** **** *****:

********* *****

***** * ~**° ********** ***** ** ****, ***** ***** ********* were ******** ** * ******* ***** ** ***** ***' ***** *** 1080p ***** ****** *** ***. **** ** ***** **% **** ******** than *** ******** *** ******, ***** ******* ~***' ****.

**** **** **** ** *** ******* ***** ******** ********* *** possible (>**%). ****** ****, ******** ** ****** ******* ****** *** **** frequently ******, ****** ********* ******.

Setup *** ***********

**** ***** ***** ********** *********** *********** *** ***** ** ***** ******** applications, **** ** *** *, ******** ** ****** ********* ******** and ****** **** ** ********* ******* ** *** ***.

**** ******** ******* ******** ** **** *******, **** *********** ***** to *** ************ "** ********* ********* **** *** ******."

******** ** *****

******** ** *****, ***** **** ******** *********** (******** **** ***), there *** ******* *** *********** ** **** *******:

  • ** ******* ****/***** ***** ******:****' *********** ************ *** ********* ** * ***** ******, **** no ****** ** ********* **** ** ******* *** ****** ** more ********** ******* ******** ** ** ***** (**** * ** the ***** *****). **** *** **** ** ********* ** ****** measurements ** *** ***** *** *** ****** ** ** ******, reducing ********.
  • ***** ********* ** ********:**** **** * **** ** ******* *******, **** * ***** dot ** ****** *** ***** ********* ****, ***** *** ******* the **** ** *** ****** ***** ********. ***** **** * capital * ***** (**** * *****) ** ******* *** ****** scroll ***** ********** ** ****, ****** ** ****** ** ****** to *** *** *** ****** ** ** ******.
  • ** ******** ***********:**** ******** **** ****** ** *******, *** ******** *********** ***** in ***** (*** *****'* ****** *** ********). (**** * ** image *****)
  • ** ******* ****:***** ******** * ******* ****** ***** ** **** ** ****** adjust *********** ** *** ******, ** **** ** ******* ******* in *** ***. **** **** ***.

************, **** ******** ** ****** ************ ***** ** ***** *** accuracy ** ************, **** ** ****** ** ********, ** ****** measurements *** ********. ***** *** ***** **** ******* **** *******.

*******, **** ******** ** ******* **** ******** ** ******* **. human **************. *** ******* *** ********, ********** ** ****/*****.

*********** *************

** *** ***** ***** ** *********** *** **** *********** ******* and *** ****** ***** *****:

VMS ***********

*********, **** ***** ***** ** ******* ** ****** *********** *** TCP ****** ** ***** ***** ****. ** ** ****** **** integrations **** ** ******** ** *** **** *** ******, **** Axis *******:

"**** ***** ***** **** ** ********* ** **** ****** ******* *** ******* *** ********* ******* *** ******* ******* ******* **** cameras’ ***** ******, ***** ********* ******* **** ******* ****** ***********. Support ** ******* ****., ***** ************ *** ***** ** ******* to ** ********* ** *** *** ** ****."

** ** ***, ***** *** ******* ***** ****** ** ********** text ** *** ***** *** ****** ** *** ******, ** seen ** *** *** ** *** ****** *****. ***** *** set * ****** **** ** ********* ****** **** ** *** area ***** **** **** *******, *********** ******** ** *** ******** rule:

***** *** **** **** *** ****** ************* **** *** ****** to ***** ***** ****** ****. **** ** ************ ** *** video ***** ***** ***** *** *********** ** ***** ***** **** Exacq, **** ***** **** ********** *** ********** ******** **** **** *****.

Comments (9)

Use digital barriers on a Axis - UK iLids approved. IMHO it just works :)

The analytics developed by Axis was informed to target a low-end segment of VCA, although a step up from their VMD and simple cross-line it was not to become a high-end VCA (yet at least). The argument was not to compromise their open platform by competing with the 3rd party higher-end VCA solutions that can run on their cameras.

What I'm a little bit surprised about is the price, I had anticipated that they were going to offer this for a much lower cost, more in line with their previous Cross Line Detection.

I think it is dangerous to offer analytics for a 'low-end segment of VCA'. It is very hard to determine how poorly the analytics will work in different scenarios, which makes it likely that they are deployed in conditions they cannot handle well, which makes customers frustrated and causes integrators to lose lots of money troubleshooting / trying to optimize.

Axis lead marketing claims for Guard Suite is "Robust and reliable indoor and outdoor detection" that "Protect and detect around the clock". If they really wanted it to be 'low-end', then they should have avoided such claims. As is, they have set expectations that they cannot meet.

Yes, unfortunately marketing talk will almost always over-sell products.

As for targeting the low-end, that's what they did with the Cross Line (in my opinion). It seemed to perform rather well compared to its competitors in the tests you performed. But I guess their marketing language wasn't as bold then.

http://ipvm.com/report/analytics-rain-axis-bosch-sony

http://ipvm.com/report/camera-analytics-tested

I would personally be extremely reluctant to use the Axis Cross Line in any outdoor environment since the consequences of missed alarms and a high false alarm rate would be devastating to our costs and brand value.

It will however be interesting to see how well the new Axis analytics perform. Hopefully it is a step up from the old Cross Line and you seemed to think that it at least reduced the false-alarm rate somewhat compared to VMD 3.0. Perhaps it can perform good enough in ideal, sterile environments (as you mentioned in the test).

If you have time for it, it would be interesting to see how the latest Bosch-analytics compares with the new Axis as the latest Bosch should be much better than the old.

Similarly to the Bosch analytics (as well as just about any other VCA on the market that is actively developed), I think we'll see improved performance over time with the new Axis solution, the question is when (or if) it will be a good enough to use in non-ideal scenarios as well and whether it will be enough to be competitive towards the other camera manufacturers. Only time will tell...

Axis Crossline definitely had more false alert issues than you see in Guard Suite. I recall headlights, shadows, and blowing leaves being bigger issues than even what you see here. Guard Suite is at least far less susceptible to the headlights/shadows.

That being said, Crossline also didn't seem to have the same rate of missed objects that Guard Suite does. That's likely because it detected everything as an object, like debris/headlights/shadows, so it was less choosy and less likely to drop an object once detected.

I would love to see an on-board analytic shoot between Pelco, Axis, Samsung, Bosch etc..

The placement of bars for calibration in you tests (see the "Axis Perspective Calibration" Figure) is certainly wrong because bars should be normal to the ground plane. Otherwise the data essential for calibration will be inaccurate. This is best seen with bars 1 and 2 drawn over the two cars. The bars should go along the front door vertical edge (which is not at all "vertical" in the picture).

I don't know if Axis has made a note of this fact in its manual but no calibration is possible with bars drawn at random.

The bars are not drawn at random. The bars you see on the cars were drawn along the door line from the roof to the ground as you point out. However, Axis snaps the lines to vertical after they're drawn, but they remain the same length. I measured the cars those lines were drawn on, as well, to be accurate.

The cars themselves don't appear level due to the angle of view. The FOV overall is level, though, as you can see the poles in the background are vertical.

Also, this scene was later recalibrated when I moved cameras back to this point for further testing.

For reference, Axis gives little guidance as to what is right/wrong in this calibration, nor any best practice in their manual.

Thanks for your clarification. Really strange that Axis "snaps" the bars to vertical... No way to double check the bars had been drawn correctly.

Login to read this IPVM report.
Why do I need to log in?
IPVM conducts unique testing and research funded by member's payments enabling us to offer the most independent, accurate and in-depth information.

Related Reports on Video Analytics

Cisco Meraki New Cameras and AI Analytics on Dec 14, 2018
Meraki has released their second generation of video surveillance with 3 new cameras, AI-based video analytics, and 2 cloud-based storage...
Top 2019 Trend - AI Video Analytics on Dec 10, 2018
160+ Integrators answered: What do you think the top industry trend will be in 2019? Why? AI / video analytics was the run-away winner with...
Alarm.com "AI" Video Analytics Tested on Nov 30, 2018
Alarm.com has announced what it calls an "artificial intelligence (AI) architecture and video analytics service", touting that  Alarm.com's...
Vintra "AI-Powered" Video Analytics Startup Profile on Nov 27, 2018
Vintra is a Silicon Valley startup focused on AI-based video analytics. They had booths at IACP and ISC West demonstrating their hosted or...
Genetec Kiwi Intrusion Detector Analytics Tested on Nov 27, 2018
Genetec has built Kiwi Security's Intrusion Detection analytics into Security Center, aiming to simplify deployment compared to separate camera...
Top Manufacturers Gaining and Losing 2018 on Nov 26, 2018
This is the 5th year IPVM has tracked manufacturers gaining and losing: Top Manufacturers Gaining and Losing 2014 Top Manufacturers Gaining and...
Intel Neural Compute Stick 2 / Movidius AI Test on Nov 21, 2018
AI is a major trend in video surveillance with manufacturers paying significant attention to Intel's Movidius Myriad chips. Indeed, Avigilon has...
Arcules Cloud VMS Tested on Nov 19, 2018
Arcules is a big bet, or as they describe themselves a 'bold company', spun out and backed by Milestone and Canon.  But how good is Arcules cloud...
Beware Amazon Go Store Hype (Tested) on Nov 13, 2018
IPVM's trip to and testing of Amazon Go's San Francisco store shows a number of significant operational and economic issues that undermine the...
Genetec Privacy Protector Tested on Nov 12, 2018
Genetec has built Kiwi Security's Privacy Protector into Security Center, an analytic which anonymizes individuals in cameras' fields of view...

Most Recent Industry Reports

ACRE-Acquired Open Options Access Company Profile on Dec 17, 2018
Who is the company ACRE is acquiring? In this note, we examine Open Options line for best customer fit, key features, pricing, and main...
Open Options Acquired By ACRE on Dec 17, 2018
ACRE is doing deals again. A year after they sold Mercury, they are buying another access control company - Open Options. In this note, we...
8MP / 4K Fixed Lens Camera Shootout - Dahua, Hikvision, TVT, Uniview on Dec 17, 2018
8MP / 4K fixed lens models are now common in lower cost lines, with nearly every Chinese brand and their OEMs now offering multiple options. To...
Network Cable Shootout - Belden, Commscope, Hikvision, Honeywell, NavePoint CCA, Windy City Wire on Dec 17, 2018
Every IP camera install needs UTP cabling. But how much of a difference is there between dirt cheap generic cables found online and the bigger,...
Hikvision Government Parent Holds Communist Party Congress on Dec 17, 2018
The Communist Party committee of Hikvision’s government parent, CETHIK, held a Party Congress earlier this month where senior executives, including...
Imperial Capital Security Investor Conference 2018 Review - ADT, Resideo, Alarm.com, Arlo, Eagle Eye, ACRE, More on Dec 14, 2018
Imperial Capital Security Investor Conference is an event matching industry executives with financiers that frequently leads to future funding...
Cisco Meraki New Cameras and AI Analytics on Dec 14, 2018
Meraki has released their second generation of video surveillance with 3 new cameras, AI-based video analytics, and 2 cloud-based storage...
Foolish Strategy: OEMing Facial Recognition on Dec 13, 2018
Almost as 'hot' as face recognition marketing right now is OEMing facial recognition. Last year, they were a who's who of company's with...
DVR Examiner - Video Recovery from Recorder Hard Drives on Dec 13, 2018
Bypassing passwords and long download times on-site, DVR Examiner collects and organizes video evidence directly from a hard drive extracted from...
2019 Access Control Book Released on Dec 12, 2018
This is the best, most comprehensive access control book in the world, based on our unprecedented research and testing has been significantly...

The world's leading video surveillance information source, IPVM provides the best reporting, testing and training for 10,000+ members globally. Dedicated to independent and objective information, we uniquely refuse any and all advertisements, sponsorship and consulting from manufacturers.

About | FAQ | Contact