Axis Video Analytics Are Weak

Author: Ethan Ace, Published on Oct 05, 2015

For more than a decade, video analytics has frustrated and disappointed users.

Now, Axis has released their own "series of robust video analytics applications" that they call Guard Suite. Unfortunately, IPVM's testing shows that Axis' video analytics are weak. We tested both the Q1615 and the P1428E with the motion and fence applications ($149 MSRP each but free 30-day trial applications are available here).

It is disappointing and surprising given Axis overall quality level and strong R&D effort.

*** **** **** * decade, ***** ********* *** frustrated *** ************ *****.

***, **** *** ******** ***** own "****** ** ****** ***** analytics ************" **** **** call***** *****. *************, ****'* ******* shows **** ****' ***** analytics *** ****. ** tested **** *** ***** and *** ****** **** the ****** *** ***** applications ($*** **** **** but **** **-*** ***** applications ************ ****).

** ** ************* *** surprising ***** **** ******* quality ***** *** ****** R&D ******.

[***************]

*********

******** ** ****' ******** effort,*** *, **** ***** ***** sources **** **** ******* or **********. ** *** tests, ***** ***** *** not ******** ******* ******, birds, ** **** ** *******, occasional ******** ** *** 3. 

** **** ***, ******** size, **********-****, **** *** weather, ********** ******** ****** slowly **** ***** ***** should work ********** *** ********.

*********

**** ***** ***** ** dumb.

***** ******* *** ** the ******, *** **** Guard ***** ********* ** reliably ****** ***-***** **********. **** modest **** ***** **********, false ********* *** ***** negatives ****** *********** **** Guard ***** ********** ** recognize **** ** * real ***** ** ******, wires, *******, ***. ***** problems *** *** **** severe ************ (******)******** ******* (******).

*********** *** **** ** intelligence is * ********** ***** process **** ***** ******** controls ** ******* **** might **** ********** *** this, ****** ********, ***** largely ******* **** *************, and ***** ******* **** provides ************* ***** *** calibration ** ******* ***********.

Market ****** / ***************

** *** *** **** Guard ***** ****** *** use ** ** ****** indoor ****** ** ****** in ** ***** *** week ** ******* ** the ***** ******** *** plan ** *** **. Despite ****' ****** ******, Guard ***** ** *** 'robust'.

** ****** *********** ***** this ** ** ********** by *** **** *********** and *** ********* ** overcome **. ** ** not *** **** ***** widely ****, ******* **** to ******** ** ***** on *** ***** ******** apps.

Performance ******

** **** ******* ** ***** down *** *** ****** impacting **** ***** ***** performance, ********* ***** ***** issues, ****** ***********, *** more, ** **** *****:


***** *********** 

****' *********, **** *** 3 ****** **, ****** from ******** ***** *********** from *****, ********, *******, and ***** **********. ** the ***** *****, *** brush ** *** ***** side ** *** *** is **** ** ******* by *** ******, ******* calibrating *** **** *** perspective. **** ******** ** far ** ** ************ cause ***** *********** ** the ******** **** ** the *****. 

***** *********, **** ** Avigilon *** ******* *** not ********** ***** ******** activations.

****** *********** 

** *** *******, ******** crossing * ******** ***** Axis ***** ***** **** often *** ********. ** the ***** *****, *** camera ****** **** *** detect *** ******* ** an ****** ***** ** is **** *** ****. Avigilon's *** ****** *** 100% ******** ***** **** same *****.

** ******* ******, **** was **** *****, **** Axis ******* ******* **** than **% ** *** time, ***** ******** *** successful >**%. ** *** example *****, ******* *** subject ***** **** ** an ****** ***** ** running, *** ****** ***** to ****** *** **** he ****** ******.

**** **** ** ***** FOVs, **** **** **** to ****** *** ******* as * ***** ******, tripwire *********** ****** **** not ** ****** ** *** examples *****. *******, **** still *** ******* ********* running ********, ******* **** about **% ** *** time.

** *****, ****** ********** at ****** ****** **** frequent, ** ****, **** with *** ******* ** well ***** ****** ******** areas ** *** *****, reviewed ** **** *****:

********* *****

***** * ~**° ********** field ** ****, ***** Suite ********* **** ******** at * ******* ***** of ***** ***' ***** *** 1080p ***** ****** *** day. **** ** ***** **% less ******** **** *** Avigilon *** ******, ***** covered ~***' ****.

**** **** **** ** the ******* ***** ******** detection *** ******** (>**%). Beyond ****, ******** ** normal ******* ****** *** **** frequently ******, ****** ********* spotty.

Setup *** ***********

**** ***** ***** ********** perspective *********** *** ***** ** their ******** ************, **** as *** *, ******** to ****** ********* ******** and ****** **** ** measuring ******* ** *** FOV.

**** ******** ******* ******** on **** *******, **** instructing ***** ** *** measurements "** ********* ********* from *** ******."

******** ** *****

******** ** *****, ***** also ******** *********** (******** does ***), ***** *** several *** *********** ** this *******:

  • ** ******* ****/***** ***** window:****' *********** ************ *** performed ** * ***** window, **** ** ****** to ********* **** ** enlarge *** ****** ** more ********** ******* ******** as ** ***** (**** A ** *** ***** below). **** *** **** it ********* ** ****** measurements ** *** ***** top *** ****** ** an ******, ******** ********.
  • ***** ********* ** ********:**** **** * **** to ******* *******, **** a ***** *** ** either *** ***** ********* size, ***** *** ******* the **** ** *** object ***** ********. ***** uses * ******* * shape (**** * *****) to ******* *** ****** scroll ***** ********** ** size, ****** ** ****** to ****** ** *** top *** ****** ** an ******.
  • ** ******** ***********:**** ******** **** ****** of *******, *** ******** calibration ***** ** ***** (and *****'* ****** *** releases). (**** * ** image *****)
  • ** ******* ****:***** ******** * ******* marker ***** ** **** to ****** ****** *********** of *** ******, ** well ** ******* ******* in *** ***. **** does ***.

************, **** ******** ** visual ************ ***** ** check *** ******** ** measurements, **** ** ****** or ********, ** ****** measurements *** ********. ***** and ***** **** ******* this *******.

*******, **** ******** ** maximum **** ******** ** vehicle **. ***** **************. All ******* *** ********, regardless ** ****/*****.

*********** *************

** *** ***** ***** we *********** *** **** calibration ******* *** *** issues ***** *****:

VMS ***********

*********, **** ***** ***** is ******* ** ****** integration *** *** ****** or ***** ***** ****. It ** ****** **** integrations **** ** ******** in *** **** *** months, **** **** *******:

"**** ***** ***** **** be ********* ** **** ****** ******* *** ******* *** ********* Genetec *** ******* ******* through **** *******’ ***** stream, ***** ********* ******* will ******* ****** ***********. Support ** ******* ****., Exacq ************ *** ***** is ******* ** ** available ** *** *** of ****."

** ** ***, ***** may ******* ***** ****** by ********** **** ** the ***** *** ****** in *** ******, ** seen ** *** *** of *** ****** *****. Users *** *** * motion **** ** ********* motion **** ** *** area ***** **** **** appears, *********** ******** ** the ******** ****:

***** *** **** **** TCP ****** ************* **** the ****** ** ***** which ****** ****. **** is ************ ** *** video ***** ***** ***** the *********** ** ***** Suite **** *****, **** event **** ********** *** ********** overlaid **** **** *****.

Comments (9)

Use digital barriers on a Axis - UK iLids approved. IMHO it just works :)

The analytics developed by Axis was informed to target a low-end segment of VCA, although a step up from their VMD and simple cross-line it was not to become a high-end VCA (yet at least). The argument was not to compromise their open platform by competing with the 3rd party higher-end VCA solutions that can run on their cameras.

What I'm a little bit surprised about is the price, I had anticipated that they were going to offer this for a much lower cost, more in line with their previous Cross Line Detection.

I think it is dangerous to offer analytics for a 'low-end segment of VCA'. It is very hard to determine how poorly the analytics will work in different scenarios, which makes it likely that they are deployed in conditions they cannot handle well, which makes customers frustrated and causes integrators to lose lots of money troubleshooting / trying to optimize.

Axis lead marketing claims for Guard Suite is "Robust and reliable indoor and outdoor detection" that "Protect and detect around the clock". If they really wanted it to be 'low-end', then they should have avoided such claims. As is, they have set expectations that they cannot meet.

Yes, unfortunately marketing talk will almost always over-sell products.

As for targeting the low-end, that's what they did with the Cross Line (in my opinion). It seemed to perform rather well compared to its competitors in the tests you performed. But I guess their marketing language wasn't as bold then.

http://ipvm.com/report/analytics-rain-axis-bosch-sony

http://ipvm.com/report/camera-analytics-tested

I would personally be extremely reluctant to use the Axis Cross Line in any outdoor environment since the consequences of missed alarms and a high false alarm rate would be devastating to our costs and brand value.

It will however be interesting to see how well the new Axis analytics perform. Hopefully it is a step up from the old Cross Line and you seemed to think that it at least reduced the false-alarm rate somewhat compared to VMD 3.0. Perhaps it can perform good enough in ideal, sterile environments (as you mentioned in the test).

If you have time for it, it would be interesting to see how the latest Bosch-analytics compares with the new Axis as the latest Bosch should be much better than the old.

Similarly to the Bosch analytics (as well as just about any other VCA on the market that is actively developed), I think we'll see improved performance over time with the new Axis solution, the question is when (or if) it will be a good enough to use in non-ideal scenarios as well and whether it will be enough to be competitive towards the other camera manufacturers. Only time will tell...

Axis Crossline definitely had more false alert issues than you see in Guard Suite. I recall headlights, shadows, and blowing leaves being bigger issues than even what you see here. Guard Suite is at least far less susceptible to the headlights/shadows.

That being said, Crossline also didn't seem to have the same rate of missed objects that Guard Suite does. That's likely because it detected everything as an object, like debris/headlights/shadows, so it was less choosy and less likely to drop an object once detected.

I would love to see an on-board analytic shoot between Pelco, Axis, Samsung, Bosch etc..

The placement of bars for calibration in you tests (see the "Axis Perspective Calibration" Figure) is certainly wrong because bars should be normal to the ground plane. Otherwise the data essential for calibration will be inaccurate. This is best seen with bars 1 and 2 drawn over the two cars. The bars should go along the front door vertical edge (which is not at all "vertical" in the picture).

I don't know if Axis has made a note of this fact in its manual but no calibration is possible with bars drawn at random.

The bars are not drawn at random. The bars you see on the cars were drawn along the door line from the roof to the ground as you point out. However, Axis snaps the lines to vertical after they're drawn, but they remain the same length. I measured the cars those lines were drawn on, as well, to be accurate.

The cars themselves don't appear level due to the angle of view. The FOV overall is level, though, as you can see the poles in the background are vertical.

Also, this scene was later recalibrated when I moved cameras back to this point for further testing.

For reference, Axis gives little guidance as to what is right/wrong in this calibration, nor any best practice in their manual.

Thanks for your clarification. Really strange that Axis "snaps" the bars to vertical... No way to double check the bars had been drawn correctly.

Login to read this IPVM report.
Why do I need to log in?
IPVM conducts unique testing and research funded by member's payments enabling us to offer the most independent, accurate and in-depth information.

Related Reports on Video Analytics

Verint Victimized By Ransomware on Apr 18, 2019
Verint, which is best known in the physical security industry for video surveillance but has built a sizeable cybersecurity business as well, was...
ISC West 2019 Report on Apr 12, 2019
The IPVM team has finished at the Sands looking at what companies are offering and how they are changing their positioning. See below for 50+...
Bosch AI Camera Trainer Released And Tested on Apr 09, 2019
Bosch is releasing a highly unusual new AI feature - 'Camera Trainer'. Now, coming as a standard feature in Bosch IVA/EVA analytics, one can train...
Airship VMS Profile on Apr 03, 2019
Airship has been developing VMS software for over 10 years, however, with no outside investment, and minimal marketing, the company is not well...
IBM / Genetec Surveillance System Investigated Over Philippines Human Rights Abuses on Mar 22, 2019
A lengthy investigation into an IBM video surveillance project in the Philippines, raising concerns IBM helped local police conduct a bloody...
Large Hospital Security End User Interview on Mar 21, 2019
This large single-state healthcare system consists of many hospitals, and hundreds of health parks, private practices, urgent care facilities, and...
BCDVideo Expansion And Switch From HP To Dell Examined on Mar 11, 2019
BCDVideo says they have more than tripled revenue in the past 5 years and are continuing to grow, powered most recently by switching their lead...
Church Technology Director Security Interview on Mar 07, 2019
With 40+ years of experience in IT from a wide array of verticals, including US and foreign military, and large corporate and industrial settings,...
Police Department Surveillance Manager Interview on Feb 28, 2019
Former Memphis PD Surveillance Manager, Lt. Joseph Patty retired months ago, but kept busy during his decades on the force, working to build up...
BluB0x Company Profile on Feb 20, 2019
BluB0x has doubled in revenue every year since its founding in 2013, according to CEO Patrick Barry. We originally reported on them in 2015. At the...

Most Recent Industry Reports

19 Facial Recognition Providers Profiled on Apr 23, 2019
IPVM interviewed 19 facial recognition providers at ISC West to understand their claimed accuracy, success and positioning. 9 from China, where...
Locking Down Network Connections Guide on Apr 23, 2019
Accidents and inside attacks are risks when network connections are not locked down. Security and video surveillance systems should be protected...
Hikvision Admits USA Sales Falling on Apr 22, 2019
Hikvision, in a new Chinese financial filing, has admitted that its USA sales are now falling. Less than a year after the US government passed a...
Speco Ultra Intensifier Tested on Apr 22, 2019
While ISC West 2019 named Speco's Ultra Intensifier the best new "Video Surveillance Cameras IP", IPVM testing shows the camera suffers from...
Arecont Favorability Results 2019 on Apr 22, 2019
Arecont's net negativity remained the same in IPVM's 2019 integrator study, though integrator's feeling became relatively more neutral compared to...
H.265 Usage Statistics on Apr 19, 2019
H.265 has been available in IP cameras for more than 5 years and, in the past few years, the number of manufacturers supporting this codec has...
ACRE Acquires RS2, Explains Acquisition Strategy on Apr 19, 2019
ACRE continues to buy, now acquiring RS2, just 5 months after buying Open Options. One is a small access control manufacturer from Texas, the...
Access Control Course Spring 2019 - Last Chance on Apr 19, 2019
 Register for the Spring 2019 Access Control Course----Closed IPVM offers the most comprehensive access control course in the industry. Unlike...
Riser vs Plenum Cabling Explained on Apr 18, 2019
You could be spending twice as much for cable as you need. The difference between 'plenum' rated cable and 'riser' rated cable is subtle, but the...
Verint Victimized By Ransomware on Apr 18, 2019
Verint, which is best known in the physical security industry for video surveillance but has built a sizeable cybersecurity business as well, was...

The world's leading video surveillance information source, IPVM provides the best reporting, testing and training for 10,000+ members globally. Dedicated to independent and objective information, we uniquely refuse any and all advertisements, sponsorship and consulting from manufacturers.

About | FAQ | Contact