Axis Video Analytics Are Weak

Published Oct 05, 2015 04:00 AM

For more than a decade, video analytics has frustrated and disappointed users.

Now, Axis has released their own "series of robust video analytics applications" that they call Guard Suite [link no longer available]. Unfortunately, IPVM's testing shows that Axis' video analytics are weak. We tested both the Q1615 and the P1428E with the motion and fence applications ($149 MSRP each but free 30-day trial applications are available here [link no longer available]).

Compared to Axis' previous effort, VMD 3, some false alarm sources have been reduced or eliminated. In our tests, Guard Suite did not classify blowing debris, birds, or rain as objects, occasional problems in VMD 3. 

In well lit, moderate size, vegetation-free, calm dry weather, monitoring subjects moving slowly Axis Guard Suite should work accurately and reliably.

Negatives

Axis Guard Suite is dumb.

Smart cameras may be the future, but Axis Guard Suite struggles to reliably handle non-ideal conditions. With modest real world conditions, false positives and false negatives become significant with Guard Suite struggling to recognize what is a real human vs leaves, wires, windows, etc. These problems are far more severe than Avigilon (tested) and DVTEl ioimage (tested).

Compounding the lack of intelligence is a cumbersome setup process that lacks advanced controls or options that might help compensate for this, unlike Avigilon, which largely handles this automatically, and DVTel ioimage that provides sophisticated setup and calibration to improve performance.

Market Impact / Recommendations

Do not use Axis Guard Suite unless you use it in simple indoor scenes or verify in at least one week of testing in the exact location you plan to use it. Despite Axis' market claims, Guard Suite is not 'robust'.

We expect integrators using this to be frustrated by the poor performance and the inability to overcome it. We do not see this being widely used, forcing Axis to continue to focus on 3rd party analytic apps.

Performance Review

In this section we break down the key issues impacting Axis Guard Suite performance, including false alarm issues, missed activations, and more, in this video:


False Activations 

Axis' analytics, like VMD 3 before it, suffer from numerous false activations from brush, branches, foliage, and other vegetation. In the scene below, the brush to the right side of the FOV is seen as objects by the camera, despite calibrating for size and perspective. This extended so far as to occasionally cause false activations on the tripwire seen in the grass. 

Other analytics, such as Avigilon and ioimage did not experience these frequent activations.

Missed Activations 

In our testing, subjects crossing a tripwire using Axis Fence Guard were often not detected. In the scene below, the camera simply does not detect our subject as an object until he is past the line. Avigilon's H3A camera was 100% accurate using this same setup.

At running speeds, this was even worse, with Axis missing runners more than 50% of the time, while Avigilon was successful >90%. In the example below, despite the subject being seen as an object prior to running, the camera fails to detect him once he begins moving.

Note that in wider FOVs, with more time to detect the subject as a human object, tripwire reliability issues were not as severe as the examples above. However, Axis still had trouble detecting running subjects, missing them about 20% of the time.

At night, missed detections at normal speeds were frequent, as well, even with the subject in well light higher contrast areas of the scene, reviewed in this video:

Detection Range

Using a ~60° horizontal field of view, Guard Suite analytics were accurate at a maximum width of about 175' using the 1080p Q1615 during the day. This is about 20% less coverage than the Avigilon H3A bullet, which covered ~225' HFOV.

Note this HFOV is the maximum where reliable detection was possible (>95%). Beyond this, subjects at normal walking speeds are more frequently missed, making detection spotty.

Setup and Calibration

Axis Guard Suite introduces perspective calibration not found in their previous applications, such as VMD 3, intended to better calculate distance and object size by measuring objects in the FOV.

Axis provides limited guidance on this process, only instructing users to use measurements "at different distances from the camera."

Compared to DVTel

Compared to DVTel, which also requires calibration (Avigilon does not), there are several key limitations in this process:

  • No digital zoom/small video window: Axis' perspective measurements are performed in a small window, with no option to digitally zoom or enlarge the screen to more accurately measure subjects as in DVTel (item A in the image below). This may make it difficult to adjust measurements to the exact top and bottom of an object, reducing accuracy.
  • Lines difficult to position: Axis uses a line to measure objects, with a large dot at either end while adjusting size, which can obscure the edge of the object being measured. DVTel uses a capital I shape (item B below) to measure and allows scroll wheel adjustment of size, making it easier to adjust to the top and bottom of an object.
  • No distance calibration: Axis provides only height of objects, not distance calibration found in DVTel (and Bosch's latest IVA releases). (Item C in image below)
  • No horizon line: DVTel includes a horizon marker which is used to better adjust perspective of the camera, as well as exclude objects in the sky. Axis does not.

Additionally, Axis provides no visual verification tools to check the accuracy of measurements, such as height or distance, to ensure measurements are accurate. DVTel and Bosch both provide this feature.

Finally, Axis provides no maximum size settings or vehicle vs. human discrimination. All objects are detected, regardless of size/shape.

Calibration Demonstration

In the video below we demonstrate the Axis calibration process and the issues noted above:

VMS Integration

Currently, Axis Guard Suite is limited to manual integration via TCP serial or other means only. It is likely that integrations will be complete in the next few months, with Axis stating:

"AXIS Guard Suite will be supported by AXIS Camera Station and leading VMS providers Genetec and Salient Systems through Axis cameras’ event stream, while Milestone Systems will provide direct integration. Support by Aimetis Corp., Exacq Technologies and OnSSI is planned to be available by the end of 2015."

As of now, users may display alarm events by overlaying text on the video via events in the camera, as seen at the top of the screen below. Users may set a motion zone to detection motion only in the area where this text appears, effectively alerting on the analytic rule:

Users may also send TCP serial notifications from the camera to VMSes which accept them. This is demonstrated in the video below which shows the integration of Guard Suite with Exacq, with event text searchable and optionally overlaid onto live video.

Comments are shown for subscribers only. Login or Join