Advanced VMD - Axis / Mobotix TestedBy: Ethan Ace, Published on Aug 26, 2013
Surveillance professionals prefer motion based recording over continuous but the main weakness of motion detection is its inaccuracy. Sure, it saves a lot of storage over recording continuously but it still can be very inefficient, triggering against leaves, trees, shadows, low light noise and more, requiring time consuming optimization that rarely works completely.
Axis and Mobotix Advanced Offerings
Recently, both Axis and Mobitix have released advanced video motion detection offerings. Axis has VMD 2.1 that it bills as having 'easy setup with few false alarms' while Mobotix promotes MxActivitySensor as offering 'unique motion detection in the camera'.
The IPVM Test
We tested both of these offerings to understand how much of an improvement they made over 'traditional' video motion detection plus how accurate they were.
To do so, we bought 2 Axis cameras (same model) and loaded one with traditional and one with VMD 2.1, running them simultaneously.
We started with a simple conference room and performed tests with moving subjects as well as running the cameras 24/7 to measure false positives:
We then moved on to a more complex, larger area to see how the offerings handled this:
Finally, we went outdoors to a very large area with vegetation to see how they would fare in an adverse environment:
Finally, we ran the Mobotix camera through similar conditions to gauge its performance.
Our testing answers the following key questions:
- How much of an advantage did advanced VMD provide over traditional?
- Where was advanced VMD better than traditional and where was it worse?
- Is it worth using advanced VMD? If so, where?
- What barrier or issues are there in using 'advanced' VMD?
- Significant advantages exist using either Axis and Mobotix advanced VMD over traditional. Common false alerts were nearly eliminated.
- The only limitation of advanced VMD was in the dark where they would miss objects. However, traditional VMD was far more likely to generate huge numbers of false alerts in the dark.
- Using advanced VMD is worth it in pretty much any application. In dark areas, traditional VMD is fairly useless anyway so it would be better to add illumination if detection was critical.
- VMS support is the main issue. While both Axis and Mobotix offerings are free and can be easily added to most of their cameras, VMS support is a problem. Mobotix only supports Mobotix (no surprise). Axis VMD 2.1 is surprisingly only supported by their own ACS VMS and Genetec.
- Axis Video Motion Detection 2.1 performs better in most conditions than their default motion detection, with better detection rates and ranges, and very few false alarms.
- However, in low light scenes (~2 lux and below in our tests) VMD 2.1 missed more activations than default. Its false alarm rate remained very low.
- Axis' default motion detection was highly sensitive to changes in light, such as flashes, moving car headlghts, shadows, etc., creating large numbers of false alerts due to these effects.
- Even at its smallest object size and higher sensitivity, standard VMD missed small moving objects which VMD 2.1 did not.
- VMD performance was relatively close between P and M series cameras, though P series cameras performed at lower light levels, where M series cameras produce practically no image.
- VMD 2.1 requires nearly no setup, only a single polygonal included and optional excluded zone with no sensitivity configuration.
- Standard VMD allows more windows (both included and exclude), though zones are restricted to purely rectangular shapes, instead of polygonal, making coverage of odd-shaped areas more difficult. Users must configure object size for best performance in each VMD zone.
- Mobotix MxActivitySensor (installed on a Q24) had a very high detection rate up to about 30' in full light, displaying direction of objects via a video overlay, as well. However, the camera's poor low light performance resulted in nearly 100% missed detections at 2 lux or below.
- MxActivitySensor's working range was much larger when the object was moving across the FOV, able to detect a subject at over 100', vs. ~30' when moving toward/away from the camera.
- MxActivitySensor is also nearly immune to lights and repetitive motion caused by stationary objects, such as tree branches and leaves. Activation is based on directional travel only.
Key Test Findings
There are several key findings from our tests:
Axis Video Motion 2.1 is available as a free download directly from Axis. MxActivitySensor is included standard in all Mobotix cameras running firmware version 4.1.6 and above.
For predominantly daytime scenes, users may find VMD 2.1 offers easier setup and better performance than Axis' default motion detection, at no additional cost. Low light scenes remain problematic as VMD 2.1 misses events frequently while standard motion detection generates a ton of false alerts. Using boost recording (motion + continuous) with advanced motion detection is likely an ideal approach.
For those using Axis' standard motion detection, optimizing camera positioning so that headlights, flashes, or substantial shadows are out of the FOV is key though often unrealistic. False alerts at night due to these issues can easily eliminate the bandwidth and storage benefits of motion recording. However, moving to VMD 2.1 would rectify this without any optimization.
MxActivitySensor offers easy setup and solid performance, including directional indication, with high false alert immunity. Users should consider camera location and typical direction of objects when deploying it, however, as its range and accuracy is much greater with objects moving across the FOV.
Although Axis VMD 2.1 and MxActivitySensor both have distinct advantages over each manufacturer's standard motion detection, VMS support remains a key weakness.
Axis Configuration Comparison
This screencast reviews setup procedure for both traditional VMD and VMD 2.1, as well as the differences in configuration, showing the various options in traditional vs advanced:
Indoor Conference Room
In our indoor conference room, in full light, detection of a human subject was reliable at all ranges, with 30' the farthest tested range. Both VMD 2.1 and standard VMD detected our subject 100% of the time.
VMD 2.1 was able to capture very small movements, however, such as the moving chair leg in the bottom corner of this image. Standard VMD, even at high sensitivity and 1% object size, could not.
In low light, both VMD 2.1 and standard were accurate in this scene, as well. VMD 2.1 missed one activation out of numerous tests, while standard did not miss any.
However, in low light, standard VMD was highly prone to false alarms from changes in light, such as flashes, lights being switched on or off, computer screens, and shadows created as people moved past the doorway. VMD 2.1 was completely immune to these in this scene.
Indoor Open Office
Moving the cameras to an open office scene, about 45' long, full light performance was reliable for both VMDs, up to about 25'.
Beyond this, VMD 2.1 missed very few detections, while standard VMD missed nearly all.
In low light, our subject moving down the aisle triggering both, though standard VMD was more accurate, missing no detections. VMD 2.1 missed three out of 18 in this case (~83% accurate).
However, moving across the FOV from desk to desk, VMD 2.1 never activated. Standard VMD was nearly 100% accurate up to ~15', but missed all detections beyond this.
As was the case in the conference room, standard VMD once again had a huge false alarm rate, caused by changes in light.
Outdoors, in daylight, moving leaves (a common cause of nuisance alarms in VMD and analytics) triggered constant motion alarms on both standard VMD and 2.1. We excluded these areas from both, as we were unable to test detection otherwise.
This scene is about 25' wide from building to fence, and about 280' the nearby road. With a subject walking through the FOV, both cameras activated. However, VMD 2.1 picked up our subject at ~80-90' away from the camera, while standard VMD triggered only around 45'.
Though standard failed to detect subjects moving toward or away from the camera past ~45', both cameras were able to detect our subject as he walked across the field of view at approximately 65'.
VMD 2.1 captured cars on the nearby road during the day. Standard VMD detected only about 5% of the vehicles on the near road.
VMD 2.1 was also able to detect trucks on the distant highway, over 600' from the camera.
In low light, standard VMD's performance was similar to daytime, though range was slightly shorter, about 40'. VMD 2.1 was unable to detect our subject in this scene at all, missing all activations.
At night, VMD 2.1 captured about 75% of vehicles, while standard detected only about 30%.
Interestingly, VMD 2.1 was better able to detect vehicles on the distant highway at night, triggering not only on large trucks, but also on cars.
As the sun rose, glare reduced VMD's effectiveness to zero. VMD 2.1 performed better, capturing about 40% of events, including even a groundhog moving throughout the scene.
We adjusted Axis' default VMD sensitivity settings to see if there was a "sweet spot" at which false alarms were reduced, without impacting detection rates. We found that the default setting, 90%, performed best in the scenes we tested. Lowering sensitivity to 80% resulted in similar detection performance, without any decrease in false alerts. Below this, missed detections become more common, though changes in light still caused false events. Conversely, increasing sensitivity past 90% only marginally improved detection, but false motion due to digital noise and artifacts became constant.
M Series Performance
In order to test performance on other Axis camera models, we installed VMD 2.1 on two M series Axis minidomes, the M3004 and M3006. We found that performance in full light was comparable to the P3354, but detection in low light was impaired, since the M series produced nearly no image at low light levels.
Mobotix configuration is moderately complex, though probably not by Mobotix's standards:
Mobotix Activity Sensor
In indoor tests, MxActivitySensor captured our subject at 30' range 100% of the time in full light. Note the overlays of object direction (top left) and activity graph (bottom left).
Accuracy was unchanged regardless of whether the subject moved across the FOV or toward the camera. In this case detection began at about 25', with the subject approaching the camera.
In low light, however, due to the Q24's poor nighttime performance, it was unable to capture our subject at all. It was completely immune to false alarms from flashes, shadows, and other changes in light, however.
Outdoors, MxActivitySensor performed similarly to indoors, with reliable detection out to about 30' with the subject moving toward the camera. Also, notably, the large vegetation area to the right of the FOV which was moving in the breeze, was not excluded from the motion zone, yet no false alarms were triggered.
Moving across the FOV, however, range was much greater. The camera was able to detect our subject at over 100', as shown here: