VMD Camera Shootout

Author: Ethan Ace, Published on Oct 28, 2013

Video motion detection's inaccuracy can have disastrous effects on your VMS system, with recordings due to positives easily consuming far more storage than estimated. On the other hand, optimizing VMD to reject these alarms can decrease detection rates, missing real events.

In order to test whose on-camera VMD performed best, we used cameras from Avigilon, Axis, Bosch, and Sony in multiple scenes, day and night, starting with a simple indoor conference room:

Moving on to a more complex open office area:

And finally, an outdoor area with moving vegetation and headlights on a nearby road:

Our tests looked to answer the following key questions:

  • What issues most commonly cause false positives?
  • Whose VMD was best at rejecting these events?
  • How did detection range vary from manufacturer to manufacturer?
  • Can settings be optimized for better detection without increasing false alerts and vice versa?

***** ****** *********'* ********** *** **** ********** ******* ** **** *** ******, with ********** *** ** ********* ****** ********* *** **** ******* than *********. ** *** ***** ****, ********** *** ** ****** these ****** *** ******** ********* *****, ******* **** ******.

** ***** ** **** ***** **-****** *** ********* ****, ** used ******* **** ********, ****, *****, *** **** ** ******** scenes, *** *** *****, ******** **** * ****** ****** ********** room:

****** ** ** * **** ******* **** ****** ****:

*** *******, ** ******* **** **** ****** ********** *** ********** on * ****** ****:

*** ***** ****** ** ****** *** ********* *** *********:

  • **** ****** **** ******** ***** ***** *********?
  • ***** *** *** **** ** ********* ***** ******?
  • *** *** ********* ***** **** **** ************ ** ************?
  • *** ******** ** ********* *** ****** ********* ******* ********** ***** alerts *** **** *****?

[***************]

Key ********

***** ********* **** ***** **** *** ******* ***** ** *** tests, **** *** ********* ******** ****** **********:

  • ** ****** ***** ***** ** ***** "****" ** ********* ***** positives, ***** *** ******** ******** *********** **** ********* **********, *** and *****, ** ***** *** **** *****. *** ******** ** 1MP ******** ****** *** ** ******* ** ***** ****** **** others. *** **** ***** *** ********* ** ********* ********** **** than ***** *******. ***** **** ***** *** ******, *********** *** about *****.
  • *** ******* ******** ******** ***** ********* *** ** *** ******** of ********** ** *** *****, *** ** *****, ******* **** continuous *********. ** *** *****, **** **** ***** ** ****** out ******* ********* ***********, ** ** ***** ******** *** ******** in **** ****.
  • *******, ******* ****** ** ****** ******** ****** **** ***** ********* which ***** *** ** ******** *** ****** *****, ***** **** fell ********** *** ****** ********* ****.
  • ** *****, *** ******* **** ********* ** *** ********** ** our *****. *** **** ***** *** ********* ** **** **** when *** **** *** ******** **** *** ****** ****.
  • *******, ***** ******* **** *** ******* ***** ** ***** ******. These **** *** ** ***** *******/******* **** * *** *******, subjects ****** **** *** ****, ** *********** ** ********** **********. The **** ***** ** **** ***** ** ***** ****** ** this *****, ******** ** ***** ***-****, **** *****, *** ******** *.*-**-**. 

*** ********* *** ******** ******* *********** ** ******* **** ****** using * ***** *******:

  • *** ******** *.*-**-** ********* **** ** ****** *** ******* ****** at *** ***** ******, ******* **** **** ***** ** **** (<1lux). *** ****** *** *** ******* ********* ***** ***** ****** cameras *** ** ******, ** ****, ****** ********* ** ****** and ******** ** *** **** **********.  
  • ** ******* ******* *** **** ***** ****** *****, **** *****, Bosch ***-**** *** **** ***-***** ******* ******* ****** ********* *********** in ***** ** ********* ****** *** ***** ***** *****. 
  • ** *** *****, *** *********** ** ***** ***-**** *** **** SNC-VB600 **** **** *******, **** ***** ******** **** ******** ****** detection ******  *** ******** ***** ***** ***** *****.
  • **** *****, ** *** ***** ****, ****** ************* ****** ***** cameras ** *** *****, **** **** ***** ********* ******.

Optimization ***************

***** ** *** *****, ** **** ** *********** ** **** practices **** **** ************, **** *** *** *************** ** *** each ****** *** ** ********* *** **** ***********:

  • *** *** ******** *.*-**-**, ******* ******** ******** ****** *********** ** *** ******, **** ** *********** *******.
  • **** ****' ***** ******** ***, *** ******* ****** **** ******* is *** ***** *** **** ******.  ** ***** ** *** ****** detection ******, *** ****** **** ******* ****** ** ** ******* carefully ** * ***** ******** ** *** *****, ********* * or *****.
  • *** *** ***** ***-****, **** *** *********** *** **** ******** by ***** *** ******* ******** ** *** ****** ** ******. Lowering *** ****** **** ******** ** ************* ****** ***** ****** due ** ******* *** ****. 
  • *** *** **** ***-*****, *** ****** **** ****** ** ******* to *** ******* ***** *** **** ***********. ** *** *******, *********** can **** ** ******** ********* **** *** ******* ******* ** *** a ****** ********* *****. *** ** ****** ***** ***** ***** ** *** *****, ******* *********** ** ***** ** ** **** *********** in ***** **********.

***************

*** ** *************** ***** ****** **** ****** ******, ********, *******, *** ***** variations, ***** ****** ********* ******** ****** ********* *** ******* ******* when ******** ** *** ***** ****** *********. ****** ******* **** from ********** *** ****** ******** **** ******** **** ******* ******* performance, **** ****** ***** ******* ******* ** ***** *****.

******* *** ******** ****** *** ** ******* ******* *** *** cameras ** *** ******. ** **** ***** ***** ******** **** result ** ******** ***** ********* ** ****** **********. ***** ****** plan ** ****** ***** ******** **** ******* *** ** *****, and ******** ****** ** ********* ************ ** ****** *********** ** still***********.

*** ***** ********* ******** ****** ********* ** ****** ****** (>***' during *** **** *** >**' *****-****), *** ******** *.*-**-** ****** out ** *** **** ****** ***** *** ****** *******. *******, since ** ********** **** ***** ******* *** *****, **** **** will *** ******* ****** ****** **** *** ******, ******** *** use ** ***-**-*** ******** *******.

Comparison ** *****-** *********

******** ** *** ******** ****** ****** ********* ****** ****, **** ***** ****** ********* *.* ********* ****** ** **** ******. ****** *********, ********* ** *******, and ***** **** *** ********, *** *** ***** *********** *** reduced ********. ** ********* ** ** **** **** **** ******* whenever ********, *** *** *********** ** ******* *** ** *********** with **** *** **** (**** *** ******* ****).

******** ** ***** *****-** ***, ****, *****, *** **** ****** ********* **** ** ***** **** ********* ******** **** **** ***** ***** ******** *****. **** ** *** to *** **** ****** ********* ***** *** ******** ***** ******** for **********. ***** *** **** ** ******** ***** ********* ******* of ******** *** ***** ********.

*************

*** ********* ****** ****** ************* ** **** ************'* ***.

******** **

**** *****

**** ********** ******************** ** ****' ***. **** **** **** ***** **** ******** configuration ** **** ***** ****** ********* *.*, ***** ** ********* over ******** *** ******** ********. *******, **** ****** ******* **** on *******' ******** ******** ***. 

***** ***-****

**** *** ***

Outdoor ******* 

********, ** ********, ****** **** ******** *** ****** ********* ********** nuisance ****** ** *** *******. ** ******* ***** ***** ***** ******* windows, ** ******* ** ****** ********* *** *** ******** *********.

**** ****** ** ******** *** ********** ** *** *****, *** cameras **** ***** ********* *** ** *** ******** ** ******* in *** *****. ***** ***** ***** *** ** ******** ** they **** ** *** ****** ** *** ****** ****.

*** ***** ***** ***** * ******* ** * ****** *** ** front ** *** ******. *** ******** ** **** *** **** the *** ******* *** ** ** ****** **** ********** ********* for *** *****.

********* *****

*** *** ********* ***** ****, ** **** * **** ***** about **' **** **** ******** ** *****, *** ***** ***' *** ****** ****. *** ******* ********* due ** ** ****** ******* ******* *** *******. *******, ******** 1.0-H3-B2 ****** ** *** ******* **  ~ ***-***' **** **** the ******, ***** **** *****, ***** ***-**** *** **** ***-***** triggered ** *** **-**' *****. **** ****, **** ***** ******** VMD's ****** **** ****** ** ****** **** ** (*******) ** 1 ** ***** ** ******** ********* ***** **** **' ** 45'. ****, *** **** ***-***** ***'* *********** ******* *** ********* to *** **** *** ******* ***, ***** ******** *** ********* range ** ****** **'. ******* ******** ** ****'* *********** ******** in ********** ****** ********* *** ** ***** *********.

******** *.*-**-** **** ******** ******** ********** ** *** ****** **** (~280') ****** *** ***. **** ***-*****, ***** ***-**** *** **** ***** detected **** ***** **%, **%, *** *% ** ********, ************.

******** *.*-**-** *** **** **** ** ****** ****** ** *** distant *******, **** ***' **** *** ******.

 

Outdoor *********

** *** ******* ********* *****, ***** **** *** *** ******** of ***** *********. ** ** *** *******, ******* **** ****** and ********** ********* ******:

****, ******** ** ****** *** *** **** ** ******* ***** factors ** *** **** **********, **** ***-***** **** ***** **** sensitive ** *** ********** *** ** **** ********** ************ **** *** **********.

********* *****

** *** *****, ******** *.*-**-**'* ***** ********* ***** ********* ** about **' *** ***** ******* *** ******* ***** (***** ** ~40', **** ** ~**', *** **** ** ~**'). **** **** we *** *** ******** *** *********** ******* ** **** ***-***** this ****, ***** ** *** **** ******** ** ******** ***** positives *** ** ******* ** *** *****.

**** *** ******* ****** ****** *** *** ** ~**', ******** 1.0-H3-B2 *** ***** ***-**** ******** *** **********. **** ***-***** ****** him ***** **** ** *** ****, ***** **** ***** *** not ******* ** ***.

** **', **** ** *** ******* *********, **** *** ******* simply *** ***** ** ********.

Indoor ********** ****

*** **** ***** *** *** * ***** ***** **** ** 10%-15% **** *** ****** ** *** **********, ***** ***** ********* about *% ** *** ****.

******* *** **** ******* *** ***** ** *** ***** *** ****** the ********* ***** ***** *****: **** **%, **** **%, ***** 20%. 

**** ****** * ********** ********** *** ****, 

** *** ********** **** ** **** *****, *** ****** **** able ** ******** ****** * ***** ******* **  *** ****** up ** **'. **** **** *** **** ***** ***** *** detect *** ******* ** ******* ******** ** ******* ****** **** (15), *** *** ** ** ******* ** * *** ******** detection.

** *** ***** (~***), *** **** ******** *** ****** ** 100% ******** ** ** **' ** * ******* ****** ** the **** ***** ****.

 

Open ******

****** *** ******* ** ** **** ****** *****, ***** **' long, **** ***** *********** ** *** ******* **** ********. **** again, *** **** *****'* **** ** ******* ** * ** improve ********* *****

** *** *****, *** **** ** *** ****** (~**') *** lightened ** * ***** ********** ******** *****. *** ** ****, all ******* ****** **** ***** ******** *** ****** ** **** as ** ******* *** **** **' ****.

*******, ** *** ****** ***** **** *** ***** ** *** darker ****, **** ***-***** *** ***** ***-**** **** ********* ** ****** **' and **-******** *** ****** ** ***** **'. 

*** **** *****, ** *** ***** ****, ***** **** ****** the ****** ** *' **** **** *** ******. 

Cameras/Firmware ****

*** ********* ******* *** ******** ******** **** **** ** *** test:

  • ******** *.*-**-**: *.*.**.*
  • **** *****: *.**.**
  • ***** ***-****: *.**
  • **** ***-*****: *.*.*

 

Comments (7)

Nice report John and glad to see Avigilon was at the top in the majority of the tests. From an integrator point of view this = peace of mind knowing the cameras are doing their job and catching motion so that we arent getting the angry call from a client asking why the system we just spend a lot of money on missed an event. I was also surprised to see Sony on the low end of the test and creating the most false positives with lighting changes.

Keep up the good tests!

Just trying to keep our reputation up as being biased for Avigilon.... :)

That said, this does underlie that even strong VMD performance still has issues. Clearly, there is room for improvement (camera manufacturers offering VMD++ as well as VMS manufacturers adding support for Axis' VMD2.1).

Did you use multiple windows to configure different sensitivities for wide and/or long scenes? That's one thing I disliked about the Sony cameras all the motion windows are set to the same sensitivity.

Hey Jason, we did not end up using multiple windows in this test because we found that sensitivity had only modest effects in most cases. I believe only Axis in this test would allow us to change sensitivity for individual windows, too, making it a moot point, anyway.

Thanks for the clarification :) I used it quite frequently on Axis cameras and it seemed to help some.

Ethan:

Thanks for the report. The key findings seem to indicate that none of these cameras using motion detection were suitable outdoors, however, elsewhere in the report in comparing one against the other it seemed that maybe they might.

Can you give some idea for the best conditions (masked out moving leaves and cars on street) and using the best settings that would give decent detection, what was the false alarm rate at night over a 12 hour period for that strip of yard area beside the building?

A general clarification to this, "The key findings seem to indicate that none of these cameras using motion detection were suitable outdoors"

None of these cameras are likely suitable for outdoor motion alerts because they are not accurate enough not to drive responders crazy. However, for storage reduction, a key driver of motion detection, there is still benefits, just not as much as there could be if the VMD was more advanced/accurate (see Axis 2.1 and Mobotix test results for comparison).

As for 'false alarm rate', in general, our philosophy is not to cite specific overall percentages - i.e., this camera has an 87.6% alarm rate or accuracy or whatever, because there are enough site specific variances that a single number would be misleading.

That said, I'll let Ethan comment on ranges / patterns of performance for your scenario of masked out leaves and cars, etc.

Login to read this IPVM report.
Why do I need to log in?
IPVM conducts unique testing and research funded by member's payments enabling us to offer the most independent, accurate and in-depth information.

Related Reports on Low Light

Dahua 12MP Fisheye Camera Tested (NK8BR4) on May 16, 2018
Continuing our coverage of 12MP sensor fisheye cameras, we bought and tested the Dahua NK8BR4, examining: Default vs. Optimized...
Vivotek 12MP Fisheye Camera Tested (FE9391-EV) on May 08, 2018
Next in our 12MP fisheye camera evaluation, we bought and tested Vivotek's latest generation FE9391-EV, a new model claiming improved smart IR...
Last Chance - May 2018 Camera Course on May 03, 2018
This is the last chance to register as the course starts next week. This is the only independent surveillance camera course, based on in-depth...
Avigilon 12MP Fisheye Camera Tested on May 03, 2018
12MP fisheye cameras have become mainstream, with nearly every major manufacturer offering their take on this segment. To see how these new higher...
Hikvision DarkfighterX Vs Darkfighter PTZ Tested on Apr 26, 2018
Hikvision has focused on improving low-light performance for PTZs, an area that has traditionally been a problem, even more so than fixed cameras,...
Axis Launches First Stereographic Panoramic Camera on Apr 05, 2018
Fisheye panoramics have not had much innovation in the past few years, with their multi-imager cousins getting most of the advances, e.g.,...
Dahua 4K Micro 4/3 Camera Tested (IPC-HF8835F) on Apr 03, 2018
Dahua is going big. Their new 4K HF8835F Micro Four Thirds sensor is one of the largest now on the market. Dahua is claming: Starlight...
Axis Z-Wave IP Camera Tested Poorly on Mar 20, 2018
Z-Wave is drawing notable interest for video surveillance use. In IPVM's initial coverage, 84% expressed interest in it, with nearly half being...
Camera Form Factor Guide on Mar 16, 2018
When selecting surveillance cameras, users may choose from a number of different form factors, each with its own unique strengths and weaknesses,...
Next Gen 5MP / 6MP Camera Shootout (Axis vs Dahua vs Hanwha vs Hikvision) on Feb 28, 2018
Many manufacturers have released new generation 5MP / 6MP cameras that tout super low light, WDR and other features historically typical in 1080p...

Most Recent Industry Reports

Stealth / UCIT - Remote Video Monitoring Provider Profile on May 18, 2018
Can 2 remote video monitoring companies, Stealth Monitoring from the US and UCIT from Canada combine to impact the market and compete in a changing...
Cybersecurity for IP Video Surveillance Guide on May 18, 2018
Keeping surveillance networks secure can be a daunting task, but there are several methods that can greatly reduce risk, especially when used in...
Forced Entry / Duress Access Tutorial on May 17, 2018
Even though access control normally keeps people safe, tragedies have revealed a significant issue. If users are forced to unlock doors for...
ADT Stock Drops 50% Since IPO on May 17, 2018
It has been a brutal 4 months for ADT. They first expected to IPO at ~$18. They IPOed at $14, dropping immediately to $12.39 And now, not even...
Dahua 12MP Fisheye Camera Tested (NK8BR4) on May 16, 2018
Continuing our coverage of 12MP sensor fisheye cameras, we bought and tested the Dahua NK8BR4, examining: Default vs. Optimized...
Worst Manufacturer Technical Support 2018 on May 16, 2018
5 manufacturers stood out as providing the worst technical support to 190+ integrators in new IPVM results. These integrators answered: In the...
Installing Cameras in Plenums Tutorial on May 15, 2018
There is often confusion about plenum ceilings, with misinformation about what is required when running cables through them and mounting cameras...
Top Benefits Of Attending Trade Shows (Statistics) on May 15, 2018
150 integrators told IPVM: What are the top benefits of going to trade shows? The clear top 2 responses in order: (1) Networking (2) New...
Measuring For Security Installation Guide on May 15, 2018
Accurate measurement is a fundamental skill, yet many installers do not know how to do it. Using these tools are a key skill needed for security...
Arecont Files Ch.11, Agrees To Sell To Turnaround Specialist on May 14, 2018
The long-anticipated sale of Arecont Vision is finally happening. After raising $80 million in debt in 2014, a failed sale in 2017 and the...

The world's leading video surveillance information source, IPVM provides the best reporting, testing and training for 10,000+ members globally. Dedicated to independent and objective information, we uniquely refuse any and all advertisements, sponsorship and consulting from manufacturers.

About | FAQ | Contact