VMD Camera Shootout

Author: Ethan Ace, Published on Oct 28, 2013

Video motion detection's inaccuracy can have disastrous effects on your VMS system, with recordings due to positives easily consuming far more storage than estimated. On the other hand, optimizing VMD to reject these alarms can decrease detection rates, missing real events.

In order to test whose on-camera VMD performed best, we used cameras from Avigilon, Axis, Bosch, and Sony in multiple scenes, day and night, starting with a simple indoor conference room:

Moving on to a more complex open office area:

And finally, an outdoor area with moving vegetation and headlights on a nearby road:

Our tests looked to answer the following key questions:

  • What issues most commonly cause false positives?
  • Whose VMD was best at rejecting these events?
  • How did detection range vary from manufacturer to manufacturer?
  • Can settings be optimized for better detection without increasing false alerts and vice versa?

***** ****** *********'* ********** *** **** ********** ******* ** **** *** ******, with ********** *** ** ********* ****** ********* *** **** ******* than *********. ** *** ***** ****, ********** *** ** ****** these ****** *** ******** ********* *****, ******* **** ******.

** ***** ** **** ***** **-****** *** ********* ****, ** used ******* **** ********, ****, *****, *** **** ** ******** scenes, *** *** *****, ******** **** * ****** ****** ********** room:

****** ** ** * **** ******* **** ****** ****:

*** *******, ** ******* **** **** ****** ********** *** ********** on * ****** ****:

*** ***** ****** ** ****** *** ********* *** *********:

  • **** ****** **** ******** ***** ***** *********?
  • ***** *** *** **** ** ********* ***** ******?
  • *** *** ********* ***** **** **** ************ ** ************?
  • *** ******** ** ********* *** ****** ********* ******* ********** ***** alerts *** **** *****?

[***************]

Key ********

***** ********* **** ***** **** *** ******* ***** ** *** tests, **** *** ********* ******** ****** **********:

  • ** ****** ***** ***** ** ***** "****" ** ********* ***** positives, ***** *** ******** ******** *********** **** ********* **********, *** and *****, ** ***** *** **** *****. *** ******** ** 1MP ******** ****** *** ** ******* ** ***** ****** **** others. *** **** ***** *** ********* ** ********* ********** **** than ***** *******. ***** **** ***** *** ******, *********** *** about *****.
  • *** ******* ******** ******** ***** ********* *** ** *** ******** of ********** ** *** *****, *** ** *****, ******* **** continuous *********. ** *** *****, **** **** ***** ** ****** out ******* ********* ***********, ** ** ***** ******** *** ******** in **** ****.
  • *******, ******* ****** ** ****** ******** ****** **** ***** ********* which ***** *** ** ******** *** ****** *****, ***** **** fell ********** *** ****** ********* ****.
  • ** *****, *** ******* **** ********* ** *** ********** ** our *****. *** **** ***** *** ********* ** **** **** when *** **** *** ******** **** *** ****** ****.
  • *******, ***** ******* **** *** ******* ***** ** ***** ******. These **** *** ** ***** *******/******* **** * *** *******, subjects ****** **** *** ****, ** *********** ** ********** **********. The **** ***** ** **** ***** ** ***** ****** ** this *****, ******** ** ***** ***-****, **** *****, *** ******** *.*-**-**. 

*** ********* *** ******** ******* *********** ** ******* **** ****** using * ***** *******:

  • *** ******** *.*-**-** ********* **** ** ****** *** ******* ****** at *** ***** ******, ******* **** **** ***** ** **** (<1lux). *** ****** *** *** ******* ********* ***** ***** ****** cameras *** ** ******, ** ****, ****** ********* ** ****** and ******** ** *** **** **********.  
  • ** ******* ******* *** **** ***** ****** *****, **** *****, Bosch ***-**** *** **** ***-***** ******* ******* ****** ********* *********** in ***** ** ********* ****** *** ***** ***** *****. 
  • ** *** *****, *** *********** ** ***** ***-**** *** **** SNC-VB600 **** **** *******, **** ***** ******** **** ******** ****** detection ******  *** ******** ***** ***** ***** *****.
  • **** *****, ** *** ***** ****, ****** ************* ****** ***** cameras ** *** *****, **** **** ***** ********* ******.

Optimization ***************

***** ** *** *****, ** **** ** *********** ** **** practices **** **** ************, **** *** *** *************** ** *** each ****** *** ** ********* *** **** ***********:

  • *** *** ******** *.*-**-**, ******* ******** ******** ****** *********** ** *** ******, **** ** *********** *******.
  • **** ****' ***** ******** ***, *** ******* ****** **** ******* is *** ***** *** **** ******.  ** ***** ** *** ****** detection ******, *** ****** **** ******* ****** ** ** ******* carefully ** * ***** ******** ** *** *****, ********* * or *****.
  • *** *** ***** ***-****, **** *** *********** *** **** ******** by ***** *** ******* ******** ** *** ****** ** ******. Lowering *** ****** **** ******** ** ************* ****** ***** ****** due ** ******* *** ****. 
  • *** *** **** ***-*****, *** ****** **** ****** ** ******* to *** ******* ***** *** **** ***********. ** *** *******, *********** can **** ** ******** ********* **** *** ******* ******* ** *** a ****** ********* *****. *** ** ****** ***** ***** ***** ** *** *****, ******* *********** ** ***** ** ** **** *********** in ***** **********.

***************

*** ** *************** ***** ****** **** ****** ******, ********, *******, *** ***** variations, ***** ****** ********* ******** ****** ********* *** ******* ******* when ******** ** *** ***** ****** *********. ****** ******* **** from ********** *** ****** ******** **** ******** **** ******* ******* performance, **** ****** ***** ******* ******* ** ***** *****.

******* *** ******** ****** *** ** ******* ******* *** *** cameras ** *** ******. ** **** ***** ***** ******** **** result ** ******** ***** ********* ** ****** **********. ***** ****** plan ** ****** ***** ******** **** ******* *** ** *****, and ******** ****** ** ********* ************ ** ****** *********** ** still***********.

*** ***** ********* ******** ****** ********* ** ****** ****** (>***' during *** **** *** >**' *****-****), *** ******** *.*-**-** ****** out ** *** **** ****** ***** *** ****** *******. *******, since ** ********** **** ***** ******* *** *****, **** **** will *** ******* ****** ****** **** *** ******, ******** *** use ** ***-**-*** ******** *******.

Comparison ** *****-** *********

******** ** *** ******** ****** ****** ********* ****** ****, **** ***** ****** ********* *.* ********* ****** ** **** ******. ****** *********, ********* ** *******, and ***** **** *** ********, *** *** ***** *********** *** reduced ********. ** ********* ** ** **** **** **** ******* whenever ********, *** *** *********** ** ******* *** ** *********** with **** *** **** (**** *** ******* ****).

******** ** ***** *****-** ***, ****, *****, *** **** ****** ********* **** ** ***** **** ********* ******** **** **** ***** ***** ******** *****. **** ** *** to *** **** ****** ********* ***** *** ******** ***** ******** for **********. ***** *** **** ** ******** ***** ********* ******* of ******** *** ***** ********.

*************

*** ********* ****** ****** ************* ** **** ************'* ***.

******** **

**** *****

**** ********** ******************** ** ****' ***. **** **** **** ***** **** ******** configuration ** **** ***** ****** ********* *.*, ***** ** ********* over ******** *** ******** ********. *******, **** ****** ******* **** on *******' ******** ******** ***. 

***** ***-****

**** *** ***

Outdoor ******* 

********, ** ********, ****** **** ******** *** ****** ********* ********** nuisance ****** ** *** *******. ** ******* ***** ***** ***** ******* windows, ** ******* ** ****** ********* *** *** ******** *********.

**** ****** ** ******** *** ********** ** *** *****, *** cameras **** ***** ********* *** ** *** ******** ** ******* in *** *****. ***** ***** ***** *** ** ******** ** they **** ** *** ****** ** *** ****** ****.

*** ***** ***** ***** * ******* ** * ****** *** ** front ** *** ******. *** ******** ** **** *** **** the *** ******* *** ** ** ****** **** ********** ********* for *** *****.

********* *****

*** *** ********* ***** ****, ** **** * **** ***** about **' **** **** ******** ** *****, *** ***** ***' *** ****** ****. *** ******* ********* due ** ** ****** ******* ******* *** *******. *******, ******** 1.0-H3-B2 ****** ** *** ******* **  ~ ***-***' **** **** the ******, ***** **** *****, ***** ***-**** *** **** ***-***** triggered ** *** **-**' *****. **** ****, **** ***** ******** VMD's ****** **** ****** ** ****** **** ** (*******) ** 1 ** ***** ** ******** ********* ***** **** **' ** 45'. ****, *** **** ***-***** ***'* *********** ******* *** ********* to *** **** *** ******* ***, ***** ******** *** ********* range ** ****** **'. ******* ******** ** ****'* *********** ******** in ********** ****** ********* *** ** ***** *********.

******** *.*-**-** **** ******** ******** ********** ** *** ****** **** (~280') ****** *** ***. **** ***-*****, ***** ***-**** *** **** ***** detected **** ***** **%, **%, *** *% ** ********, ************.

******** *.*-**-** *** **** **** ** ****** ****** ** *** distant *******, **** ***' **** *** ******.

 

Outdoor *********

** *** ******* ********* *****, ***** **** *** *** ******** of ***** *********. ** ** *** *******, ******* **** ****** and ********** ********* ******:

****, ******** ** ****** *** *** **** ** ******* ***** factors ** *** **** **********, **** ***-***** **** ***** **** sensitive ** *** ********** *** ** **** ********** ************ **** *** **********.

********* *****

** *** *****, ******** *.*-**-**'* ***** ********* ***** ********* ** about **' *** ***** ******* *** ******* ***** (***** ** ~40', **** ** ~**', *** **** ** ~**'). **** **** we *** *** ******** *** *********** ******* ** **** ***-***** this ****, ***** ** *** **** ******** ** ******** ***** positives *** ** ******* ** *** *****.

**** *** ******* ****** ****** *** *** ** ~**', ******** 1.0-H3-B2 *** ***** ***-**** ******** *** **********. **** ***-***** ****** him ***** **** ** *** ****, ***** **** ***** *** not ******* ** ***.

** **', **** ** *** ******* *********, **** *** ******* simply *** ***** ** ********.

Indoor ********** ****

*** **** ***** *** *** * ***** ***** **** ** 10%-15% **** *** ****** ** *** **********, ***** ***** ********* about *% ** *** ****.

******* *** **** ******* *** ***** ** *** ***** *** ****** the ********* ***** ***** *****: **** **%, **** **%, ***** 20%. 

**** ****** * ********** ********** *** ****, 

** *** ********** **** ** **** *****, *** ****** **** able ** ******** ****** * ***** ******* **  *** ****** up ** **'. **** **** *** **** ***** ***** *** detect *** ******* ** ******* ******** ** ******* ****** **** (15), *** *** ** ** ******* ** * *** ******** detection.

** *** ***** (~***), *** **** ******** *** ****** ** 100% ******** ** ** **' ** * ******* ****** ** the **** ***** ****.

 

Open ******

****** *** ******* ** ** **** ****** *****, ***** **' long, **** ***** *********** ** *** ******* **** ********. **** again, *** **** *****'* **** ** ******* ** * ** improve ********* *****

** *** *****, *** **** ** *** ****** (~**') *** lightened ** * ***** ********** ******** *****. *** ** ****, all ******* ****** **** ***** ******** *** ****** ** **** as ** ******* *** **** **' ****.

*******, ** *** ****** ***** **** *** ***** ** *** darker ****, **** ***-***** *** ***** ***-**** **** ********* ** ****** **' and **-******** *** ****** ** ***** **'. 

*** **** *****, ** *** ***** ****, ***** **** ****** the ****** ** *' **** **** *** ******. 

Cameras/Firmware ****

*** ********* ******* *** ******** ******** **** **** ** *** test:

  • ******** *.*-**-**: *.*.**.*
  • **** *****: *.**.**
  • ***** ***-****: *.**
  • **** ***-*****: *.*.*

 

Comments (7)

Nice report John and glad to see Avigilon was at the top in the majority of the tests. From an integrator point of view this = peace of mind knowing the cameras are doing their job and catching motion so that we arent getting the angry call from a client asking why the system we just spend a lot of money on missed an event. I was also surprised to see Sony on the low end of the test and creating the most false positives with lighting changes.

Keep up the good tests!

Just trying to keep our reputation up as being biased for Avigilon.... :)

That said, this does underlie that even strong VMD performance still has issues. Clearly, there is room for improvement (camera manufacturers offering VMD++ as well as VMS manufacturers adding support for Axis' VMD2.1).

Did you use multiple windows to configure different sensitivities for wide and/or long scenes? That's one thing I disliked about the Sony cameras all the motion windows are set to the same sensitivity.

Hey Jason, we did not end up using multiple windows in this test because we found that sensitivity had only modest effects in most cases. I believe only Axis in this test would allow us to change sensitivity for individual windows, too, making it a moot point, anyway.

Thanks for the clarification :) I used it quite frequently on Axis cameras and it seemed to help some.

Ethan:

Thanks for the report. The key findings seem to indicate that none of these cameras using motion detection were suitable outdoors, however, elsewhere in the report in comparing one against the other it seemed that maybe they might.

Can you give some idea for the best conditions (masked out moving leaves and cars on street) and using the best settings that would give decent detection, what was the false alarm rate at night over a 12 hour period for that strip of yard area beside the building?

A general clarification to this, "The key findings seem to indicate that none of these cameras using motion detection were suitable outdoors"

None of these cameras are likely suitable for outdoor motion alerts because they are not accurate enough not to drive responders crazy. However, for storage reduction, a key driver of motion detection, there is still benefits, just not as much as there could be if the VMD was more advanced/accurate (see Axis 2.1 and Mobotix test results for comparison).

As for 'false alarm rate', in general, our philosophy is not to cite specific overall percentages - i.e., this camera has an 87.6% alarm rate or accuracy or whatever, because there are enough site specific variances that a single number would be misleading.

That said, I'll let Ethan comment on ranges / patterns of performance for your scenario of masked out leaves and cars, etc.

Login to read this IPVM report.
Why do I need to log in?
IPVM conducts unique testing and research funded by member's payments enabling us to offer the most independent, accurate and in-depth information.

Related Reports on Low Light

May 2018 Camera Course on Apr 20, 2018
Save $50 on early registration until this Thursday, the 26th. Register now (save $50) for the Spring 2018 Camera Course This is the only...
Axis Launches First Stereographic Panoramic Camera on Apr 05, 2018
Fisheye panoramics have not had much innovation in the past few years, with their multi-imager cousins getting most of the advances, e.g.,...
Dahua 4K Micro 4/3 Camera Tested (IPC-HF8835F) on Apr 03, 2018
Dahua is going big. Their new 4K HF8835F Micro Four Thirds sensor is one of the largest now on the market. Dahua is claming: Starlight...
Axis Z-Wave IP Camera Tested Poorly on Mar 20, 2018
Z-Wave is drawing notable interest for video surveillance use. In IPVM's initial coverage, 84% expressed interest in it, with nearly half being...
Camera Form Factor Guide on Mar 16, 2018
When selecting surveillance cameras, users may choose from a number of different form factors, each with its own unique strengths and weaknesses,...
Next Gen 5MP / 6MP Camera Shootout (Axis vs Dahua vs Hanwha vs Hikvision) on Feb 28, 2018
Many manufacturers have released new generation 5MP / 6MP cameras that tout super low light, WDR and other features historically typical in 1080p...
Hikvision 6MP Camera Tested (DS-2CD4565-IZH) on Feb 27, 2018
In the next installment of our ongoing testing of 5MP/6MP cameras, we test the Hikvision DS-2CD4565F-IZH, a 'Smart Series' vandal dome combining...
Hanwha Wisenet X 5MP Camera Tested (XNV-8080R) on Feb 13, 2018
Wisenet X is Hanwha's high-end camera line. We tested their Wisenet X 1080p camera last year. Now Hanwha is offering 5MP cameras listing super low...
Axis 'Best' Q3517-LVE 5MP Camera Tested on Feb 05, 2018
Axis boasts of its Q3517-LVE: Simply put, AXIS Q3517-LVE delivers the best video quality possible. But is it really the 'best'? We bought...
Dahua 5MP Starlight Camera Tested (N52BM3Z) on Jan 30, 2018
Is 5MP the new 1080p? According to our recent statistics, average resolution continues to trend upwards. And now, manufacturers releasing new...

Most Recent Industry Reports

May 2018 Camera Course on Apr 20, 2018
Save $50 on early registration until this Thursday, the 26th. Register now (save $50) for the Spring 2018 Camera Course This is the only...
Global Real-Time Video Surveillance - EarthNow on Apr 20, 2018
A new company, EarthNow, with backing from Bill Gates, Airbus and more, is claiming that: Users will be able to see places on Earth with a delay...
Dedicated Vs Converged Access Control Networks (Statistics) on Apr 20, 2018
Running one's access control system on a converged network, with one's computers and phones, can save money. On the other hand, hand, doing so can...
April 2018 IP Networking Course on Apr 19, 2018
This is the last chance to register for our IP Networking course. Register now. NEW - 2 sessions per class, 'day' and 'night' to give you double...
Rare Video Surveillance Fundraising - Verkada $15 Million on Apr 19, 2018
Fundraising in video surveillance (and the broader physical security market) has been poor recently. Highlights are few and far in between...
'Best In Show' Fails on Apr 19, 2018
ISC West's "Best In Show" has failed. For more than a decade, it has become increasingly irrelevant as the selections exhibit a cartoon level...
Security Camera Cleaning Frequency Statistics on Apr 18, 2018
150+ integrators told IPVM how often they clean cameras on customer's sites and why.  Inside we examine their answers and break down feedback...
Worst Access Control 2018 on Apr 18, 2018
Three access control providers stood out as providing the most problems for integrators. In this report, we analyze the answers to: "In the...
Axis VMD4 Analytics Tested on Apr 17, 2018
Axis is now on its 4th generation of video motion detection (VMD), which Axis calls "a free video analytics application." In this generation, Axis...
Arecont CEO And President Resign on Apr 17, 2018
This is good news for Arecont. Arecont's problems have been well known for years (e.g., most recently Worst Camera Manufacturers 2018 and starting...

The world's leading video surveillance information source, IPVM provides the best reporting, testing and training for 10,000+ members globally. Dedicated to independent and objective information, we uniquely refuse any and all advertisements, sponsorship and consulting from manufacturers.

About | FAQ | Contact