VMD Camera Shootout

Author: Ethan Ace, Published on Oct 28, 2013

Video motion detection's inaccuracy can have disastrous effects on your VMS system, with recordings due to positives easily consuming far more storage than estimated. On the other hand, optimizing VMD to reject these alarms can decrease detection rates, missing real events.

In order to test whose on-camera VMD performed best, we used cameras from Avigilon, Axis, Bosch, and Sony in multiple scenes, day and night, starting with a simple indoor conference room:

Moving on to a more complex open office area:

And finally, an outdoor area with moving vegetation and headlights on a nearby road:

Our tests looked to answer the following key questions:

  • What issues most commonly cause false positives?
  • Whose VMD was best at rejecting these events?
  • How did detection range vary from manufacturer to manufacturer?
  • Can settings be optimized for better detection without increasing false alerts and vice versa?

***** ****** *********'* ********** *** **** ********** ******* ** **** *** ******, with ********** *** ** ********* ****** ********* *** **** ******* than *********. ** *** ***** ****, ********** *** ** ****** these ****** *** ******** ********* *****, ******* **** ******.

** ***** ** **** ***** **-****** *** ********* ****, ** used ******* **** ********, ****, *****, *** **** ** ******** scenes, *** *** *****, ******** **** * ****** ****** ********** room:

****** ** ** * **** ******* **** ****** ****:

*** *******, ** ******* **** **** ****** ********** *** ********** on * ****** ****:

*** ***** ****** ** ****** *** ********* *** *********:

  • **** ****** **** ******** ***** ***** *********?
  • ***** *** *** **** ** ********* ***** ******?
  • *** *** ********* ***** **** **** ************ ** ************?
  • *** ******** ** ********* *** ****** ********* ******* ********** ***** alerts *** **** *****?

[***************]

Key ********

***** ********* **** ***** **** *** ******* ***** ** *** tests, **** *** ********* ******** ****** **********:

  • ** ****** ***** ***** ** ***** "****" ** ********* ***** positives, ***** *** ******** ******** *********** **** ********* **********, *** and *****, ** ***** *** **** *****. *** ******** ** 1MP ******** ****** *** ** ******* ** ***** ****** **** others. *** **** ***** *** ********* ** ********* ********** **** than ***** *******. ***** **** ***** *** ******, *********** *** about *****.
  • *** ******* ******** ******** ***** ********* *** ** *** ******** of ********** ** *** *****, *** ** *****, ******* **** continuous *********. ** *** *****, **** **** ***** ** ****** out ******* ********* ***********, ** ** ***** ******** *** ******** in **** ****.
  • *******, ******* ****** ** ****** ******** ****** **** ***** ********* which ***** *** ** ******** *** ****** *****, ***** **** fell ********** *** ****** ********* ****.
  • ** *****, *** ******* **** ********* ** *** ********** ** our *****. *** **** ***** *** ********* ** **** **** when *** **** *** ******** **** *** ****** ****.
  • *******, ***** ******* **** *** ******* ***** ** ***** ******. These **** *** ** ***** *******/******* **** * *** *******, subjects ****** **** *** ****, ** *********** ** ********** **********. The **** ***** ** **** ***** ** ***** ****** ** this *****, ******** ** ***** ***-****, **** *****, *** ******** *.*-**-**. 

*** ********* *** ******** ******* *********** ** ******* **** ****** using * ***** *******:

  • *** ******** *.*-**-** ********* **** ** ****** *** ******* ****** at *** ***** ******, ******* **** **** ***** ** **** (<1lux). *** ****** *** *** ******* ********* ***** ***** ****** cameras *** ** ******, ** ****, ****** ********* ** ****** and ******** ** *** **** **********.  
  • ** ******* ******* *** **** ***** ****** *****, **** *****, Bosch ***-**** *** **** ***-***** ******* ******* ****** ********* *********** in ***** ** ********* ****** *** ***** ***** *****. 
  • ** *** *****, *** *********** ** ***** ***-**** *** **** SNC-VB600 **** **** *******, **** ***** ******** **** ******** ****** detection ******  *** ******** ***** ***** ***** *****.
  • **** *****, ** *** ***** ****, ****** ************* ****** ***** cameras ** *** *****, **** **** ***** ********* ******.

Optimization ***************

***** ** *** *****, ** **** ** *********** ** **** practices **** **** ************, **** *** *** *************** ** *** each ****** *** ** ********* *** **** ***********:

  • *** *** ******** *.*-**-**, ******* ******** ******** ****** *********** ** *** ******, **** ** *********** *******.
  • **** ****' ***** ******** ***, *** ******* ****** **** ******* is *** ***** *** **** ******.  ** ***** ** *** ****** detection ******, *** ****** **** ******* ****** ** ** ******* carefully ** * ***** ******** ** *** *****, ********* * or *****.
  • *** *** ***** ***-****, **** *** *********** *** **** ******** by ***** *** ******* ******** ** *** ****** ** ******. Lowering *** ****** **** ******** ** ************* ****** ***** ****** due ** ******* *** ****. 
  • *** *** **** ***-*****, *** ****** **** ****** ** ******* to *** ******* ***** *** **** ***********. ** *** *******, *********** can **** ** ******** ********* **** *** ******* ******* ** *** a ****** ********* *****. *** ** ****** ***** ***** ***** ** *** *****, ******* *********** ** ***** ** ** **** *********** in ***** **********.

***************

*** ** *************** ***** ****** **** ****** ******, ********, *******, *** ***** variations, ***** ****** ********* ******** ****** ********* *** ******* ******* when ******** ** *** ***** ****** *********. ****** ******* **** from ********** *** ****** ******** **** ******** **** ******* ******* performance, **** ****** ***** ******* ******* ** ***** *****.

******* *** ******** ****** *** ** ******* ******* *** *** cameras ** *** ******. ** **** ***** ***** ******** **** result ** ******** ***** ********* ** ****** **********. ***** ****** plan ** ****** ***** ******** **** ******* *** ** *****, and ******** ****** ** ********* ************ ** ****** *********** ** still***********.

*** ***** ********* ******** ****** ********* ** ****** ****** (>***' during *** **** *** >**' *****-****), *** ******** *.*-**-** ****** out ** *** **** ****** ***** *** ****** *******. *******, since ** ********** **** ***** ******* *** *****, **** **** will *** ******* ****** ****** **** *** ******, ******** *** use ** ***-**-*** ******** *******.

Comparison ** *****-** *********

******** ** *** ******** ****** ****** ********* ****** ****, **** ***** ****** ********* *.* ********* ****** ** **** ******. ****** *********, ********* ** *******, and ***** **** *** ********, *** *** ***** *********** *** reduced ********. ** ********* ** ** **** **** **** ******* whenever ********, *** *** *********** ** ******* *** ** *********** with **** *** **** (**** *** ******* ****).

******** ** ***** *****-** ***, ****, *****, *** **** ****** ********* **** ** ***** **** ********* ******** **** **** ***** ***** ******** *****. **** ** *** to *** **** ****** ********* ***** *** ******** ***** ******** for **********. ***** *** **** ** ******** ***** ********* ******* of ******** *** ***** ********.

*************

*** ********* ****** ****** ************* ** **** ************'* ***.

******** **

**** *****

**** ********** ******************** ** ****' ***. **** **** **** ***** **** ******** configuration ** **** ***** ****** ********* *.*, ***** ** ********* over ******** *** ******** ********. *******, **** ****** ******* **** on *******' ******** ******** ***. 

***** ***-****

**** *** ***

Outdoor ******* 

********, ** ********, ****** **** ******** *** ****** ********* ********** nuisance ****** ** *** *******. ** ******* ***** ***** ***** ******* windows, ** ******* ** ****** ********* *** *** ******** *********.

**** ****** ** ******** *** ********** ** *** *****, *** cameras **** ***** ********* *** ** *** ******** ** ******* in *** *****. ***** ***** ***** *** ** ******** ** they **** ** *** ****** ** *** ****** ****.

*** ***** ***** ***** * ******* ** * ****** *** ** front ** *** ******. *** ******** ** **** *** **** the *** ******* *** ** ** ****** **** ********** ********* for *** *****.

********* *****

*** *** ********* ***** ****, ** **** * **** ***** about **' **** **** ******** ** *****, *** ***** ***' *** ****** ****. *** ******* ********* due ** ** ****** ******* ******* *** *******. *******, ******** 1.0-H3-B2 ****** ** *** ******* **  ~ ***-***' **** **** the ******, ***** **** *****, ***** ***-**** *** **** ***-***** triggered ** *** **-**' *****. **** ****, **** ***** ******** VMD's ****** **** ****** ** ****** **** ** (*******) ** 1 ** ***** ** ******** ********* ***** **** **' ** 45'. ****, *** **** ***-***** ***'* *********** ******* *** ********* to *** **** *** ******* ***, ***** ******** *** ********* range ** ****** **'. ******* ******** ** ****'* *********** ******** in ********** ****** ********* *** ** ***** *********.

******** *.*-**-** **** ******** ******** ********** ** *** ****** **** (~280') ****** *** ***. **** ***-*****, ***** ***-**** *** **** ***** detected **** ***** **%, **%, *** *% ** ********, ************.

******** *.*-**-** *** **** **** ** ****** ****** ** *** distant *******, **** ***' **** *** ******.

 

Outdoor *********

** *** ******* ********* *****, ***** **** *** *** ******** of ***** *********. ** ** *** *******, ******* **** ****** and ********** ********* ******:

****, ******** ** ****** *** *** **** ** ******* ***** factors ** *** **** **********, **** ***-***** **** ***** **** sensitive ** *** ********** *** ** **** ********** ************ **** *** **********.

********* *****

** *** *****, ******** *.*-**-**'* ***** ********* ***** ********* ** about **' *** ***** ******* *** ******* ***** (***** ** ~40', **** ** ~**', *** **** ** ~**'). **** **** we *** *** ******** *** *********** ******* ** **** ***-***** this ****, ***** ** *** **** ******** ** ******** ***** positives *** ** ******* ** *** *****.

**** *** ******* ****** ****** *** *** ** ~**', ******** 1.0-H3-B2 *** ***** ***-**** ******** *** **********. **** ***-***** ****** him ***** **** ** *** ****, ***** **** ***** *** not ******* ** ***.

** **', **** ** *** ******* *********, **** *** ******* simply *** ***** ** ********.

Indoor ********** ****

*** **** ***** *** *** * ***** ***** **** ** 10%-15% **** *** ****** ** *** **********, ***** ***** ********* about *% ** *** ****.

******* *** **** ******* *** ***** ** *** ***** *** ****** the ********* ***** ***** *****: **** **%, **** **%, ***** 20%. 

**** ****** * ********** ********** *** ****, 

** *** ********** **** ** **** *****, *** ****** **** able ** ******** ****** * ***** ******* **  *** ****** up ** **'. **** **** *** **** ***** ***** *** detect *** ******* ** ******* ******** ** ******* ****** **** (15), *** *** ** ** ******* ** * *** ******** detection.

** *** ***** (~***), *** **** ******** *** ****** ** 100% ******** ** ** **' ** * ******* ****** ** the **** ***** ****.

 

Open ******

****** *** ******* ** ** **** ****** *****, ***** **' long, **** ***** *********** ** *** ******* **** ********. **** again, *** **** *****'* **** ** ******* ** * ** improve ********* *****

** *** *****, *** **** ** *** ****** (~**') *** lightened ** * ***** ********** ******** *****. *** ** ****, all ******* ****** **** ***** ******** *** ****** ** **** as ** ******* *** **** **' ****.

*******, ** *** ****** ***** **** *** ***** ** *** darker ****, **** ***-***** *** ***** ***-**** **** ********* ** ****** **' and **-******** *** ****** ** ***** **'. 

*** **** *****, ** *** ***** ****, ***** **** ****** the ****** ** *' **** **** *** ******. 

Cameras/Firmware ****

*** ********* ******* *** ******** ******** **** **** ** *** test:

  • ******** *.*-**-**: *.*.**.*
  • **** *****: *.**.**
  • ***** ***-****: *.**
  • **** ***-*****: *.*.*

 

Comments (7)

Nice report John and glad to see Avigilon was at the top in the majority of the tests. From an integrator point of view this = peace of mind knowing the cameras are doing their job and catching motion so that we arent getting the angry call from a client asking why the system we just spend a lot of money on missed an event. I was also surprised to see Sony on the low end of the test and creating the most false positives with lighting changes.

Keep up the good tests!

Just trying to keep our reputation up as being biased for Avigilon.... :)

That said, this does underlie that even strong VMD performance still has issues. Clearly, there is room for improvement (camera manufacturers offering VMD++ as well as VMS manufacturers adding support for Axis' VMD2.1).

Did you use multiple windows to configure different sensitivities for wide and/or long scenes? That's one thing I disliked about the Sony cameras all the motion windows are set to the same sensitivity.

Hey Jason, we did not end up using multiple windows in this test because we found that sensitivity had only modest effects in most cases. I believe only Axis in this test would allow us to change sensitivity for individual windows, too, making it a moot point, anyway.

Thanks for the clarification :) I used it quite frequently on Axis cameras and it seemed to help some.

Ethan:

Thanks for the report. The key findings seem to indicate that none of these cameras using motion detection were suitable outdoors, however, elsewhere in the report in comparing one against the other it seemed that maybe they might.

Can you give some idea for the best conditions (masked out moving leaves and cars on street) and using the best settings that would give decent detection, what was the false alarm rate at night over a 12 hour period for that strip of yard area beside the building?

A general clarification to this, "The key findings seem to indicate that none of these cameras using motion detection were suitable outdoors"

None of these cameras are likely suitable for outdoor motion alerts because they are not accurate enough not to drive responders crazy. However, for storage reduction, a key driver of motion detection, there is still benefits, just not as much as there could be if the VMD was more advanced/accurate (see Axis 2.1 and Mobotix test results for comparison).

As for 'false alarm rate', in general, our philosophy is not to cite specific overall percentages - i.e., this camera has an 87.6% alarm rate or accuracy or whatever, because there are enough site specific variances that a single number would be misleading.

That said, I'll let Ethan comment on ranges / patterns of performance for your scenario of masked out leaves and cars, etc.

Login to read this IPVM report.
Why do I need to log in?
IPVM conducts unique testing and research funded by member's payments enabling us to offer the most independent, accurate and in-depth information.

Related Reports on Low Light

Panoramic Fisheye Camera Shootout - Avigilon, Axis, Dahua, Hanwha, Hikvision, Oncam And Vivotek on Jun 27, 2018
IPVM tested Avigilon, Axis, Dahua, Hanwha, Hikvision, Oncam And Vivotek 12MP panoramic fisheye cameras head to head, as shown in the test setup...
Hikvision 12MP Fisheye Camera Tested (DS-2CD63C2F-IV) on Jun 14, 2018
Hikvision's DS-2CD63C2F-IV is their flagship panoramic camera, with a 12MP imager, 15m integrated IR, smart codec, and more. We tested the 63C2 in...
Introducing Effective PPF (ePPF) - Improving Video Surveillance Designs on Jun 11, 2018
Pixel density (PPF / PPM) is the best metric the industry has to define and project video quality. It allows simple communication of estimated...
Hanwha Low-Cost 4MP Camera Tested (QNV-7010R) on Jun 11, 2018
4MP usage is increasing noticeably, as IPVM 2018 resolution statistics show. And low-cost, fixed focal cameras, are popular for budget...
Hikvision PanoVu 20MP Flexible Camera Tested on Jun 01, 2018
Hikvision has released their first repositionable multi imager cameras with integrated IR included, atypical in competitors. We bought and tested...
Stats: Upgrading Cameras Far More Common Than Replacing Failed Cameras on May 30, 2018
The old saying "If It Ain't Broke, Don't Fix It" does not apply here. New IPVM statistics show that 60% of cameras that are replaced still work but...
Oncam 12MP Fisheye Camera Tested on May 29, 2018
Oncam has made their name since the early 2000s as a fisheye specialist, focusing only on panoramic cameras. To see how this specialist stacks up...
Dahua 12MP Fisheye Camera Tested (NK8BR4) on May 16, 2018
Continuing our coverage of 12MP sensor fisheye cameras, we bought and tested the Dahua NK8BR4, examining: Default vs. Optimized...
Vivotek 12MP Fisheye Camera Tested (FE9391-EV) on May 08, 2018
Next in our 12MP fisheye camera evaluation, we bought and tested Vivotek's latest generation FE9391-EV, a new model claiming improved smart IR...
Last Chance - May 2018 Camera Course on May 03, 2018
This is the last chance to register as the course starts next week. This is the only independent surveillance camera course, based on in-depth...

Most Recent Industry Reports

AI Startup Anyvision Raises $28 Million Led By Bosch on Jul 20, 2018
Anyvision is the most ambitious heavy-spending video surveillance startup in many years. And, now, the startup has raised $28 million led by...
Fail: Dahua "Didn't Check The Lux Levels but It Was Dark" on Jul 20, 2018
Dahua UK has been promoting their camera quality on LinkedIn: I, and others, asked what the lux level of the scene was. (background: Lux Rating...
Free 100+ Manufacturer-Customized Camera Calculator Released on Jul 19, 2018
Now, any manufacturer has a customized IPVM Camera Calculator, free. The goal is to make it easier for companies to help their customers better...
Improved Security And Surveillance Bidding - 2018 MasterFormat Divisions Examined) on Jul 19, 2018
Navigating the world of system specifications and bidding work can be complex and confusing, but a standard format exists, and understanding it...
Last Chance - Security Sales Course Summer 2018 on Jul 19, 2018
Today is the last day to register. Based on member's interest, IPVM is offering a security sales course this summer. Register Now - IPVM Security...
Directory of Video Surveillance Startups on Jul 18, 2018
This directory provides a list of video surveillance startups to help you see and research what companies are new or not yet broadly known entity...
Ladder Lockdown and Ladder Levelizer Tested on Jul 18, 2018
Ladders are a daily necessity for surveillance and security installers, but working on an unstable surface can be extremely dangerous. In addition...
FST Fails on Jul 17, 2018
FST was one of the hottest startups of the decade, selected as the best new product at ISC West 2011 and backed with tens of millions in...
Axis ~$100 Camera Tested on Jul 17, 2018
Axis has released their lowest cost camera ever, the Companion Eye Mini L, setting their sights on a market dominated by Hikvision and Dahua. Can...
Amazon Ring Alarm System Tested on Jul 16, 2018
Amazon Ring is going to hurt traditional dealers, and especially ADT, new IPVM test results of Ring's Alarm system underscore. IPVM found that...

The world's leading video surveillance information source, IPVM provides the best reporting, testing and training for 10,000+ members globally. Dedicated to independent and objective information, we uniquely refuse any and all advertisements, sponsorship and consulting from manufacturers.

About | FAQ | Contact