VMD Camera Shootout

Author: Ethan Ace, Published on Oct 28, 2013

Video motion detection's inaccuracy can have disastrous effects on your VMS system, with recordings due to positives easily consuming far more storage than estimated. On the other hand, optimizing VMD to reject these alarms can decrease detection rates, missing real events.

In order to test whose on-camera VMD performed best, we used cameras from Avigilon, Axis, Bosch, and Sony in multiple scenes, day and night, starting with a simple indoor conference room:

Moving on to a more complex open office area:

And finally, an outdoor area with moving vegetation and headlights on a nearby road:

Our tests looked to answer the following key questions:

  • What issues most commonly cause false positives?
  • Whose VMD was best at rejecting these events?
  • How did detection range vary from manufacturer to manufacturer?
  • Can settings be optimized for better detection without increasing false alerts and vice versa?

***** ****** *********'* ********** *** **** ********** ******* ** **** *** ******, with ********** *** ** ********* ****** ********* *** **** ******* than *********. ** *** ***** ****, ********** *** ** ****** these ****** *** ******** ********* *****, ******* **** ******.

** ***** ** **** ***** **-****** *** ********* ****, ** used ******* **** ********, ****, *****, *** **** ** ******** scenes, *** *** *****, ******** **** * ****** ****** ********** room:

****** ** ** * **** ******* **** ****** ****:

*** *******, ** ******* **** **** ****** ********** *** ********** on * ****** ****:

*** ***** ****** ** ****** *** ********* *** *********:

  • **** ****** **** ******** ***** ***** *********?
  • ***** *** *** **** ** ********* ***** ******?
  • *** *** ********* ***** **** **** ************ ** ************?
  • *** ******** ** ********* *** ****** ********* ******* ********** ***** alerts *** **** *****?

[***************]

Key ********

***** ********* **** ***** **** *** ******* ***** ** *** tests, **** *** ********* ******** ****** **********:

  • ** ****** ***** ***** ** ***** "****" ** ********* ***** positives, ***** *** ******** ******** *********** **** ********* **********, *** and *****, ** ***** *** **** *****. *** ******** ** 1MP ******** ****** *** ** ******* ** ***** ****** **** others. *** **** ***** *** ********* ** ********* ********** **** than ***** *******. ***** **** ***** *** ******, *********** *** about *****.
  • *** ******* ******** ******** ***** ********* *** ** *** ******** of ********** ** *** *****, *** ** *****, ******* **** continuous *********. ** *** *****, **** **** ***** ** ****** out ******* ********* ***********, ** ** ***** ******** *** ******** in **** ****.
  • *******, ******* ****** ** ****** ******** ****** **** ***** ********* which ***** *** ** ******** *** ****** *****, ***** **** fell ********** *** ****** ********* ****.
  • ** *****, *** ******* **** ********* ** *** ********** ** our *****. *** **** ***** *** ********* ** **** **** when *** **** *** ******** **** *** ****** ****.
  • *******, ***** ******* **** *** ******* ***** ** ***** ******. These **** *** ** ***** *******/******* **** * *** *******, subjects ****** **** *** ****, ** *********** ** ********** **********. The **** ***** ** **** ***** ** ***** ****** ** this *****, ******** ** ***** ***-****, **** *****, *** ******** *.*-**-**. 

*** ********* *** ******** ******* *********** ** ******* **** ****** using * ***** *******:

  • *** ******** *.*-**-** ********* **** ** ****** *** ******* ****** at *** ***** ******, ******* **** **** ***** ** **** (<1lux). *** ****** *** *** ******* ********* ***** ***** ****** cameras *** ** ******, ** ****, ****** ********* ** ****** and ******** ** *** **** **********.  
  • ** ******* ******* *** **** ***** ****** *****, **** *****, Bosch ***-**** *** **** ***-***** ******* ******* ****** ********* *********** in ***** ** ********* ****** *** ***** ***** *****. 
  • ** *** *****, *** *********** ** ***** ***-**** *** **** SNC-VB600 **** **** *******, **** ***** ******** **** ******** ****** detection ******  *** ******** ***** ***** ***** *****.
  • **** *****, ** *** ***** ****, ****** ************* ****** ***** cameras ** *** *****, **** **** ***** ********* ******.

Optimization ***************

***** ** *** *****, ** **** ** *********** ** **** practices **** **** ************, **** *** *** *************** ** *** each ****** *** ** ********* *** **** ***********:

  • *** *** ******** *.*-**-**, ******* ******** ******** ****** *********** ** *** ******, **** ** *********** *******.
  • **** ****' ***** ******** ***, *** ******* ****** **** ******* is *** ***** *** **** ******.  ** ***** ** *** ****** detection ******, *** ****** **** ******* ****** ** ** ******* carefully ** * ***** ******** ** *** *****, ********* * or *****.
  • *** *** ***** ***-****, **** *** *********** *** **** ******** by ***** *** ******* ******** ** *** ****** ** ******. Lowering *** ****** **** ******** ** ************* ****** ***** ****** due ** ******* *** ****. 
  • *** *** **** ***-*****, *** ****** **** ****** ** ******* to *** ******* ***** *** **** ***********. ** *** *******, *********** can **** ** ******** ********* **** *** ******* ******* ** *** a ****** ********* *****. *** ** ****** ***** ***** ***** ** *** *****, ******* *********** ** ***** ** ** **** *********** in ***** **********.

***************

*** ** *************** ***** ****** **** ****** ******, ********, *******, *** ***** variations, ***** ****** ********* ******** ****** ********* *** ******* ******* when ******** ** *** ***** ****** *********. ****** ******* **** from ********** *** ****** ******** **** ******** **** ******* ******* performance, **** ****** ***** ******* ******* ** ***** *****.

******* *** ******** ****** *** ** ******* ******* *** *** cameras ** *** ******. ** **** ***** ***** ******** **** result ** ******** ***** ********* ** ****** **********. ***** ****** plan ** ****** ***** ******** **** ******* *** ** *****, and ******** ****** ** ********* ************ ** ****** *********** ** still***********.

*** ***** ********* ******** ****** ********* ** ****** ****** (>***' during *** **** *** >**' *****-****), *** ******** *.*-**-** ****** out ** *** **** ****** ***** *** ****** *******. *******, since ** ********** **** ***** ******* *** *****, **** **** will *** ******* ****** ****** **** *** ******, ******** *** use ** ***-**-*** ******** *******.

Comparison ** *****-** *********

******** ** *** ******** ****** ****** ********* ****** ****, **** ***** ****** ********* *.* ********* ****** ** **** ******. ****** *********, ********* ** *******, and ***** **** *** ********, *** *** ***** *********** *** reduced ********. ** ********* ** ** **** **** **** ******* whenever ********, *** *** *********** ** ******* *** ** *********** with **** *** **** (**** *** ******* ****).

******** ** ***** *****-** ***, ****, *****, *** **** ****** ********* **** ** ***** **** ********* ******** **** **** ***** ***** ******** *****. **** ** *** to *** **** ****** ********* ***** *** ******** ***** ******** for **********. ***** *** **** ** ******** ***** ********* ******* of ******** *** ***** ********.

*************

*** ********* ****** ****** ************* ** **** ************'* ***.

******** **

**** *****

**** ********** ******************** ** ****' ***. **** **** **** ***** **** ******** configuration ** **** ***** ****** ********* *.*, ***** ** ********* over ******** *** ******** ********. *******, **** ****** ******* **** on *******' ******** ******** ***. 

***** ***-****

**** *** ***

Outdoor ******* 

********, ** ********, ****** **** ******** *** ****** ********* ********** nuisance ****** ** *** *******. ** ******* ***** ***** ***** ******* windows, ** ******* ** ****** ********* *** *** ******** *********.

**** ****** ** ******** *** ********** ** *** *****, *** cameras **** ***** ********* *** ** *** ******** ** ******* in *** *****. ***** ***** ***** *** ** ******** ** they **** ** *** ****** ** *** ****** ****.

*** ***** ***** ***** * ******* ** * ****** *** ** front ** *** ******. *** ******** ** **** *** **** the *** ******* *** ** ** ****** **** ********** ********* for *** *****.

********* *****

*** *** ********* ***** ****, ** **** * **** ***** about **' **** **** ******** ** *****, *** ***** ***' *** ****** ****. *** ******* ********* due ** ** ****** ******* ******* *** *******. *******, ******** 1.0-H3-B2 ****** ** *** ******* **  ~ ***-***' **** **** the ******, ***** **** *****, ***** ***-**** *** **** ***-***** triggered ** *** **-**' *****. **** ****, **** ***** ******** VMD's ****** **** ****** ** ****** **** ** (*******) ** 1 ** ***** ** ******** ********* ***** **** **' ** 45'. ****, *** **** ***-***** ***'* *********** ******* *** ********* to *** **** *** ******* ***, ***** ******** *** ********* range ** ****** **'. ******* ******** ** ****'* *********** ******** in ********** ****** ********* *** ** ***** *********.

******** *.*-**-** **** ******** ******** ********** ** *** ****** **** (~280') ****** *** ***. **** ***-*****, ***** ***-**** *** **** ***** detected **** ***** **%, **%, *** *% ** ********, ************.

******** *.*-**-** *** **** **** ** ****** ****** ** *** distant *******, **** ***' **** *** ******.

 

Outdoor *********

** *** ******* ********* *****, ***** **** *** *** ******** of ***** *********. ** ** *** *******, ******* **** ****** and ********** ********* ******:

****, ******** ** ****** *** *** **** ** ******* ***** factors ** *** **** **********, **** ***-***** **** ***** **** sensitive ** *** ********** *** ** **** ********** ************ **** *** **********.

********* *****

** *** *****, ******** *.*-**-**'* ***** ********* ***** ********* ** about **' *** ***** ******* *** ******* ***** (***** ** ~40', **** ** ~**', *** **** ** ~**'). **** **** we *** *** ******** *** *********** ******* ** **** ***-***** this ****, ***** ** *** **** ******** ** ******** ***** positives *** ** ******* ** *** *****.

**** *** ******* ****** ****** *** *** ** ~**', ******** 1.0-H3-B2 *** ***** ***-**** ******** *** **********. **** ***-***** ****** him ***** **** ** *** ****, ***** **** ***** *** not ******* ** ***.

** **', **** ** *** ******* *********, **** *** ******* simply *** ***** ** ********.

Indoor ********** ****

*** **** ***** *** *** * ***** ***** **** ** 10%-15% **** *** ****** ** *** **********, ***** ***** ********* about *% ** *** ****.

******* *** **** ******* *** ***** ** *** ***** *** ****** the ********* ***** ***** *****: **** **%, **** **%, ***** 20%. 

**** ****** * ********** ********** *** ****, 

** *** ********** **** ** **** *****, *** ****** **** able ** ******** ****** * ***** ******* **  *** ****** up ** **'. **** **** *** **** ***** ***** *** detect *** ******* ** ******* ******** ** ******* ****** **** (15), *** *** ** ** ******* ** * *** ******** detection.

** *** ***** (~***), *** **** ******** *** ****** ** 100% ******** ** ** **' ** * ******* ****** ** the **** ***** ****.

 

Open ******

****** *** ******* ** ** **** ****** *****, ***** **' long, **** ***** *********** ** *** ******* **** ********. **** again, *** **** *****'* **** ** ******* ** * ** improve ********* *****

** *** *****, *** **** ** *** ****** (~**') *** lightened ** * ***** ********** ******** *****. *** ** ****, all ******* ****** **** ***** ******** *** ****** ** **** as ** ******* *** **** **' ****.

*******, ** *** ****** ***** **** *** ***** ** *** darker ****, **** ***-***** *** ***** ***-**** **** ********* ** ****** **' and **-******** *** ****** ** ***** **'. 

*** **** *****, ** *** ***** ****, ***** **** ****** the ****** ** *' **** **** *** ******. 

Cameras/Firmware ****

*** ********* ******* *** ******** ******** **** **** ** *** test:

  • ******** *.*-**-**: *.*.**.*
  • **** *****: *.**.**
  • ***** ***-****: *.**
  • **** ***-*****: *.*.*

 

Comments (7)

Nice report John and glad to see Avigilon was at the top in the majority of the tests. From an integrator point of view this = peace of mind knowing the cameras are doing their job and catching motion so that we arent getting the angry call from a client asking why the system we just spend a lot of money on missed an event. I was also surprised to see Sony on the low end of the test and creating the most false positives with lighting changes.

Keep up the good tests!

Just trying to keep our reputation up as being biased for Avigilon.... :)

That said, this does underlie that even strong VMD performance still has issues. Clearly, there is room for improvement (camera manufacturers offering VMD++ as well as VMS manufacturers adding support for Axis' VMD2.1).

Did you use multiple windows to configure different sensitivities for wide and/or long scenes? That's one thing I disliked about the Sony cameras all the motion windows are set to the same sensitivity.

Hey Jason, we did not end up using multiple windows in this test because we found that sensitivity had only modest effects in most cases. I believe only Axis in this test would allow us to change sensitivity for individual windows, too, making it a moot point, anyway.

Thanks for the clarification :) I used it quite frequently on Axis cameras and it seemed to help some.

Ethan:

Thanks for the report. The key findings seem to indicate that none of these cameras using motion detection were suitable outdoors, however, elsewhere in the report in comparing one against the other it seemed that maybe they might.

Can you give some idea for the best conditions (masked out moving leaves and cars on street) and using the best settings that would give decent detection, what was the false alarm rate at night over a 12 hour period for that strip of yard area beside the building?

A general clarification to this, "The key findings seem to indicate that none of these cameras using motion detection were suitable outdoors"

None of these cameras are likely suitable for outdoor motion alerts because they are not accurate enough not to drive responders crazy. However, for storage reduction, a key driver of motion detection, there is still benefits, just not as much as there could be if the VMD was more advanced/accurate (see Axis 2.1 and Mobotix test results for comparison).

As for 'false alarm rate', in general, our philosophy is not to cite specific overall percentages - i.e., this camera has an 87.6% alarm rate or accuracy or whatever, because there are enough site specific variances that a single number would be misleading.

That said, I'll let Ethan comment on ranges / patterns of performance for your scenario of masked out leaves and cars, etc.

Login to read this IPVM report.
Why do I need to log in?
IPVM conducts unique testing and research funded by member's payments enabling us to offer the most independent, accurate and in-depth information.

Related Reports on Low Light

Axis Vs. Hikvision IR PTZ Shootout on Sep 20, 2018
Hikvision has their high-end dual-sensor DarkfighterX. Axis has their high-end concealed IR Q6125-LE. Which is better? We bought both and tested...
October 2018 Camera Course on Sep 13, 2018
Today is the last day to save $50 on the October 2018 Camera Course, register now. This is the only independent surveillance camera course,...
Axis First IR PTZ Tested (Q6125-LE) on Aug 21, 2018
Axis is very late in releasing IR PTZs. While competitors such as Hikvision and Dahua have offered them for years, Axis has just released their...
Hikvision PanoVu Mini Tested (Multi-imager + PTZ For ~$500) on Aug 07, 2018
Hikvision has released their first PanoVu Mini multi imager, the PanoVu DS-2PT3326IZ-DE3, with four 1080p imagers, including a PTZ and integrated...
Panasonic 9MP Panoramic Fisheye Tested (WV-X4571L) on Aug 02, 2018
Panasonic has released their latest fisheye camera, the WV-X4571L, with 12MP sensor and 9MP resolution, claiming "extreme image quality" under...
Panoramic Fisheye Camera Shootout - Avigilon, Axis, Dahua, Hanwha, Hikvision, Oncam And Vivotek on Jun 27, 2018
IPVM tested Avigilon, Axis, Dahua, Hanwha, Hikvision, Oncam And Vivotek 12MP panoramic fisheye cameras head to head, as shown in the test setup...
Hikvision 12MP Fisheye Camera Tested (DS-2CD63C2F-IV) on Jun 14, 2018
Hikvision's DS-2CD63C2F-IV is their flagship panoramic camera, with a 12MP imager, 15m integrated IR, smart codec, and more. We tested the 63C2 in...
Introducing Effective PPF (ePPF) - Improving Video Surveillance Designs on Jun 11, 2018
Pixel density (PPF / PPM) is the best metric the industry has to define and project video quality. It allows simple communication of estimated...
Hanwha Low-Cost 4MP Camera Tested (QNV-7010R) on Jun 11, 2018
4MP usage is increasing noticeably, as IPVM 2018 resolution statistics show. And low-cost, fixed focal cameras, are popular for budget...
Hikvision PanoVu 20MP Flexible Camera Tested on Jun 01, 2018
Hikvision has released their first repositionable multi imager cameras with integrated IR included, atypical in competitors. We bought and tested...

Most Recent Industry Reports

Alexa Guard Expands Amazon's Security Offerings, Boosts ADT's Stock on Sep 21, 2018
Amazon is expanding their security offerings yet again, this time with Alexa Guard that delivers security audio analytics and a virtual "Fake...
UTC, Owner of Lenel, Acquires S2 on Sep 20, 2018
UTC now owns two of the biggest access control providers, one of integrator's most hated access control platforms, Lenel, and one of their...
BluePoint Aims To Bring Life-Safety Mind-Set To Police Pull Stations on Sep 20, 2018
Fire alarm pull stations are commonplace but police ones are not. A self-funded startup, BluePoint Alert Solutions is aiming to make police pull...
SIA Plays Dumb On OEMs And Hikua Ban on Sep 20, 2018
OEMs widely pretend to be 'manufacturers', deceiving their customers and putting them at risk for cybersecurity attacks and, soon, violation of US...
Axis Vs. Hikvision IR PTZ Shootout on Sep 20, 2018
Hikvision has their high-end dual-sensor DarkfighterX. Axis has their high-end concealed IR Q6125-LE. Which is better? We bought both and tested...
Avigilon Announces AI-Powered H5 Camera Development on Sep 19, 2018
Avigilon will be showcasing "next-generation AI" at next week's ASIS GSX. In an atypical move, the company is not actually releasing these...
Favorite Request-to-Exit (RTE) Manufacturers 2018 on Sep 19, 2018
Request To Exit devices like motion sensors and lock releasing push-buttons are a part of almost every access install, but who makes the equipment...
25% China Tariffs Finalized For 2019, 10% Start Now, Includes Select Video Surveillance on Sep 18, 2018
A surprise move: In July, when the most recent tariff round was first announced, the tariffs were only scheduled for 10%. However, now, the US...
Central Stations Face Off Against NFPA On Fire Monitoring on Sep 18, 2018
Central stations are facing off against the NFPA over what they call anti-competitive language in NFPA 72, the standard that covers fire alarms....
Hikvision USA Starts Layoffs on Sep 18, 2018
Hikvision USA has started layoffs, just weeks after the US government ban was passed into law. Inside this note, we examine: The important...

The world's leading video surveillance information source, IPVM provides the best reporting, testing and training for 10,000+ members globally. Dedicated to independent and objective information, we uniquely refuse any and all advertisements, sponsorship and consulting from manufacturers.

About | FAQ | Contact