VMD Camera Shootout

Published Oct 28, 2013 04:00 AM

***** ****** *********'* ********** *** **** ********** ******* on **** *** ******, **** ********** due ** ********* ****** ********* *** more ******* **** *********. ** *** other ****, ********** *** ** ****** these ****** *** ******** ********* *****, missing **** ******.

** ***** ** **** ***** **-****** VMD ********* ****, ** **** ******* from ********, ****, *****, *** **** in ******** ******, *** *** *****, starting **** * ****** ****** ********** room:

****** ** ** * **** ******* open ****** ****:

*** *******, ** ******* **** **** moving ********** *** ********** ** * nearby ****:

*** ***** ****** ** ****** *** following *** *********:

  • **** ****** **** ******** ***** ***** positives?
  • ***** *** *** **** ** ********* these ******?
  • *** *** ********* ***** **** **** manufacturer ** ************?
  • *** ******** ** ********* *** ****** detection ******* ********** ***** ****** *** vice *****?

Key ********

***** ********* **** ***** **** *** largest ***** ** *** *****, **** the ********* ******** ****** **********:

  • ** ****** ***** ***** ** ***** "best" ** ********* ***** *********, ***** all ******** ******** *********** **** ********* conditions, *** *** *****, ** ***** the **** *****. *** ******** ** 1MP ******** ****** *** ** ******* in ***** ****** **** ******. *** Sony ***** *** ********* ** ********* headlights **** **** ***** *******. ***** than ***** *** ******, *********** *** about *****.
  • *** ******* ******** ******** ***** ********* due ** *** ******** ** ********** in *** *****, *** ** *****, causing **** ********** *********. ** *** scene, **** **** ***** ** ****** out ******* ********* ***********, ** ** human ******** *** ******** ** **** area.
  • *******, ******* ****** ** ****** ******** caused **** ***** ********* ***** ***** not ** ******** *** ****** *****, since **** **** ********** *** ****** detection ****.
  • ** *****, *** ******* **** ********* to *** ********** ** *** *****. The **** ***** *** ********* ** this **** **** *** **** *** excluded **** *** ****** ****.
  • *******, ***** ******* **** *** ******* cause ** ***** ******. ***** **** due ** ***** *******/******* **** * lit *******, ******** ****** **** *** door, ** *********** ** ********** **********. The **** ***** ** **** ***** to ***** ****** ** **** *****, ******** by ***** ***-****, **** *****, *** Avigilon *.*-**-**. 

*** ********* *** ******** ******* *********** in ******* **** ****** ***** * human *******:

  • *** ******** *.*-**-** ********* **** ** indoor *** ******* ****** ** *** light ******, ******* **** **** ***** to **** (the ******* ********* ***** ***** ****** cameras *** ** ******, ** ****, missed ********* ** ****** *** ******** in *** **** **********.  
  • ** ******* ******* *** **** ***** indoor *****, **** *****, ***** ***-**** and **** ***-***** ******* ******* ****** detection *********** ** ***** ** ********* ranges *** ***** ***** *****. 
  • ** *** *****, *** *********** ** Bosch ***-**** *** **** ***-***** **** also *******, **** ***** ******** **** slightly ****** ********* ******  *** ******** lower ***** ***** *****.
  • **** *****, ** *** ***** ****, lagged ************* ****** ***** ******* ** low *****, **** **** ***** ********* ranges.

Optimization ***************

***** ** *** *****, ** **** as *********** ** **** ********* **** each ************, **** *** *** *************** on *** **** ****** *** ** optimized *** **** ***********:

  • *** *** ******** *.*-**-**, ******* ******** ******** ****** *********** ** *** ******, **** no *********** *******.
  • **** ****' ***** ******** ***, *** default ****** **** ******* ** *** large *** **** ******.  ** ***** ** get ****** ********* ******, *** ****** size ******* ****** ** ** ******* carefully ** * ***** ******** ** the *****, ********* * ** *****.
  • *** *** ***** ***-****, **** *** performance *** **** ******** ** ***** the ******* ******** ** *** ****** we ******. ******** *** ****** **** resulted ** ************* ****** ***** ****** due ** ******* *** ****. 
  • *** *** **** ***-*****, *** ****** size ****** ** ******* ** *** minimum ***** *** **** ***********. ** the *******, *********** *** **** ** ******** ********* from *** ******* ******* ** *** a ****** ********* *****. *** ** ****** ***** ***** ***** in *** *****, ******* *********** ** ***** to ** **** *********** ** ***** conditions.

***************

*** ** *************** ***** ****** **** ****** ******, branches, *******, *** ***** **********, ***** should ********* ******** ****** ********* *** exclude ******* **** ******** ** *** video ****** *********. ****** ******* **** from ********** *** ****** ******** **** possible **** ******* ******* ***********, **** beyond ***** ******* ******* ** ***** areas.

******* *** ******** ****** *** ** assumed ******* *** *** ******* ** all ******. ** **** ***** ***** settings **** ****** ** ******** ***** positives ** ****** **********. ***** ****** plan ** ****** ***** ******** **** cameras *** ** *****, *** ******** should ** ********* ************ ** ****** performance ** ****************.

*** ***** ********* ******** ****** ********* at ****** ****** (>***' ****** *** time *** >**' *****-****), *** ******** 1.0-H3-B2 ****** *** ** *** **** option ***** *** ****** *******. *******, since ** ********** **** ***** ******* via *****, **** **** **** *** receive ****** ****** **** *** ******, limiting *** *** ** ***-**-*** ******** systems.

Comparison ** *****-** *********

******** ** *** ******** ****** ****** detection ****** ****, **** ***** ****** ********* *.* ********* ****** ** **** ******. ****** positives, ********* ** *******, *** ***** were *** ********, *** *** ***** performance *** ******* ********. ** ********* it ** **** **** **** ******* whenever ********, *** *** *********** ** limited *** ** *********** **** **** few **** (**** *** ******* ****).

******** ** ***** *****-** ***, ****, Bosch, *** **** ****** ********* **** ** ***** **** detection ******** **** **** ***** ***** ******** rates. **** ** *** ** *** much ****** ********* ***** *** ******** rules ******** *** **********. ***** *** want ** ******** ***** ********* ******* of ******** *** ***** ********.

*************

*** ********* ****** ****** ************* ** each ************'* ***.

******** **

**** *****

**** ********** ******************** ** ****' ***. **** **** this ***** **** ******** ************* ** Axis ***** ****** ********* *.*, ***** we ********* **** ******** *** ******** possible. *******, **** ****** ******* **** on *******' ******** ******** ***. 

***** ***-****

**** *** ***

Outdoor ******* 

********, ** ********, ****** **** ******** and ****** ********* ********** ******** ****** on *** *******. ** ******* ***** ***** using ******* *******, ** ******* ** motion ********* *** *** ******** *********.

**** ****** ** ******** *** ********** in *** *****, *** ******* **** false ********* *** ** *** ******** of ******* ** *** *****. ***** areas ***** *** ** ******** ** they **** ** *** ****** ** our ****** ****.

*** ***** ***** ***** * ******* ** a ****** *** ** ***** ** one ******. *** ******** ** **** web **** *** *** ******* *** of ** ****** **** ********** ********* for *** *****.

********* *****

*** *** ********* ***** ****, ** used * **** ***** ***** **' wide **** ******** ** *****, *** ***** ***' *** ****** road. *** ******* ********* *** ** an ****** ******* ******* *** *******. However, ******** *.*-**-** ****** ** *** subject **  ~ ***-***' **** **** the ******, ***** **** *****, ***** NBN-733V *** **** ***-***** ********* ** the **-**' *****. **** ****, **** P3354 ******** ***'* ****** **** ****** to ****** **** ** (*******) ** 1 ** ***** ** ******** ********* range **** **' ** **'. ****, the **** ***-***** ***'* *********** ******* was ********* ** *** **** *** default ***, ***** ******** *** ********* range ** ****** **'. ******* ******** in ****'* *********** ******** ** ********** motion ********* *** ** ***** *********.

******** *.*-**-** **** ******** ******** ********** on *** ****** **** (~***') ****** the ***. **** ***-*****, ***** ***-**** *** Axis ***** ******** **** ***** **%, 15%, *** *% ** ********, ************.

******** *.*-**-** *** **** **** ** detect ****** ** *** ******* *******, over ***' **** *** ******.

 

Outdoor *********

** *** ******* ********* *****, ***** were *** *** ******** ** ***** positives. ** ** *** *******, ******* from ****** *** ********** ********* ******:

****, ******** ** ****** *** *** road ** ******* ***** ******* ** the **** **********, **** ***-***** **** still **** ********* ** *** ********** *** ** **** ********** ************ **** the **********.

********* *****

** *** *****, ******** *.*-**-**'* ***** detection ***** ********* ** ***** **' but ***** ******* *** ******* ***** (Bosch ** ~**', **** ** ~**', and **** ** ~**'). **** **** we *** *** ******** *** *********** setting ** **** ***-***** **** ****, since ** *** **** ******** ** generate ***** ********* *** ** ******* in *** *****.

**** *** ******* ****** ****** *** FOV ** ~**', ******** *.*-**-** *** Bosch ***-**** ******** *** **********. **** SNC-VB600 ****** *** ***** **** ** the ****, ***** **** ***** *** not ******* ** ***.

** **', **** ** *** ******* triggered, **** *** ******* ****** *** small ** ********.

Indoor ********** ****

*** **** ***** *** *** * false ***** **** ** **%-**% **** the ****** ** *** **********, ***** Bosch ********* ***** *% ** *** time.

******* *** **** ******* *** ***** ** the ***** *** ****** *** ********* false ***** *****: **** **%, **** 20%, ***** **%. 

**** ****** * ********** ********** *** room, 

** *** ********** **** ** **** light, *** ****** **** **** ** reliably ****** * ***** ******* **  all ****** ** ** **'. **** that *** **** ***** ***** *** detect *** ******* ** ******* ******** at ******* ****** **** (**), *** had ** ** ******* ** * for ******** *********.

** *** ***** (~***), *** **** detected *** ****** ** ***% ******** up ** **' ** * ******* manner ** *** **** ***** ****.

 

Open ******

****** *** ******* ** ** **** office *****, ***** **' ****, **** light *********** ** *** ******* **** reliable. **** *****, *** **** *****'* must ** ******* ** * ** improve ********* *****

** *** *****, *** **** ** the ****** (~**') *** ********* ** a ***** ********** ******** *****. *** to ****, *** ******* ****** **** P3354 ******** *** ****** ** **** as ** ******* *** **** **' away.

*******, ** *** ****** ***** **** the ***** ** *** ****** ****, **** SNC-VB600 *** ***** ***-**** **** ********* ** around **' *** **-******** *** ****** at ***** **'. 

*** **** *****, ** *** ***** hand, ***** **** ****** *** ****** at *' **** **** *** ******. 

Cameras/Firmware ****

*** ********* ******* *** ******** ******** were **** ** *** ****:

  • ******** *.*-**-**: *.*.**.*
  • **** *****: *.**.**
  • ***** ***-****: *.**
  • **** ***-*****: *.*.*

 

Comments (7)
Avatar
Paul Grefenstette
Oct 28, 2013

Nice report John and glad to see Avigilon was at the top in the majority of the tests. From an integrator point of view this = peace of mind knowing the cameras are doing their job and catching motion so that we arent getting the angry call from a client asking why the system we just spend a lot of money on missed an event. I was also surprised to see Sony on the low end of the test and creating the most false positives with lighting changes.

Keep up the good tests!

Avatar
Brian Karas
Oct 28, 2013
Pelican Zero

"Nice report John"

You should give Ethan some credit too, considering he wrote the report and all ;)

JH
John Honovich
Oct 28, 2013
IPVM

Just trying to keep our reputation up as being biased for Avigilon.... :)

That said, this does underlie that even strong VMD performance still has issues. Clearly, there is room for improvement (camera manufacturers offering VMD++ as well as VMS manufacturers adding support for Axis' VMD2.1).

JC
Jason Clement
Oct 28, 2013

Did you use multiple windows to configure different sensitivities for wide and/or long scenes? That's one thing I disliked about the Sony cameras all the motion windows are set to the same sensitivity.

Avatar
Ethan Ace
Oct 29, 2013

Hey Jason, we did not end up using multiple windows in this test because we found that sensitivity had only modest effects in most cases. I believe only Axis in this test would allow us to change sensitivity for individual windows, too, making it a moot point, anyway.

JC
Jason Clement
Oct 29, 2013

Thanks for the clarification :) I used it quite frequently on Axis cameras and it seemed to help some.

Avatar
Robert Baxter
Oct 29, 2013

Ethan:

Thanks for the report. The key findings seem to indicate that none of these cameras using motion detection were suitable outdoors, however, elsewhere in the report in comparing one against the other it seemed that maybe they might.

Can you give some idea for the best conditions (masked out moving leaves and cars on street) and using the best settings that would give decent detection, what was the false alarm rate at night over a 12 hour period for that strip of yard area beside the building?

JH
John Honovich
Oct 29, 2013
IPVM

A general clarification to this, "The key findings seem to indicate that none of these cameras using motion detection were suitable outdoors"

None of these cameras are likely suitable for outdoor motion alerts because they are not accurate enough not to drive responders crazy. However, for storage reduction, a key driver of motion detection, there is still benefits, just not as much as there could be if the VMD was more advanced/accurate (see Axis 2.1 and Mobotix test results for comparison).

As for 'false alarm rate', in general, our philosophy is not to cite specific overall percentages - i.e., this camera has an 87.6% alarm rate or accuracy or whatever, because there are enough site specific variances that a single number would be misleading.

That said, I'll let Ethan comment on ranges / patterns of performance for your scenario of masked out leaves and cars, etc.