Color Low Light Mega Camera Shootout 2019

By Ethan Ace and Derek Ward, Published Nov 12, 2019, 01:39pm EST (Research)

[[****: *** *** **** *********** *** ***** **** ******** ******* **** **-**-**** *******, ********* *** models, ********** ***** ***** ******, *** more ******.]]

**** ** *** ******* ***** *** light ******** ****, ******* **+ ***** low ***** ****** **** ** *************:

IPVM Image

************, **** ************ ****** '*****' *** light ******, **** ********* ***** ******* from *********** ** ******** ************* ** StarLight, ***.

** ******** ***** ****** ******* **** 1080p ** ** **********, *** **** to ****, ** *** *** **** compared ** ******* ****** ** ***** low ***** *********** ** ****:

** **** ****** ** *******:

  • ***** ****** ****** ******* **** *** worst *******?
  • *** ** ******* ****** ****** ********?
  • *** **** *** ****** ** *** light?
  • *** ** ****** **. ****** ******* differ?
  • ** ******* ************ ************** ******* **** world ***********?
  • *** **** ********** ****** ***********?
  • *** **** ********* ******* ******?
  • **** ****** *** ************ ************* **** on ***********?

**** ** * ***** ******** ** what ** *****:

Upcoming ***** *** *******

****: ** **** *** ******* ****** and **** *** ***** ***** *** light ****** ** **** ******'* ******** as **** *** ********. ************, ** have **** ******* ** ******* ******* Super ********* ****** *** **** **** a *****, **** *** ******** ** have ******* *** ** **** *** them ** **** ****** **** ** receive ****.

~1 *** *********** **********

**** ******* ********* **** ** ~* lux ******, **** *** **** *****-* best ******* ** *** *****, **** strong ******* ** ******, *** *****, and ****** ***** ********. *** *****-*/* Axis ** *****-* *** ***** *****-*** also ********* ****.

***** ************* ****** **** ******, **** some *****, ******, *** ********* ****** performing ***** ******* *** ****** *****.

IPVM Image

*** ******* ************** ******* **** *** worst ******* ** **** ***** *** facial ******* ** ****** ********. ****** performers **** ** ****' *****-**/*****-* *** Avigilon's ** *** *** *** ****** from *********** ****, ******* ******* ******* details **** ******, ***** *********, *** Dahua *******, *** ****** ******* *** subject ** ***********, ******** *******.

IPVM Image

<1 *** *********** **********

***** * ***, *** ********* *******'* white ***** *********** ***** ** * significant *********, **** **** ****** ****** and ***** ********. ****** ~* *** performers **** *** **** *****-* *** Q3515-LVE, ** **** ** *** ******** H4 ***, ******* **** ***** ** this *****, *** ***** ******* **** usable ******* ******.

*** **** *****-**'* *********** ***** **** drastically, **** *** ***** **** **** at ~*.* *** *** ** ****** subject *******.

IPVM Image

~2-5 ***: *********** *********** **** **** **********

** *** *******, *********** *********** ** "bright" *** ***** **** *****. *** example, ** * ***, *** ***-********** cameras ** *** **** (**** *****-* and *****-**) ******* ******* ******* ** some *****-******* ******, *** ******* **** and ***** *******.

IPVM Image

**** ******* **** ********** ****** **** as *** ***** ***** *********** *** Sony ***** ******* ******* ***** ** top ********** ** **** *****. *** Sony ****** ***** ******* ***** **** others *** ***** ******** ************ *******. Only *** ********* **-****** ******** *********** worse *******, **** ********* ********* ********* the *******'* ****, ***** ****:

IPVM Image

******* ** ***** ************, ** *** not ******* ***** *-* *** ***********.

Best *** **** (

*** **** ***-**** ****** ****** **** both **** *********, *****-*********-* ******* ******************** ****** ***** ****(~$*** *** ~$*** ******). *** ******* camera ********* **** **** **** *** white ***** ****** *** *** **, discussed ** **** ****** *****.

IPVM Image

***** ***-**** ****** **** ** *** Dahua "***** *****" ******* ******** ***** *************** ******** **** *********** **** *** artifact ****** ** ****** *******, ******* their ****** ******.

Best **** ********** (***+)

*** **** ********** ****** ********** ****** were *** ** **** *****-** *** Avigilon ** *** **. *** **** camera ******** ******** ****** ****** ** moving ******* *** ***** ****** **** the ******** ***.

IPVM Image

**** **** **** ** ******* ** this **** **** ***** *******, **** as *" (**** ******), ***** */* (**** *****-** ******** ****), ** ***-* (******** ** ********** *****). **** ********** ******-******** * **** ******** **** ****** (1/1.8").

Results ****** *******

** ******** ******* ** **** ******** in * * **** *******. ******* can **** **** ********* ****:

Hikvision ******* ***** ***** ************ *********

********* (**** *******) ** *** **** major ********** ************ **** ********** ********** white ***** ************, ****** ** **** to ******* ******* ***** ***** ****** cannot. *** *******, ** ~*.** *** it ** *** **** ****** ***** provides ********* ****** ****** ** *** subject, ***** ****** ******* ********* **** or ** ****** ******* ******.

IPVM Image

*******, ** ***** *********, *******'* ***** light ********* *********. ** ~* ***, other ******* ******* ******* ******* ** the ******* **** **** **** ********, such ** *** *****-** *****.

IPVM Image

***** *******'* ***** ***** ** ********** to **** ******* ** **'* *****-** light, ** *** *** *** ******:

  • ************: ** *** ***** ******, ******* overexposed *** ******* ** ***** *****. For *******, ** *** ***** ***** the ***** ** **** ***********, *** our ******* ******* *********** **** ~**' and ******, ************ *** ******* ****** details.

  • *********** ******** ********: ***** ***** **** little ** ** ******* ******** *** find ******* **********, ***** **** ********* nearby ** ****** ********* ******** *** find *** *****-** ***** ***** *** harsh *** ********, ** ** ** not ** ******* ** ******* ******* lights *** ****** *** ******* ***** lights ***** **.

Color *** ***** ****** ****

***** *** ***** ***** ******* *** improved ****** ************* ** *** **** several *****. *********** *** ******* ****** produced ***** ****** ** ~* *** down ** ~* ***. ~* *** and ***** ** ***** *********** ****** most, ********* ** *** ********** ****** rankings.

***** ** **** ****, ***** *** several ******* *** ********:

  • Super *** ***** ** ****** *******: In 2019, "super low light" models are now common at all price ranges, with low cost fixed lens models from Dahua and Hikvision to high end larger imager models from Avigilon and Axis at ~$2,000+. What was once a rare high end feature is now found in a broad range of models.
  • Low ***** ***** ** *** ******* ***********: Based on tests of color low light cameras spec'd from 0.1 to 0.00022 lux, it is more clear than ever that minimum illumination ratings are meaningless. Cameras with "high" minimum illumination ratings ratings (i.e. 0.1 lux) frequently outperformed those claiming 0.005 lux or lower specs. Users should be skeptical of extreme minimum illumination specifications.
  • Resolution *** ******** *******: Camera resolution was not directly correlated to performance. For example, the Avigilon H4 Pro and Axis Q1798-LE tested above average, while Dahua and Vivotek 4K models were below average. Hikvision's 4MP ColorVu camera outperformed many other more expensive 1080p models, as well.

Newer **. ***** ****** ******

** *******, ***** ******* ********* ****** ** this ********, **** *** **** *****-** and *****-* (**** *********** *), ********* ColorVu *** *********** ******, ******** ********* ******** ******** ** *** **** ****.

*******, **** ***** ****** ********* ****, especially********'* ** *** ******** *********'* ******** ****, **** ******** ** ****.

************, **** *** ******, **** ** Dahua's ***** ***** ******* ********* ****** in *** *****.

No ******* ******* ***** ******* ** ****** ** ***** ******

** *** *****, ** ***** **** all ******* ****** ** ******* ******** identifying ******* ** ******** ******* ****** the ***** ** **** ** ****** speed. **** ******** ********* ** ********, they *** ** ************, *** ******* subjects **** ********* ** ********.

** *** *****, ** ****** ******** clear *********** ***** ****** ****** ** this ********. *********** ** ****, *********, and ***** **** *******. *** **********, the ***** ***** ***** **** ** the ********* ********** ** **** *****, the ******** ** ***, **** *****-*, and ********* ******* ****, **. ****** models, *** **** ****** *** ********* WV-S2531L.

IPVM Image

*** ******** ** *** ** ****** was *** ********* ********* ** **** scene, **** **** ****** ****** **** or *********, **** ***** ****** ******* provided.

*** ********* **-****** *** *** ******* performer ** **** *****, **** ***** artifacting *** ****, **** * ********** artifact ***** ******* ** *** ******* crosses *** *****. **** **** **** blur ** *** ** (*******) **** levels ** *** ******* ** ********* Smart ****** ***** *****, ******** ** our********* ***** ****** ****.

**** *** ****** **** **** ******* visible ** *** ***** ***** ***********, shown ****, **** *** *******'* ***** and ***** **** ******** **** *** background.

IPVM Image

Subject *********** ****** / *********** *********** ********

******, ** ****** ****** **** *** subject *********** *** ******. *********** ****** much **** ** **** *****, **** some ******* *********** *** ******* *** others ******** ** ************.

** *** **** *****-**, *** *******'* face ** ************ ********** *** *****. Minimal ******* ** ***** *** **** are ******* *** *** ***** ** evenly *******.

** *** ***** ******, *** *******'* face ** ******* ** ***-***** *** beyond, *** *********** **** ~**' *** closer. ************, **** ******** ****** *** present ****** *** ******* *** ** DNR.

***** ******* *********** **** *********** ** poor ********, ****, *** ******** *****, as ****, **** ** *** **** VM772R *** ***** ***** ***********, ***** in ******** ** *** *****-** *****.

IPVM Image

*** ********** ***** ***** *** ***-********** models ** **** *****, *** **** Q1798-LE, ******** ** *** **, *** Axis *****-*, ***** ******* ******* ****** of ************ *******, ****** *** ***** performers, *** ********* **-****** *** ***** HF8242, ** ***** *** ******* ** unrecognizable.

IPVM Image

Color ******** ****** ***********

****, ** ****** **** *******'***** ************* * **********-****** ************ *****.

** ~* ***, *** *****-** ******** the ****** ****** ** ******* ******, with *******'* ****** *****. *** ***** N45EM63 *** ********* **-****** ********* ***** here, **** ************ ******* ****** *** shades ** ****.

IPVM Image

** ***** *****, ******'* ******** *** the ***** ****** ******* *** **** accurate ******, ****** ***** ********* **********. Note ****

IPVM Image

Low ****** ***** *******

*******, ** ****** ** * ****** scene ** *** ***** ******* **** "brightest" ******* *** ********* ****** ** still *******, *********** ****** *** ***** who *** ******* *** ********* **** or ***** ******* ** ***** ****** where ******** **** **** *** **** at *** ******.

** ~* ***, *********** ******, **** Axis, *****, *** ******* ****** ********** much ******** ******* **** ** ****** scenes. *********'* **** *** *** ***** N45EM63 ********* ***** ** ~* *** due ** ********* ***** *** ************ of *** **** *****.

IPVM Image

Minimum ************ ***** ** *** ******* ****** ***********

** **** *** ******* *** *** some ****, ******* ************ ************** ****** *** be ******* ** ** ********* ******* of * ******'* *** ***** ***********. In *** *****, *** ***** ******* had *** ****** ******* ************ ****, 0.00022 ***, *** ** ******** **** more ***** *** ********* **** ******, with ****** ****** ** ******. ** contrast, *** **** *****-* ***** * more "************" *.** *** (****+ ******/'*****' than *****) *** ********* **** *** around, **** ***** ******* ** ****** and ****** ***** ********.

IPVM Image

Even **** ***** *** ***** *********** ******** ** *******

**** **** ***** ***** *** ***** camera *********** ****** ****** ** **** test, **** ***** ********** ***** ******* provide ****** ******* ** **** ****** than ***-***** *** ***** ******.

*** *******, ** * ~* *** scene, ****' *****-*, ***** **** *** claim *********** ***** *** *****, ******** no ******* ** *** ******* ** test *****, ***** **** ** *** worst ********** ** **** ***** (**** and *****) ******* ***** *******.

IPVM Image

** ***** *****, ~*.* ***, *********** are **** **** ********** ** *** test ***** ** **** ******* *** less ***** ** *** ***** *** light ****** **** *** *****-*, **** Panasonic *** ***** **** ********* ****** detection ** *** *******, **** **** rough ****** ****** ** *****.

IPVM Image

Pricing / *********** *********

** **** ********, ******-****** ******* *** not ************ **** ****** ***** *** light ***********. *** ******* **** ****** tested *** *** **** *****-*, ***** sells ***~$*** *** ******.

* **** **** ** ********* ******/****** price ******** ** ****** ** *****, listed **** ****** ** ******* *****:

IPVM Image

Bitrate *** *********

*********** ******* ** **/* *** *********, bandwidth *********** ******* ******* ******* ***********, with ****** ***** **** **** ***** producing ***** ********. **********, ***** ***** performed *****, **** **** ********* *** noise, ******** ****** ********.

IPVM Image

Test ***********

** **** *******, ** ****** ***** on ******** ********:

  • ******* ******** (**** *********):**** * ******* ******* ****** *** scene, ** ******** ******* ****** ******* could ** ********* *** *** **** blur *** *********** **** *******.
  • ******* *********** (*********):**** *** ******* *********** *** *******, we ***** ******* **** *** *********.
  • ***** ********, ~*.* *** ~* ***:** ******* ***** ******** ***** * standard ******* ************ ***** ** * ~0.1 *** *****, ******** *** **** colors **** ******** ** ********, *** whether *** ******** **** ****** ***. We ******* ***** ******** ***** ** ~1 ***, ********* ** **** ******* as ** ** ****** *** **** models ******.
  • ****** *******, ~*.* *** ~* *** (low ******):*******, ** ****** ***** ******* ******* at ~* ***, **** *** ****** as ****** **** *** ********* **** not ****** *** ** *** **** of ********.

Manufacturer ******** ******

******* **** ********* ****** ***** ******* settings *** ******** **** ** ************* so ******** ***** ** ******* ** improved. ** *** *****, ***** ******** changes ********* ********* ** ******** ******** digital ***** ********* ** ****** ****. However, *** ******** ** ******* ***** offset *** ******* ****, ********** *********** similar ******* ** *** *****.

Test **********

*** ****** *** ********* ***** ** testing **** * *** *********** ** standardize ******** **** **:

  • ******** *** *** ** * ******* of */*** / ****
  • *********** *** ************ ** ************ ~**-**
  • *** ********/***

Firmware ******** ****

*** ********* ******** ******** **** **** in **** ****:

  • ******** **-*****-*: *.**.*.***
  • ******** *.*-***-***-**: *.**.*.**
  • **** *****-*: *.**.*
  • **** *****-*: *.**.*.*
  • **** *****: *.**.*
  • **** *****-**: *.**.*.*
  • **** *****-***: *.**.*
  • **** *****-***: *.**.*
  • ***** ***-****-*: *.**.****
  • ***** ***-*****-***: *.**.****
  • ***** ***-********: *.***.*******.*.*
  • ***** *******: *.***.*******.*.*
  • ***** ***-*******-**: *.***.*******.*.*
  • ****** ***-*****: *.**.**
  • ****** ***-*****: *.**.**
  • ********* **-*********-*: **.*.**
  • ********* **-***********-***: **.*.**
  • ********* **-*********-****: **.*.**
  • ********* **-*********-****: **.*.*
  • ********* **-******: *.**
  • **** ***-******: *.*.*
  • ***** *****: *.*.*.*
  • ******* ******-****: *****
  • ******* ******-****: *****

Comments (66)

Derek, Ethan, good work!

As I mentioned in Hikvision ColorVu is Smart Marketing, these test results show that integrated white light illumination, while technologically not complex, provides value in very dark scenarios even against top-performing expensive low-light cameras.

Agree: 7
Disagree
Informative
Unhelpful
Funny

Great report guys!

Agree: 4
Disagree
Informative
Unhelpful
Funny

This is what makes IPVM useful and worth the subscription price!

Agree: 13
Disagree
Informative
Unhelpful
Funny

Great stuff! This is where you guys are at your best! Who needs a data sheet or specification? Take me to Missouri and show me!!

Agree: 10
Disagree: 1
Informative
Unhelpful
Funny: 1

I only disagree with the Mizzery part 😉

Agree: 2
Disagree: 2
Informative
Unhelpful
Funny

Sorry...take me to Kansas and, "to the stars through difficulties!"

(Just doesn't have the same ring as "show me" but it's beautiful heartland America for both)

Agree
Disagree
Informative
Unhelpful
Funny: 1

Good to see you do motion scenes. To complete the picture you also should provide produced bitrate values, IMHO.

And what does it mean "default settings"? Default settings where? Which software did you use to get video form cameras? Just a web page? Or some manufacture software?

Agree
Disagree
Informative: 1
Unhelpful
Funny

Default settings means default for each manufacturer's camera. We mainly only adjusted exposure to make sure nothing would use slow shutter, so shutter was a max of 1/30s / 33ms.

We used Exacq for recording on this test, as we do for most, because it's highly compatible and doesn't interact much with cameras' settings as some (Genetec/Milestone) are more prone to do.

Agree
Disagree
Informative
Unhelpful
Funny

Ethan,

I do not know about exacq but for example, our software at least sets the quality/bitrate for the camera before it pulls the stream out. I assume exacq does the same. And it might be different bitrate for different codecs manufactures etc( it's up to exacq internal logic). So unless you pay attention that exacq does not change settings differently for every camera ( which you probably do) your test is about how cameras work well with exacq specifically( not in general).

I'm very impressed with the work you have done and only say for the sake of constructive improvement.

Agree: 1
Disagree
Informative
Unhelpful: 1
Funny

It's a fair concern and it's one of the reasons we use Exacq. It generally does not change camera settings when adding cameras. There are some exceptions, which we've seen throughout years of testing with it. The only time it changes bitrate or compression settings is if you manually change them in the client.

All of that aside, we also double check settings and re-check codec and other settings after adding to the VMS to confirm.

Related question: If I turn off the Nx Witness "automatically optimize settings" option, does it still make changes to the stream? In our tests it didn't seem so but curious to get a definitive answer.

Agree
Disagree
Informative
Unhelpful
Funny

if the option is off NX does not change a single thing on the camera.

Agree
Disagree
Informative
Unhelpful
Funny

I am glad members like this! Keep in mind we can only do some many of these reports even with a 12,000sqft test lab, even with a team of a half dozen people testing since it takes months to put this all together.

In other words, next time someone complains that IPVM writes too much about China or business, keep in mind those China or business posts take a few hours (or days) to write, this one takes a few months.

Agree: 13
Disagree
Informative: 12
Unhelpful
Funny: 1

Understood. But for many of us this is the primary reason we subscribe.

Agree: 6
Disagree
Informative
Unhelpful
Funny

Jamie, I am sure 'many' people subscribe just for tests but when we look at what people read, people read far more on the business reporting so it seems that 'many' care more about the business reporting.

To be clear, I am not favoring one or the other, we are firmly committed to both, I am trying to have members understand the dynamics involved - the massive cost to doing such tests and the much lower read levels for them. All I ask is that when we do business reports, try to keep this in mind if you are unhappy or uninterested with those reports.

Agree: 3
Disagree: 1
Informative: 3
Unhelpful
Funny

Would it be worth considering in future, these type of side by side test results were released without first revealing the manufacturers?

The after test report discussion maybe less biased towards any manufacturer.

Once there is a good agreement on the test results, the manufacturer of each tested camera revealed. :)

My 2 cent worth.

Agree
Disagree
Informative
Unhelpful
Funny

these type of side by side test results were released without first revealing the manufacturers?

What would people do without revealing the manufacturers? Speculate that the manufacturers they use are the best and the ones they don't are the worse? Serious question.

Agree
Disagree
Informative
Unhelpful
Funny

Hi John,

I am not saying not to release the manufacturers of each tested camera. I was just suggesting maybe a delay to release.

The spectrum of the selected cameras are arranged to be tested in the same use case scenarios but the sensors used in each camera are of different class. The newer and better gen normally outperform the earlier gen.

Any manufacturer who has an account here will come in and try to defend their own product (inside the tested list). Believe you can see from the ongoing discussion here.

If the shortlisted cameras were to pit against each other in the shootout, they should be compared in the same sensor class or budgetary type.

Agree
Disagree
Informative
Unhelpful
Funny

they should be compared in the same sensor class or budgetary type.

We've talked about sensor class before and the practical problem is that almost all manufacturers hide this information. Even if we could find it out, separating shootouts into sensor class would not make sense for most users as most want the best image quality, not a specific sensor class.

The newer and better gen normally outperform the earlier gen.

Sure, but it is what it is. What point would it be to say that, e.g., Sony is the best performer among those using 2015 sensors? You may care but I don't think most buyers are interested in that as much as sheer performance.

As for budgetary type, we list the prices prominently in the report even in the ranking itself. Price is clearly a factor and users can contrast the price vs the performance.

Agree
Disagree
Informative
Unhelpful
Funny

Was shutter speed the same for all cameras or different?

Agree
Disagree
Informative
Unhelpful
Funny

Shutter speed was set to a max of 1/30s or 33ms, depending on how the camera listed it in its UI. No cameras were allowed to use slow shutter.

Agree
Disagree: 1
Informative: 9
Unhelpful
Funny

"Second, we tested motion with the subject approaching the camera. Performance varied much more in this scene, with some clearly identifying the subject and others blurring or overexposing."

What was the min distance between the camera and the walking person?

Thanks

Agree
Disagree
Informative
Unhelpful
Funny

The subject walked from ~50' to 0, directly past the cameras. I'd say within a few feet of the cameras. We have seen super low light overexposure at times in past tests and called it out, but we haven't looked at it as thoroughly as we do here.

Agree
Disagree
Informative
Unhelpful
Funny

Excellent Report

Keep it up Ethan, Great Job

could you do the same report on the same cameras for say 100' from moon lite to no moon conditions and looking at conditions like , parking lots vs street lites

or even driveways with no lite , @ say 100 to 400' conditions

Harsh weather , vs cold weather

Raining @ nite same conditions and Fog areas

More concerned with lots, parking, Entrys, Streets and conditions that surround them

Real life situations which you would run into

nites with Car head lites or Flashlites in the dark areas , or alleys with not lite with types of lite effecting the response of the cameras

Types of lites flashing towards the cameras or under vs over the camera lites

Many of the Reps at the shows show fantastic cameras in Perfect settings , But when you put them out in real world conditions or settings outside perfect , they don't always perform .

This would really be of value to the industry Techs, and reps

Agree
Disagree
Informative
Unhelpful
Funny: 1

could you do the same report on the same cameras for say 100' from moon lite to no moon conditions and looking at conditions like , parking lots vs street lites

We did our tests indoors so we could control the lighting. Doing outdoor tests risks shifting light levels that could bias the test.

We can do more to explain how this maps to parking lots or street lights, etc. but we want to be careful that the tests are normalized and fair across the board.

Agree
Disagree
Informative: 3
Unhelpful
Funny

That's a pretty extensive list of conditions there. I marked funny because at first I thought it was satire. Now I'm not so sure. 😒

Agree: 1
Disagree
Informative
Unhelpful
Funny: 1

Its such a pity that Hik delivers an excellent product in terms of performance and value. Lucky for Axis and Avigilon this pity exists.

Agree
Disagree: 1
Informative
Unhelpful
Funny

If Axis and Avigilon had unlimited government funds they could do the same.

Agree: 3
Disagree: 2
Informative
Unhelpful
Funny: 2

They still remain lucky.

Agree
Disagree: 2
Informative
Unhelpful
Funny

What, if any, factor does resolution play in these outcomes? For instance, since light sensitivity per pixel doesn't seem to apply as it used to in the past, wouldn't one expect the 4K ULL model to outperform the 2MP ULL equivalent?

For instance, the Hikvision 2MP DS-2CD2725FHWD-IZS vs the Hikvision DS-2CD2347G1-L comparison. Since all cameras are rated as Darkfighter, wouldn't you naturally expect the 4MP version to be better at static images? (I'm not trying to account for motion here, for arguments sake)

My point is, wouldn't it be more fair to compare all of the cameras at a standardized resolution at some point? That would negate the pixel superiority, just as you standardize on shutter. Just a suggestion.

Wonderful test non the less. Home run guys!

Agree
Disagree
Informative
Unhelpful
Funny

Sensitivity definitely still matters, just less than it did in the past. There is still a sensitivity difference if you look at the same manufacturer's specs, most often. Or, in the case of the Hikvision Smart Series Darkfighters (5526 and 5546), they are both spec'd at the same minimum illumination (0.002 lux), but there is a noticeable, but not major, difference.

From a practical standpoint, yes, we could test at a standard resolution, but it's not very practically useful, since people are generally buying the resolution they need. An academic test of standardized resolution which doesn't show potential differences due to higher/lower pixel counts would not translate well into real world performance. Do you agree?

Agree
Disagree
Informative
Unhelpful: 1
Funny

I guess the real issue is that you guys can’t realistically test every single model of every brand for an exhaustive test. For example, my curiosity asks which of the Hikvision DS-2CD21XXFWD-I models has the best low light image quality. What is the actual difference between the 2MP, 4MP, and 8MP versions? How much, if any, quality do I loose in dark scenes with the 8MP vs the 2MP. I know what I gain in well lit scenes. And I’m not expecting IPVM to test to this level because as soon as you do this for one brand, you set yourself up to do it for all brands. Instead you ask for input for certain models and democratically choose a bunch of units from varied brands. However you will never satisfy all because you didn’t pick “their model”. It’s classical damned if you do or don’t.

I am glad you did these tests, but it just makes me want more. 🤣

Agree: 1
Disagree
Informative
Unhelpful
Funny

That's a good explanation! We actually have talked about doing bigger shootouts within a manufacturer's line. So in the Hikvision example, shooting out Value Express vs. Value vs. Performance vs. Smart, various resolutions, multiple form factors. I think it would be too large to add into this test alone but we are asked fairly frequently how camera A compares to camera B from a certain manufacturer. It's gotten especially muddy as the Axis P series has improved to be closer to Q and Hik Performance Series is more competitive with Smart. We'll keep it in mind for future tests.

Agree: 1
Disagree
Informative
Unhelpful
Funny

I would love to see pixel size and F number of lens to see if (for the same type of sensor) that would take some of the mystery out of how cameras from the same manufacturer can appear at both the top and bottom of the list. Maybe there is a straight forward calculation off the data-sheets that could help camera selection.

Agree
Disagree
Informative
Unhelpful
Funny

We can certainly gather imager size and lens F stop. I don't think there is a straight forward calculation, though. I do think that in general a bigger imager and lower F-stop compared to a smaller imager and higher F-stop in the same manufacturer will have an obvious difference.

The Hanwha ExtraLux, for example, is a 1/2" sensor with a very low 0.94 F-stop, while the standard Wisenet X 6080 cameras are 1/2.8" with a more typical F1.4 lens. ExtraLux has a clear advantage in our tests in low light.

It's not this simple across manufacturers, though. A larger imager and lower F-stop definitely helps, but it's not a guarantee that one manufacturer will be better than another.

Agree
Disagree
Informative: 2
Unhelpful
Funny

One of the best lab analysis reports on cameras, specifically, low ligth, color fidelity, aproaching, moving, cost, excellent, overall performance, congrats, cheers,

Agree: 4
Disagree
Informative
Unhelpful
Funny

Great report guys - very informative and you've covered many important aspects of low light performance testing.

As the mobile sector of the market now represents a significant portion, it would be great to see a test of all the latest cameras that are rail rated (particularly to the EN50155 standards). Most of the leading manufactures have these in both dome or wedge form factor. I know what Im suggesting below means a combination of dynamic testing on an actual vehicle, or a complex simulation in your lab, but it could include test parameters such as:

- scene hot spot management inside a crowded public transit vehicle

- rapid light changes from sunlight to darkness, inside the same vehicle as it enters/exits tunnels

- how these cameras perform as dash-cams on a train or a bus (rendering traffic signal colours accurately is always a problem in poor ambient lighting)

- motion blur on track/roadside objects at various vehicle and shutter speeds

- managing rapidly flashing indicator lights and oncoming headlights against dark backgrounds

From experience, I know its not an easy task to undertake and there may be other priorities for IPVM right now. Having said that, a lot of the dash-cam road tests can be done from a van or car (of course obtaining permission to carry out tests on moving trains is much more difficult). However, the results from roads tests will give a good indicator of how the camera will also perform on a track.

I think if anyone can do this type of test thoroughly without bias - its you guys.

Keep up the good work.

Agree: 1
Disagree
Informative
Unhelpful
Funny

Have you tried conducting these tests under different Light Sources? Fluorescent is still popular, Mercury and Sodium Vapor. I would think you will get different results under the different sources.

Agree: 1
Disagree
Informative: 2
Unhelpful
Funny

We did not, but it's on our list of things to do. All the lighting in this test is LED, actually. We have had plans to purchase a few different lighting sources for lab testing, but haven't pulled the trigger yet. If you have ideas on that, shoot me an email at ethan@ipvm.com

Agree
Disagree
Informative
Unhelpful
Funny

Ethan, Thanks for your reply. I have worked in Video Testing Labs for years and have many great ideas for you. As an ASIS Member for almost 25 years I am dedicated to the industry and do not mind helping anyone to improve for the better of this industry. Just a few points and I do apologize for not reading all the comments as I am sure I am duplicating many points that were already brought up however I will list a few tidbits below:

1- Are cameras being tested out of box at Factory default?

2- Do you have spare cameras in case there is a weird result that needs to be verified?

3- Are light tests being conducted using cameras at widest lens setting?

4- Are Camera/Views framed for exact scene from camera to camera?

5- Are all Cameras being tested in same position or as close to being side by side as possible?

6- Are all Cameras tested at same Lux levels? Do you use a light control board?

7- LEDs typically do not emit IR which will result in greater differences when cameras change from Day to Night mode.

8- Different LED L.E.D. Temperatures can produce different results.

9- Testing with Sodium and Mercury may show you how well the cameras are color balanced.

10- Incandescent (Old but still available, Flourescent Low and High Temps and good old sunlight should also be used.) I will think you will be surprised at some of the results you will find.

11- For Sunlight you can use electric window shades and conduct WDR, BLC testing as well.

12- Do you test for Aliasing, Barrel roll and Complete 100% Motion across all Pixels?

Just a couple of tidbits to help improve your testing.

Thanks and have a great day.

Agree
Disagree
Informative: 1
Unhelpful
Funny

Thanks for the comment, some answers: We don't test cameras using purely default settings. Some settings are changed across the board:

  • Frame rate is set to 15 FPS
  • Compression is standardized to ~28-30 quantization
  • Exposure is set to a maximum of 1/30s or 33ms, depending on how the camera is configured
  • Slow shutter is disabled, if enabled (which is uncommon in 2019)

On these shootouts, manufacturers are shown preliminary images and given opportunity to suggest settings changes. However, in this cases, none of these adjustments really improved practical detail. Most suggestions amounted to decreasing noise reduction, which decreased some blur, but the added noise reduced details by practically the same amount.

As far as different light sources, we have tested mercury vapor, LED, sodium vapor, and others. See Camera Color Fidelity Shootout. We did not do extensive testing at multiple light levels in that test. However, while I think that there will be some color fidelity differences in brighter low light scenes under other lights, maybe 2 lux and up, in darker scenes below 1 lux, low light looks very similar. We have tested with different LED temperatures and seen this. Effectively, dark is dark, and at that point colors are poor for the vast majority of cameras.

Agree
Disagree
Informative: 1
Unhelpful
Funny

Thank you Sir!!

Agree
Disagree
Informative
Unhelpful
Funny

Great report guys. Specifically in the section noted as Newer Vs. Older Camera Models, since you have newer models from most of the manufacturers why are you comparing a newly released camera to a model that is 3 years old, instead of similar models? Seems odd since electronics always improve with time, chipsets, technology. I'd expect a new Axis model to outperform a 3 year old Panasonic or 2 year old Hanwha. Were all cameras viewed through a VMS (which) or direct through the browser? Did that make a difference?

Agree
Disagree
Informative
Unhelpful
Funny

I also would like to know how images were received(wich software was used).

Agree: 2
Disagree
Informative
Unhelpful
Funny

you have newer models from most of the manufacturers why are you comparing a newly released camera to a model that is 3 years old

If there is a specific model that you think we should have tested, please let us know. For some manufacturers, an "older" model is the highest spec'd available product for that company. Which models from those manufacturers that we did not test should we?

Agree
Disagree
Informative
Unhelpful
Funny

in a word, illuminating...

Agree
Disagree
Informative
Unhelpful
Funny: 4

You have no idea how much I hate myself for laughing at that.

Agree
Disagree
Informative
Unhelpful
Funny: 3

I would like to see more categories for the test and more variety in the categories

Say add Digital Watchdog

or IP Vs Analog HD

or 1.3,3,5 mpxl vs 12 or 20 mpxl with 4 1.3 lens camera in cam.

Variety of manufacturers

instead of the top 5 , how about a variety of others

Agree: 1
Disagree
Informative
Unhelpful
Funny

instead of the top 5 , how about a variety of others

agree.

they should double the number of mfrs, and have a wild-card entry (Speco??)

oops, spoke too soon.

This is the biggest color low light shootout ever, testing 20+ super low light models from 10 manufacturers...

Agree
Disagree
Informative
Unhelpful
Funny

Incredible that except for Axis both Hikvision and Dahua have a larger showing than any other manufacturer?

Agree: 1
Disagree
Informative
Unhelpful
Funny

Those 3 companies are bigger than all the others and have bigger R&D budgets and release cameras more frequently.

Even if Axis, Dahua, and Hikvision had no expertise, the sheer fact they release more frequently means they have an opportunity to use newer, better imagers and SoCs. Recall someone objected to us including 2 or 3-year-old cameras in the test. But that's those manufacturer's fault for not being committed enough to release more frequently.

Agree: 1
Disagree
Informative
Unhelpful
Funny

Could you share test images for all the cameras is all test scenarios?

Agree
Disagree
Informative
Unhelpful
Funny

We don't have that packaged but we are having a webinar tomorrow where we will answer questions and show videos and image that anyone wants to see:

Agree
Disagree
Informative
Unhelpful
Funny

For anyone interested, the webinar recording is here (~1 hour):

Agree
Disagree
Informative: 1
Unhelpful
Funny

Does anyone know how the Sony SNC-VB770 "ultra-high sensitivity camera" compares in very low light? I have a couple of them, and they are better than my other cameras, but very expensive. Can I now get similar performance to the VB770 with a better price?

Agree: 1
Disagree
Informative
Unhelpful
Funny

Based on our past tests, I do not think any of the cameras in this shootout are going to produce images as bright as the VB770 in very low light. For example, when we tested the VB770, we measured 0.01 lux at the point the subject is standing in this shot:

Now that lux level was measured facing the camera, there is obviously additional ambient light in the scene from other lights, but the lights you see on the right side of the FOV are actually from soccer fields ~1/2 mile away. There were no other lights on closer than that.

By contrast, here is one of the overall brightest images from our test, the Dahua N45EM63 in 0.01 measured inside.

There is very little ambient light, aside from a couple of cracks through doors and next to ceiling joists, so it's closer to a "true" 0.01 lux, but the image is obviously much darker, much noisier, and detection at longer range is difficult.

Even in the higher priced models which would theoretically compete more with the VB770, like the Q1798-LE, performance in very dark scenes is not really on the same level (full field of view below).

If your light level was a bit higher, something closer to 1 lux, you are much more likely to be able to get away with something lower cost, but if it is approaching 0, likely not.

Agree: 1
Disagree
Informative
Unhelpful
Funny

Thanks for the quick reply. This helps a lot.

Agree
Disagree
Informative
Unhelpful
Funny

I'm quite disappointed in the Panasonic WV-S2531Ls performance. Some in my company move a lot of this model, on the order of millions of dollars per year. Panasonic has always touted that they have the best optics but acknowledge weakness in other categories. At this point the poor results on a relatively new camera aren't backing that statement up.

I expected the Avigilon H5A to perform mediocre. It seems like a slight processor bump over the H4A.

Agree
Disagree
Informative: 1
Unhelpful
Funny: 1

Good Morning

Some camera's in this test have onboard IR lighting. This was turned off constantly during this test i presume?

Agree
Disagree
Informative
Unhelpful
Funny

IR was turned off during testing on all cameras.

Agree
Disagree
Informative
Unhelpful
Funny

Ethan, I think your test included cameras from 9 manufacturers. Do you have a sense of which software had the best and easiest to use GUI? Not a real test but an educated preference?

Agree
Disagree
Informative
Unhelpful
Funny

Did you ever receive and test the Uniview Starlight? - Additionally, we have been working on getting Uniview Super Starlight models for more than a month, they are supposed to have shipped and we will add them to this report when we receive them.

Especially the IPC3635ER3-DUPZ has my attention ;-)

Agree
Disagree
Informative
Unhelpful
Funny

any update on the IPC3635ER3-DUPZ

Agree
Disagree
Informative
Unhelpful
Funny

Just curious to know what IPVM does with all the cameras after the test. Do you keep them around for further tests or do you re-sell them? Looks like not only a time-intensive test but an expensive one, too. Well done.

Agree
Disagree
Informative
Unhelpful
Funny

Do you keep them around for further tests or do you re-sell them?

We keep them around and often do use them for further / future tests. I don't think we ever re-sold any camera and have hundreds of camera models from many years collected.

Agree
Disagree
Informative
Unhelpful
Funny

That's an inventory I'd like to see! Must be some pretty eclectic stuff in there.

Agree
Disagree
Informative
Unhelpful
Funny

Here's a video we did a while back that shows some of our inventory and setup:

Agree
Disagree
Informative
Unhelpful
Funny
Subscribe to IPVM Research to read the full report.
Why do I need to subscribe?
The IPVM Research Service includes products tests and shootouts plus competitive and financial analysis, helping decision-makers better evaluate purchasing, partnering, developing, and/or competing against companies in physical security.
Already have an account?
Loading Related Reports