Ranking IP Camera Low Light Performance

For clarity, I would just like to point out that the ACTi D11 is our most economical Day Only unit.

I would still be interested in a comparison of cameras that are marketed and sold as "Low Light" units.

Given the scale of this report, we're sure there are going to be questions and comments. Leave them in the comments and we'll be happy to address them.

John, you stated,

"We ensure all cameras are set at a normalized 1/30s shutter speed (eliminating sens up tricks and ghosting/slow shutter problems)."

I would be interested in seeing test results of the same cameras but with moving objects in the video, primarily a moving "person". Perhaps with the top 5 cameras you listed in the results.

Thanks!

Most useful article! Thanks. Wonder if a similar daytime test array would help as well.

In my opinion

I would group cameras with the same LUX level ( specify by manufactures ) and then do test

Very interesting report.

Something I noticed by chance: is there an error with the number for the Axis Q1604? In the tables it shows as 350 TVL but the image captions indicate 400 LL-TVL. Which is correct or have I just misunderstood?

Great testing, was just sad to see that brands such as Dahua, Hikvision & ACTi were tested but not Vivotek.

Would've loved to see what the IP8151, IP8162 and IP8173 would score fail or no fail.

Just a little note. The TVL value represents vertical resolution and should be tested on patterns with horizontal lines. These results are valid only if all the cameras have exactly square cells.

Or am I wrong?

Great Article. Would like to throw in some other cameras for the next test!

It might be useful to add a column for street pricing for these cameras and maybe form factor. The price range of the cameras on the list appears to be from just over $100 to a couple of thousand dollars. Most of us aren't familiar with all of the different models listed and it would be good to see the relationship between price and performance.

First of all thanks for an interesting test!

Let me also make clear that I’m an integrator and we don’t use ACTi cameras (and I’m not native English speaking).

I think that John has been unreasonable hard in his reply to Dennis comment. As I read Dennis comment it was not a general criticism of the test nor an attempt to hijack the comments, he just pointed out that it’s unfortunate that a day-only camera is used in a low-light test, which I totally agree on. The ACTi D11 (I had to look up the datasheet) is clearly not meant for night/low-light applications. This is like comparing a race car to a limousine on a race track. The only thing you get out of it is that you make the limousine look bad.

Dennis also asks where you get your cameras, which is actually a very good question. If you get samples/demos from vendors for shoots outs, I would highly doubt the performance of a camera you buy would in general match the one you get for the shoots out. The performance of the camera very much depends on the sensor used. The sensor performance depends on multiple factors like for example where on the wafer the sensor was cut out. So when you ask for a camera you might get a camera that performs especially well. In my previous job I worked at a machine vision camera manufacture and I know that for some camera models the performance of each sensor and each camera is measured. It’s then very easy to find a camera that performs especially well.

About the test, it would be nice to have a gray scale on the test images as well as a person’s head is not black/white and you don’t know how the gamma correction in the camera is set by default. This could result in very good performance in light colours and bad performance in dark colours or vice versa. In general it would be nice to have an evenly distribution of bits over the dynamic range of the camera.

Keep up the good work!!

Hi, I think this is a good test! Thanks!

Not often you get to see so many cameras compared. Especially I like the photos you posted at the end. As a consultant/end user this is very interesting.

I assume all have default lenses, have you done a test of how lens choice would affect the result? How many different sensors are really in the test? Do you know which use the same or are all using different?

It must be faster to do this test than for example the slow shutter/ghosting test you did, right? I would like to have seen a motion aspect on the test above as well (the ghosting test is only cameras with default slow shutter, or have I misunderstood). But I understand that would take mcuh more time.

Other test of interest would be low light PTZ-cameras and no I dont know how to do it in any efficient way?

Hi Guys,

Good test. Would it be possible to add in colour at low light testing for the next batch as a few manufacturers are now making claims in this areas with Lightfinder and starlight technology to name a few. This may be an altogether different set of tests for True day/night external cameras.

It would also be interesting to measure the bandwwith of the cameras at day/night to report on network efficiency during night operations.

Thanks

We've been doing shoot outs of the Axis and Samsung Fixed domes as a lot of our low light installs require vandal resistance that the box cameras don't offer well. It would be nice to see the fixed domes added as that's what we use primarily. It would also be nice to see some 180s in the mix like the Arecont or Scallop imaging. We've been fairly impressed with the night vision of the Arecont 180s and 360s other than their brutal bandwidth consumption. We did a shoot out of the Axis P3364-VE and the Samsung SNV-5084 just yesterday and do an amazing job in nearly zero light. We forced the Samsung into colour mode and were shocked how well it did, though there was a bit of ghosting.

Has anyone out there used any equipment from ICRealtime? I have used some, and it seems to be adequate in those particular applications. But, I am swinging more into IP, plus they have some lower cost NVRs with POE. I have some potential client applications for low light and IR assist. If you have used ICRealtime cameras, especially IP cameras in low or zero light situations, what has been your experience as compared to other cameras like in the above list?

We've updated the charts to include the manufacturer's specified lux levels.

This is a pretty darn good test, but I'm confused about some of the results. I agree with most of them, to within 50 TVL. However, to me the following are way off (I did download the zip files) :

Too Generous:
ACTI KCM5111 Nothing visible!
AXIS Q1755 Nothing visible!
SONY SNC-CH240 Very marginal visibility
VIGILON (Both) Barely visible

Too Stingy:
DVTel 4221 I could easily justify a 600, puting this one in the lead. Rated at 400.

Am I missing something on the rating criteria?

It seems like the DVTel 4221 got a harsh treatment compared to several of the other readings. From the posted pictures it seems to me like 450 is ok.

Could you please explain a bit on how the readings are done?

Disclaimer: We don't use DVTel, but do use several of the other manufacturers.

Hello, I'm looking at the Panasonic SP509 specification and it is mentioned

High sensitivity with Day & Night (IR) function:0.5 lx (Color), 0.06 lx (B/W) at F1.4

There is something doesn't make sense here?

Thanks

John, what lens and f-stop where used for the tests? Am I correct in assuming that the .5 lux was measured at the chart?

What f-stop was used? Where they all they same? If not, that could sway the results.

Regards

Great article!

Hats off to you guys for taking aim at this one. There is definitely some information to be extracted from those stills!

However the testing misses the mark considerably. I'm not tryin to rumple anyones feathers but y'all said nobody's done this kind of test before yatta, yatta, yatta. And maybe that true for cctv, but in digital videography in general that horse has left the barn a time ago.

Maybe take a look-see at the ANSI CEA-639 standard for Video Camera Low-Light Performance. You might notice that they consider resolution only one of 5 contributing factors. The basic problem is that the way you did your test, is you are ignoring the contrast level performance, which in normal illumination is not nearly as important as in low light settings.

Here's a tip-off that something done run amuck: When you can't even makeout the chart ledger values on the left side, yet you are still 'seeing' resolution on the same line on the right, then the chart is not being used correctly. It's plum crazy if you think about it.

Take a look at your Samsung snb-6004 image, I can't make out the numbers can you?

Yet this camera you rank in the top third of all tested! Maybe the resolution is greater than most, but the contrast is abysmal and makes the picture one of the worst not the best. Just one mans opinion :)

What was the video capturing methods of the images?

I think the bit-rates of the camera can influnce on the image quality.