Subscriber Discussion

What Are The Different Methods Of Measurements That Manufacturer Use To Calculate WDR

UM
Undisclosed Manufacturer #1
Apr 03, 2015

What are the different methods of measurements that manufacturer use to calculate

JH
John Honovich
Apr 03, 2015
IPVM

Manufacturers rarely disclose how or what they do to measure WDR.

The only one I have seen with a detailed explanation is the now shuttered Pixim, see Pixim's WDR whitepaper.

Even if every manufacturer disclosed the method of measurement, you still have a risk of reporting error / bias, since they do it themselves.

U
Undisclosed #2
Apr 03, 2015
IPVMU Certified

Even if every manufacturer disclosed the method of measurement, you still have a risk of reporting error / bias, since they do it themselves.

But if they did disclose it, it would be a huge improvement, no?

Manufacturers don't typically lie about about the weight of a camera, whether they report ounces, grams or stones. But if you do not know the UOM you are SOL.

(1)
JH
John Honovich
Apr 03, 2015
IPVM

The measurement process for weights (i.e., a scale) is a lot easier than the process for WDR. Yes/no?

It's easy to catch a lie in weight, not as easy for WDR or minimum illumination.

That said, it would be an improvement if they disclosed. I am not interested in debating the amount of improvement because it is a hypothetical anyway.

(2)
U
Undisclosed #3
Apr 03, 2015

But if you do not know the UOM you are SOL.

Perhaps, but do YOU have any way to measure the dynamic range of a scene you're specifying a camera for?

How do you know if you need 90db or 120db?

If a manufacturer told you their camera had 110db of dynamic range, could you prove or disprove it in the field?

My point being, it's absurd to get overly hung up on a measurement that is almost impossible for you to validate.

(2)
U
Undisclosed #2
Apr 03, 2015
IPVMU Certified

Perhaps, but do YOU have any way to measure the dynamic range of a scene you're specifying a camera for?

Yes and so do you.

Take a picture, with a high-end DSLR.

Adjust the exposure until there is no part of the scene that is over-exposed or under-exposed.

The difference in intensity between the darkest and lightest pixels, is the dynamic range of the scene.

(1)
(1)
U
Undisclosed #3
Apr 03, 2015

So...

A) everyone has a "high-end" DSLR?

B) The DSLR has perfect dynamic range and thus yields accurate results?

(1)
U
Undisclosed #2
Apr 03, 2015
IPVMU Certified

A) it needs to be better than the security cameras you would specify.

B) if the scene can not be captured by the DSLR without under-exposing or over-exposing then the security camera can't either. See A)

(1)
U
Undisclosed #3
Apr 03, 2015

A) it needs to be better than the security cameras you would specify.

Right. So now you need to know an accurate WDR rating for your DSLR *and* for the cameras you're specifying?

B) if the scene can not be captured by the DSLR without under-exposing or over-exposing then the security camera can't either.

You're basing this on the presumption that the scene you capture with the DSLR is a worst-case representation and that the DSLR in its entirety (lens, sensor, ISO settings) maps to something directly on the CCTV camera.

Rather than go around on this, here's the thing. You can't accurately measure the dynamic range of a scene with one device and know for a fact how a given CCTV camera with a given spec rating will perform in that same scene. You also can't reasonably verify or dispute the spec rating on a camera.

Dynamic range numbers are guidance at best, much like lux level ratings. They can be used to give you some idea of how a camera is expected to perform in a given environment but they're still very generic.

What happens when the range is exceeded? Does your camera push the exposure towards the shadows? Towards the highlights? Does it just average to the middle? Are any of these compromises acceptable in the scene and therefore negate the issue? Is the dynamic range the same in color and B/W? Does your scene meet or exceed specs all the time, or only during specific times?

The dynamic range rating of a camera is a summation of several factors when you get right down to it. It's very possible that a camera with a 100db range will perform better in a scene than one with a 110db range, depending on elements unique to a scene. It's also possible the image sensor accurately captures a given amount of range, but then it's lost when you get into the compression stage, depending on camera settings. Or you might only get the full dynamic range stated with some impractical image/compression settings (much like lux ratings).

U
Undisclosed #2
Apr 03, 2015
IPVMU Certified

Right. So now you need to know an accurate WDR rating for your DSLR *and* for the cameras you're specifying?

No. If you are just trying to measure the DR of the scene (like you first asked), then you need a camera that can capture it while it does not under or over expose the image.

Dynamic range numbers are guidance at best, much like lux level ratings. They can be used to give you some idea of how a camera is expected to perform in a given environment but they're still very generic.

No. At best they are one unknown better than lux. In practice though, because of the measurement bs, they are the same. Lux ratings have the added fudge factor of 'whatever a mfr considers a usable image' that makes them totally inscrutable.

Here would be my simple solution:

A snapshot of a standardized (ISO) scene(s) would be included with the cameras spec.

Dream on, right?

U
Undisclosed #3
Apr 03, 2015

No. If you are just trying to measure the DR of the scene (like you first asked), then you need a camera that can capture it while it does not under or over expose the image.

And how would you know if your camera can capture the dynamic range of the scene without under or over exposure? IMO by knowing the DR rating for the camera.

The DSLR can only tell you dynamic range of a scene within the limits of its own capabilities. If your DSLR has 90db of dynamic range, but the scene has 120db the camera is going to tell you it's 90db. If you did not know the cameras capabilites you would take 90db as an accurate reading without realizing it is the max reading for the camera.

A DSLR is not built to give an accurate dynamic range number, granted it may not be wildly inaccurate, but I'm hesitant to rely on the output of an instrument not built for the purpose it is being used.

The DSLR is going to tell you the dynamic range of the image it captured *not* of the scene itself. You'd need to spend a little bit of time metering individual areas of the scene and manually calculating EV's to get something closer to the dynamic range of the actual scene.

DSLR's also have IR cut filters, like a CCTV camera except they are not designed to be movable. You'd also need to know the cutoff values of the IR cutfilter in your DSLR vs. your CCTV camera if you *really* want to get accurate results. Otherwise the DSLR or the CCTV camera may actually be exposed to difference max light wavelengths, which increases the net dynamic range of the image they are trying to adjust exposure to. And of course it's possible your most dynamic ranged scene is at night (headlights), in which case the DSLR is likely to give you massively underrated values compared to what the CCTV camera would see.

On top of all this, it's extremely unlikely that you are capturing the worst (or best?) shot of the scene when you take your image. If you go out and shoot a test shot in the lobby of a car dealership today, in April, do you think the dynamic range will be the same in July? (I doubt it).

But let's say you have a good DSLR and it's rated for 120db of dynamic range, and you know how to use it appropriately so you're capturing an image with exposure settings to maximize DR for your scene and the DSLR tells you that you have 90db of dynamic range in the image. You select a CCTV camera rated for 100db (plenty of cushion), install it the next day and get blowout in the image. What happens when you call up the camera manufacturer and tell them you measured 90db and their 100db rated camera isn't exposing as you'd expect?

A snapshot of a standardized (ISO) scene(s) would be included with the cameras spec.

A spec that is delivered as a PDF (typically)? A spec with some amount of compression applied? You've just lost any true ability to utilize this scene reference unless you can also get everyone to agreed and a standardized spec sheet encoding format.

No. At best they are one unknown better than lux.

One unknown better, but impossible to prove/disprove, and dependant on lots of other things. "Guidance", IMO.

Taking an image with a DSLR can give you some amount of information about the scene at the point in time you snapped the image, relative to the camera's settings. But it will NOT be 100% conclusive data that you can use to pick the right/best/optimal dynamic range camera spec. It *might* let you eliminate a couple of really limited models, which if you're that concerned about dynamic range you probably had already eliminated anyway.

(1)
(1)
U
Undisclosed #2
Apr 03, 2015
IPVMU Certified

The DSLR can only tell you dynamic range of a scene within the limits of its own capabilities. If your DSLR has 90db of dynamic range, but the scene has 120db the camera is going to tell you it's 90db. If you did not know the cameras capabilites you would take 90db as an accurate reading without realizing it is the max reading for the camera.

Nonsense.  I do not know the capabilities of this camera. I would not take the camera max. reading as the range. It is overexposed, so it does not capture the scene.

Yes or No?

MI
Matt Ion
Apr 06, 2015

Nonsense. I do not know the capabilities of this camera. I would not take the camera max. reading as the range. It is overexposed, so it does not capture the scene.

Yes or No?

Poor example when talking about dynamic range, since the entire picture is clearly over-exposed, including the darkest area (the man's jacket). The total exposure could easily be reduced by at least a stop, possibly two, without losing detail in the jacket.

Edit: I feel out of place not being an "undisclosed" in this one...

U
Undisclosed #2
Apr 06, 2015
IPVMU Certified

...the entire picture is clearly over-exposed...

That was the point I was making actually, that one can tell from the image alone that the full dynamic range of scene has not been captured. Of course this is an extreme example, in other cases one would have to rely on a histogram like in the images a couple posts down...

We'd all know it's you even if you were undisclosed... ;)

U
Undisclosed #3
Apr 03, 2015

Sometimes you don't even test it in a finished camera, you just go by what the sensor/ISP OEM tells you.

U
Undisclosed #3
Apr 03, 2015

BTW, to answer the OP's question the basic concept is that you have a lightsource with a known or calibrated output (lumens at the lamp, focused to a given area giving you lux...).

On top of this light source you have a set of grayscale bars that go from allowing 100% of the light through, to 1% of the light. You point the camera at the lightbox with the grayscale transparency and you take an image. Then you look at the resultant image and look for the last bar that is not over exposed, the last bar that is not underexposed, take the delta between them and this is the range of light levels the camera can accurately capture at once. Convert that to db's or EV's and that is your dynamic range.

Like lux ratings though, you won't usually know how that was calculated. Your best ratings are going to come from having minimal compression. If the camera supports h.264 and JPEG encoding, the best numbers might come from a JPEG stream with no compression applied. But that same camera might also tout its bandwidth savings, which are likely to be based on h.264 and a moderate compression level. That means you'll never get the rated DR value in a typical deployment.

U
Undisclosed #2
Apr 03, 2015
IPVMU Certified

On top of this light source you have a set of grayscale bars that go from allowing 100% of the light through, to 1% of the light. You point the camera at the lightbox with the grayscale transparency and you take an image. Then you look at the resultant image and look for the last bar that is not over exposed, the last bar that is not underexposed, take the delta between them and this is the range of light levels the camera can accurately capture at once.

I agree with this 100%.

Question C, what delta would you actually be measuring if there were no overexposed or underexposed bars? For instance if the grayscale bars went from 10% to 60%?

You would be measuring the scene's dynamic range, not the camera's. But like you say the db calculation method varies wildly from mfr to mfr. And I agred thats the real problem. So, in case you have the impression that I think that dbs are comparable from mfr to mfr, I don't. But the delta, like you say, can be captured.

U
Undisclosed #3
Apr 03, 2015

The calibration scene is setup to have a dynamic range that exceeds any practical range of a camera.

Any camera you test is going to then tell you how good that camera is at resolving dynamic range, but it will NOT tell you the actual dynamic range of the scene itself. Different cameras will yield different results in the delta/dynamic range of the scene measured, even though they are all measuring the exact same thing.

If you're not careful with the measurement process you could interpret a reading of Xdb to mean the scene itself has a max of Xdb of dynamic range, which would be incorrect.

Using the output of a camera that is not designed to be a reference instrument to pick another camera for the given scene is therefore less than ideal.

If you have a deep understanding of the measurement camera AND the camera you want to substitute to monitor the scene on an ongoing basis you can make an educated guess about picking the optimal camera.

If you have a moderate understanding of the measurement camera, and nothing but a spec sheet for the other camera then you are in a very suboptimal situation.

For normal scenes none of this matters. It only comes in to play when the scene is going to be pushing the limits of a camera and selection criteria is critical. For me, I wouldn't try to map the output of one device to the expected output of an untested device when I'm trying to push a device to its max limits. That is my argument against your DSLR idea. When the decision matters most is when the data/specs can be least relied upon. And again, the scene you are measuring may not be representative of the highest range scenario, making the whole "test" somewhat pointless.

Unless you have deep knowledge of the DSLR, you do not know how it reacts to exceeding its limits. The average DSLR is built to try and capture as much information about the scene as possible, the raw data captured from the sensor goes through a processing circuit/DSP to try and squeeze as much out of it as possible. Your scene could have 110db of dynamic range, your DSLR could have 100db of dynamic range, and you could end up with a "beautiful" image with 90db of dynamic range, never realizing that your camera was able to throw away 20db of range without making it obvious to you.

I would wager that the average prosumer DSLR that is advertised to have 110db of dynamic range for the imager would produce a much better image from a scene with 110db of dynamic range than the average (or, above-average) CCTV camera that is advertised to have 110db of dynamic range, even with normalized settings (eg: both at 1/100s exposure, etc.).

DSLRs are NOT built to give you an accurate reading of the dynamic range of a given scene. They are built to try and give you the best overall image of a given scene, which will result in them running complex algorithms to adjust colorspace and exposure to that end. During the course of being involved with camera development, I've seen roughly what comes off an imager in the rawest format, and how the same data looks after being processed. The results are very different.

U
Undisclosed #2
Apr 03, 2015
IPVMU Certified

During the course of being involved with camera development, I've seen roughly what comes off an imager in the rawest format, and how the same data looks after being processed.

Agreed.  I shoot with a Nikon D3 in RAW mode and then run histograms in Adobe Lightroom using the RAW plugin.  What's your preference?

The RAW sensor data displayed on the histogram tells you whether the picture is underexposed or overexposed.

Look C, I'm not telling you that there is no more accurate way to capture the dynamic range delta, than with a DSLR, but, IMHO,  the differences in values would be small compared to the difference in stated dB numbers of typical cctv cameras. You know where Panasonic SDIII super dynamic range camera has 55dB and Hikvision 's latest has 120db, which is a factor of > 1000x.  

U
Undisclosed #3
Apr 04, 2015

I've been an avid photographer for over 25 years. Have had multiple shots published in magazines.

I shoot Canon for digital, had one of the first EOS 5D's in the US and have beta-tested other Canon bodies over the years.

For a while I was involved in a TV show that moved to shooting with Canon DSLR's for for all the primary footage.

IMO, the data you'd getting from shooting a scene with a DSLR would be no more helpful in making an informed decision than rolling a pair of dice. You're taking a reading from an instrument not designed for the purpose, in a scene that is most likely not representative of "worst case", and then mapping all that to yet another device with murky specs.

But you seem to be a strong proponet of this method, so you must have some reference images from sites where you've used this approach successfully? Rather than debate the theoretical I'd much rather see how you've used this to select one camera over another.

U
Undisclosed #2
Apr 04, 2015
IPVMU Certified

Oh, you're a Canon guy, that explains it! :)

Disclaimer: I don't consider myself a 'photographer'. Heck, I don't even like 'taking pictures' really. But I do love cameras! And sensors, and image manipulation and optics. Yeah, strange I know.

Anyway,

do YOU have any way to measure the dynamic range of a scene you're specifying a camera for?

This was the question you asked. Before going on to the 'you're specifying a camera for' part of the question, can I just ask you to clarify:

Do you think that a high-end DSLR camera in RAW mode can capture the dynamic range of a scene reasonably well?

Reasonably well means within 15% of scientifically calibrated equipment.

Assuming the image is neither underexposed nor overexposed. Determination of overexposure or underexposure would be done by histogram, or whatever method you allude to above in your grayscale test above.

If we don't agree on this basic point, there is no point in going any further...

BTW, here's an example of light pollution testing done with DSLR's that was later confirmed with scientific methods.

On the field we compared the numbers reduced from the DSLR images to measurements done by a professional imaging luminance meter. The resulted false colour images agreed to each other with the expected 10 percent precision. We use regularly Sky Quality Meters (SQM) to monitor the brightness of the sky. SQM values correlates well with the average luminance of the pictures taken by a wide-angle lens, when both devices are pointed to the zenith. The typical deviation between the SQM and DSLR measurement is about 0.04-0.06 mag/sqas. We also checked the linearity of the CCD by multiple exposures and different light levels. It is concluded that the raw images of a digital camera can be used for our purposes since the calibration errors do not exceed the precision needed for light pollution monitoring. Changes of weather conditions results in similar or larger errors than the interior error of our measurements.

U
Undisclosed #3
Apr 05, 2015

Do you think that a high-end DSLR camera in RAW mode can capture the dynamic range of a scene reasonably well?

Reasonably well means within 15% of scientifically calibrated equipment.

Maybe, I haven't tested it, and it seems like you haven't either. You're also cherry-picking my statements. In order for it to work, you'd need to have a decent DSLR (which I don't think the average tech would have), you'd need to have a good understanding of the limits of that DSLR and how to use it effectively, and you'd need to make sure the scene you're testing is a "worst case" scene.

This comes back to the point that I do *NOT* think a DSLR is a practical all-around dynamic range measurement tool. In some cases, with proper usage it could give you reasonable good information. In most cases I think it would give you misleading information in the sense that what you get from the DSLR would probably not help you choose a CCTV camera that much better.

If *you* have direct experience/data of having used a DSLR this way in multiple sites, please post it. Otherwise, I've tired of debating this theoretical case with you.

(1)
U
Undisclosed #2
Apr 05, 2015
IPVMU Certified

Thanks for your time... Looking back perhaps I overstated my case regarding any applicability of any measurements (accurate or not) to security cameras, hence my shameless cherry-picking. So I will defer to your greater experience and drop the matter. 1-0.

One question that I would still appreciate your opinion on would be concerning the process you describe here:

You point the camera at the lightbox with the grayscale transparency and you take an image. Then you look at the resultant image and look for the last bar that is not over exposed, the last bar that is not underexposed, take the delta between them and this is the range of light levels the camera can accurately capture at once. Convert that to db's or EV's and that is your dynamic range.

Since we both agree that something is going askew in the conversion to dB's in the last step, let me ask, if (and its a big if) the mfrs, instead of or addition to publishing the DR using their 'proprietary dB calculation', were to simply include the raw resultant image from the process above, might that help?

Then, assuming there was no out and out fraud, couldn't we compare two cameras results, in the manner you outlined above, and determine which one had the better dynamic range? And avoid the calculation debate altogether?

Thanks again.

MI
Matt Ion
Apr 06, 2015

Convert that to db's or EV's and that is your dynamic range.

Don't forget to also convert to "X-factor"...

HL
Horace Lasell
Apr 05, 2015

This has been a really informative conversation that helps "the rest of us" appreciate IPVM's value even more. Rather than some golden standard, IPVM tests and reports results of comparisons in real-world conditions. No PhD required to understand or validate the results. Thx again, IPVM!

(1)
MI
Matt Ion
Apr 06, 2015

I think B and C's discourse nicely illustrates the core issue with WDR ratings, though: there's really no agreement within the industry on how or where or when in the process to measure it, or exactly what real-world relevance those measurements have. Leaves me wondering if both B and C are John putting on a little role-play for us ;)

U
Undisclosed #3
Apr 06, 2015

I am definitely not an IPVM employee ;)

(1)
U
Undisclosed #2
Apr 06, 2015
IPVMU Certified

Leaves me wondering if both B and C are John putting on a little role-play for us ;)

If John orchestrated that interaction, including B's turgid and pedantic prose, the man deserves a blogging academy award...

JH
John Honovich
Apr 06, 2015
IPVM

I hate you both... :)

(3)
MI
Matt Ion
Apr 06, 2015

Day made.

Thanks for the grin, gentlemen!

New discussion

Ask questions and get answers to your physical security questions from IPVM team members and fellow subscribers.

Newest discussions