A Major Flaw in Long Lenses and PTZs Found
By Ethan Ace, Published Aug 26, 2014, 12:00am EDTTheoretically, long lenses should let you capture faces and license plates very far away.
For example, over 900 feet away from a 1080p camera one should get over 40ppf from a 100mm lens.
But how well does this work in practice?
In this test, we took 3 long lens cameras, including:
- An Axis 1080p PTZ
- A Dahua 720p PTZ
- A Fujinon 8-80mm varifocal lens [link no longer available] with a 720p Samsung SNB-5004 camera
We ran tests from 100 to 500 feet away from the cameras, comparing the theoretical PPF to the actual achieved.
All 3 cameras tested showed significant image quality problems at longer lens lengths, with degradation compared to theoretical max performance averaging 50%
This is a major issue when using camera calculation / estimation, one that we suspected but wanted to confirm with actual testing.
A few camera manufacturers privately confirmed that they were not surprised as the MTF, or resolving power, of long varifocal lenses have known, though not publicly acknowledged, problems. Lens manufacturers are reluctant, if not paranoid, about disclosing technical details. The poor MTF at longer lens lengths is most likely source of the image quality issues.
Handling Depth of Field Issues
Advanced surveillance professionals will recall that depth of field is a problem for longer lens, like the 80mm+ used in these tests.
To control for this, we manually focused each camera at each spot tested. For example, a focus was done at the 100' distance test, again at the 300' distance test, etc.
Though this is unrealistic for lenses added to fixed cameras, we did this to eliminate depth of field as being the cause of image quality issues.
8-80mm PPF
Using the 8-80mm Fujinon lens at its longest focal length, visible quality was equivalent to images of nearly 50% its actual PPF, seen in the following examples.
Using the 8-80mm lens, despite actual PPF being 63, image quality is roughly equivalent to ~35 PPF.
At 500', the same is true, with effective image quality of only about 20 PPF, about half of actual (38 PPF).
At close range, ~100' from the camera, the same holds true, with effective PPF around 80, less than half of actual. However, effects are much less pronounced at pixel densities this high, making them less noticeable.
PTZ Cameras
These findings held true for PTZ cameras, as well. Looking at examples from our integrated IR PTZ test.
In the example below, using a 720p PTZ with a 4.7-94mm lens, effective image quality at ~420' was similar to ~30 PPF, instead of actual (~55 PPF).
Image quality was also greatly reduced using the 1080p PTZ, effectively about ~36 PPF vs. the 87 which should be delivered.
Depth of Field
In addition to image quality reductions, users should be aware of depth of field issues when using long lenses. Focused at ~500', the field of view began to soften around 300', with severe blurring at 200' and below due to subjects being out of focus.
8 reports cite this report:
Comments (31)
The best I can tell this has been going on for years. It is something we have seen but never formally tested / proved.
This major quality degradation has been masked by offering longer lens length. Of course, longer lens lengths with degraded resolution is a poor tradeoff, as you are forced to take a narrower FoV to get the details one desires.
I am not sure what, if any, manufacturers will do. Our understanding is that size and cost constraints restrict options. Its certainly possible to build long length varifocal lens without such problems, but, at least today, these issues appear to be the norm.
However, we are planning to modify the lens calculator to factor / display the risks involved with contemporary long length lenses.
John, I am curious as to why do you give Dahua such visibility within IPVM when their cameras hide behind OEM labels? If so, then it would be helpful to know who the OEMs are. As you say within other IPVM articles, the three most value-centric camera lines appear to be Dahua, HIK Vision, and Samsung. Of the three I am least likely to ever procure Dahua because of their non-presence in America (again, per you and by personal experience).
I would find it much more helpful to have you take a Samsung or HV camera out of the value lines and pick a major name (Sony, IQeye, Panasonic, or choose one at random). And although it is tempting to always include Axis, I think a lot of systems integrators (me included) shy away from Axis due to poor margins; so too many Axis references are not helpful either.
But otherwise, keep up the terrific work! We are lucky to have a Consumer Reports for IPVS available to us.
Thanks for the feedback.
We include Axis a lot because they are the favorite camera, by far, amongst IPVM integrator members. I agree with your point about margins, also with Axis' massive availability. Despite that, it's undeniable that they are broadly used.
As for this test, we used Dahua because we literally had it set up recently making it logistically easier.
Btw, we did include Samsung in this test (it was the fixed camera) and we recently released a test on Panasonic new Series 6. Though we recently tested an IQinVision camera, IQinVision has not been a major name for a long long time (see Vicon deal), though I am sure they are flattered that you mentioned them .
We can't fit every camera in every test. And for this one specifically, it seems pretty clear that this is a category issue.
If any PTZ manufacturer believes they do not have this problem with long lenses, please speak up and we'll do a test.
Again I appreciate the feedback and just wanted to give you a sense of what we have recently tested and why we choose what we do.
Linear density is determined by the lens aperture.
Was convinced.
A Fujinon 8-80mm varifocal lens with a 720p Samsung SNB-5004 camera
Was the Samsung actually tested at 720p?
It's native resolution is 1280x1024, and since your ppf is based on horizontal pixels, you could pick up 50% more vertical pixels by using 1024, without affecting the ppf calc at all.
"Picking up" vertical pixels does not change the image quality. It would simply increase the vertical area captured by the camera.
This is a really just an example of 4:3 1.3MP vs 17:9 720p, e.g., Aspect Ratio 16:9 vs 4:3 Shootout
Would be interesting to see the same test with a Canon ES lens which is a DSLR lens that appears to have better glass than traditional lenses. I believe the Avigilon Pro Series cameras have the Canon ES lens for the 8, 12, 16 & 29 MP have the Canon ES lens.
I suspect Canon ES lens as well as some MP machine vision lenses would do better. The key limits one encounters there is cost (5x,10x, more) and size (which is an issue for speeddomes).
We are looking to test alternative lens options that would be feasible for mainstream use.
Hi Ethan, thank you for your excellent article. By any chance do you have a long, fixed lens which could be used to compare the quality of image against the varifocal lens on the Samsung SNB-5004?
Can you say at what length of lens, and below, image degradation becomes a non-issue? Does this vary much depending upon the brand of lens or other factors? Thank you for your information.
Hey Luke, we do not have a long fixed lens, and they're actually not so easy to come by. There are a couple options which are 50 or 75mm, but they're generally machine vision lenses, not IR corrected, and more expensive than even the 8-80mm we tested, in the $300+ range.
Lens manufacturer definitely makes a difference in shorter focal length lenses, but it also changes from model to model, it seems. We have some info on more typical lenses from a handful of manufacturers in our HD lens shootout.
"For the stadium, we used a 5-megapixel camera at 75-feet wide and 34 pixels per foot (ppf). Then we opened up to a higher resolution: 10-megapixels at 110-feet wide at 33ppf. We tried a 25-135mm lens that performed great with the camera and became one of the lenses they ultimately used in the installation. After seeing several different scenarios, they picked a target of 40ppf for every seat in the stadium." Quote Link
Similar setups are getting popular with NFL stadiums.
Jeremiah, It's worth differentiating between how well it worked and how well it performed relative to PPF theoretical calculations.
All the cameras in our test worked 'well' at some level. You could see details very far away, relatively speaking. The issue is that they did not work 'as well' as PPF predicts.
Btw, we've tested the model they cite, i.e., "10-megapixel AV10115DNv1 MegaVideo® Compact cameras in HSG2 housings to achieve their target of 40ppf resolution on every seat and face." That model has a lot of problems dealing with lighting variations and, to that end, I highly doubt they are getting clear facial details reliably with 40ppf, especially with a 125mm lens. Reference: 10MP vs 5MP vs HD Shootout
Btw, I think the 25- 135mm they are referring to is from CBC (see manual and auto iris versions). We'll buy one and do a follow up test.
Isn't some of the quality degradation explained by motion blur, which is intensified as you zoom? Could be due to unseen vibrations, e.g. due to nearby machinery/natural harmonics of the pole, etc. I would try putting your finger on the camera enclosure and check if it has any impact on the directionality or intensity of the blur.
Isn't the reduced quality due to anisoplanaticism across the image? The angular size of the pixels in these examples is 5 seconds of arc or thereabout. So one can expect atmospheric image warp to start to become a problem. Also in the movie (although there for illustrating depth of field issues), I think I see warping of the edge of the cones. I also think warp can be seen in the eyechart. Look at the "z" in the top line. I think warping is also present in some of the face images. If the focal lengths were doubled, or the pixels made smaller, then I think that this would become a huge problem. So I don't think there is anything wrong at all with the lenses.
Please help us out with this one. Does anisoplanticism refer to non-cartesian mapping of optical path lengths that occurs because of atmospheric diffraction which is different for each ray path?
Or, what is it?
Thanks.
Yes, isoplanticism means all ("iso") in the same plane ("planaticism"). So all of the central rays from each portion of the image are parallel and a good picture can be formed. Anisoplanaticism means that the image is not isoplanatic so all of the central rays are not parallel and a good image cannot be formed. Think of "heat waves" rising from a desert floor and the resulting "wavering" image. As image magnification goes up and up, eventually you see this. The magnfications used here are getting to this point. You always encounter it when you get to about 1 arc second, but with harsh conditions it can come in at tens of seconds of arc. And you'll note that I think the results for images taken on the grass are better than the ones over the asphalt of the parking lot. This is to be expected. So I don't think there is anything wrong with the lens. It's almost certainly a wonderful piece of optics. But you are expecting too much from the atmosphere. When it's foggy out and you can't see anything, you don't blame the lens.
Does the test hold true for PTZ at standard definition at these high focal lengths? I recall reading several years ago that the glass in the lens of most high pixel cameras don't support anywhere near the pixel densities of these compact cameras. There has to be good reason a digital SLR camera has a lens that is 10x the length and width of a lens used in a typical HD security camera.
Hi Robert, good question. We're going to get out a standard def PTZ and check that out.
As far as DSLR lens size, keep in mind that most of their sensors are much larger than surveillance imagers. A Four Thirds DSLR sensor is 9x larger than a 1/2.7" sensor common on HD cameras, and that's small for a DSLR. So with sensor areas that big, they simply need more glass and larger lenses. This image shows relative sizes:
The important point is not the size of the sensor and the focal length of the lens, you have to think about the angular size of the pixel as it maps onto your targets. So it's the ratio of these two quantities that is important. Lenses are all about bending light and angles. The lense has no idea what sort of sensor you're going to put behind it. But really, I think the problem here is not the lense at all. It is the atmosphere--see my previous note about anisoplanaticism. The angular sizes of the pixels in this test are getting pretty small and are of a size at which atmospheric interference warps and blurs the image. You always run into this if you make the magnification large enough, but it happens sooner if you are looking out over a hot desert, for example, where there is lots of atmospheric trubulence because of the rising columns of air. I think you can see in the movie that this is starting to happen in these test. Again, see my previous note.
IPVM's tests covered a variety of conditions. If the deviation from theoretical was related to atmospheric distortion, how would you explain the observation that the effect is pretty consistent across a variety of distances? Passing through only 20% of the distorting media, would one expect significantly better performance at 100' than at 500'?
Hi Horace,
No. The effect is very local. This doesn't happren at all with satellite pictures, for example. If the light is bent away at great distances from the lens, then the light doesn't make it into the camera and there is no effect. So this happens (depending on the atmosphere, of course), in the first 100 yards (closer if the turbulence is high--could be 100 feet). You can also defeat this effect a lot if you put the camera on a tower. The affect goes away very significantly if you get the camera off the ground.
My very best regards!
Rick
As an integrator many of the discussion points regarding anisoplanaticism and isoplanaticisim made my head ache but the insight is invaluable. Especially when one comment / recommendation stands out so clearly: "the effect goes away very significantly if you get the camera off the ground." Thank you Rick!
The second critical reminder is the truism that the image is only as good as the glass in front of the camera. We routinely shot surveillance photos of investigative targets 200-400 yards away and learned even the best 35mm cameras and finest grain film can't compensate for lousy glass. Compound that with tiny (comparatively speaking) imagers and the head starts aching again!
Howdy Ethan! Do y'all have a link for any additional images/detail that go with this test?
Hi Jim,
What other images / details do you want?
John
Maybe I jumped the gun there, I thought this search result was referring to the same test as in the Flaw Found one? Maybe supplemental info? Or an outline? But it says 'try back later once its posted', so I reckon I'm wondering when that will be...
(In case my image didn't load up)
Long Focal Length PPF Test
...blurry around 250-300'. [[FOV example]] To see if this was a function of long focal length or the lens itself we tested another lens, a Fujinon ___ 4-12mm megapixel rated lens, set to ~8mm, the same field of view as its 8-80mm counterpart. At this focal length, the 4-12mm lens is demonstrably better...
Howdy everyone!
It's now been over a week since I asked about any other image data on this test. First off as you can see above, John didn't say there wasn't any, but instead just asked me for exactly what I was lookin for. I obliged him generally, and then a little later I screen printed exactly what I seen in the search results to make me curious. By the next mornin the search no longer returned that result. Gone just like that, without a word to Adam. I still have my screen prints though, and this here is what I can reconstruct of what looks to be a different version of this here 'Major Flaw' report, (but this one talks about an extra fuji lens, to prove where the fault lie):
Long Focal Length PPF Test
We tested two megapixel rated lenses from Fujifilm [[part numbers]] As well as two HD PTZ cameras to see how image quality holds up under these long focal lengths.
Using the 8-80mm at its longest focal length, visible quality was equivalent to images of 50% or lower PPF. [[100' 190 vs 80, 94 vs 45 etc. Additionally depth of field was limited. Focused on our subject at ~500, images were blurry around 250-300'. [[FOV example]]
To see if this was a function of long focal length or the lens itself we tested another lens, a Fujinon ___ 4-12mm megapixel rated lens, set to ~8mm, the same field of view as its 8-80mm counterpart. At this focal length, the 4-12mm lens is demonstrably better, providing approximately the PPF levels it should (X at Y distance)
So was this written before the test or after?
If after, where are those 'demonstrably better images' which prove the lens is at fault? I'm having a hard time thinking of any answers that don't involve either:
Writing the report before even doing the test and / or
Ignoring important but not helpful results to reach a foregone conclusion
Though the reason I can't think of an explanation might just be my own lack of imagination, so make-up your own mind! But based on the reception given to my earlier question and the removal of the search data, I'm not expecting an answer.
Now that I've done said my peace, I will saddle-up and get out of y'alls hair.
We have two types of articles - 'reviews' and 'updates'. Ethan originally wrote this as an update. What you saw is an unpublished early draft that our search engine showed in the results. That was a bug (showing unpublished posts) which we fixed in the past week or so.
This report is the one and only official post on the topic.
The poor MTF at longer lens lengths is most likely source of the image quality issues.
Yes, it may very well be.
The 3MP fuji lens is listed at f/1.4 at 8mm, and although I can't find a published spec for it at 80mm, a conservative guess puts it at around at least f/3, possibly a good deal more.
Where the snb-5004's 1/3" image sensor looks to have a pixel width of ~3.5 microns
- A lens at f/2.8 corresponds to an airy disk of ~3.7 microns.
- A lens at f/4.0 corresponds to an airy disk of ~5.3 microns.
- A lens at f/5.6 corresponds to an airy disk of ~7.5 microns.
- A lens at f/8.0 corresponds to an airy disk of ~10.7 microns.
Note that severe diffractive effects aren't seen until typically the airy disk width is 2x the pixel width, although some distortion is noticeable at greater than 1x. So depending on what the effective f-number of the lens actually is, the sensor/lens combination may be indeed diffraction limited.
On the other hand it's hard to be sure because, at lower f-numbers the difference is small. One idea would be to retest the lens with a sensor with a higher resolution and smaller pixel width. Maybe try the SNB-6004 or even the 3 MegaByteMegaPixel 7004, as they may be easier set similarly, being in the same Wisenet III family.
If the image is merely resolution-limited then going to the higher resolution sensor should yield a substantial improvement. If the image is severely diffraction limited though the improvement should be minimal.
Thoughts/corrections?
