Testing Panoramic Cameras Outdoors

By Ethan Ace, Published May 12, 2013, 12:00am EDT (Research)

Outdoors, PTZs tend to be a favorite for covering large open areas but they only see whatever they are currently pointed at. Panoramics are an emerging alternative but how well do they perform and what type of video quality do they deliver?

Following our groundbreaking panoramic camera shootout, we took a group of panoramic cameras to a moderately busy intersection scene to test their performance viewing pedestrians and vehicle traffic at various distances, shown in this overhead view:

The cameras were mounted on our portable tripod, about 13' off the ground:

Delivering panoramic overviews like so:

Here are our key findings from the test:

  • Even in a well lit daytime scene, recognition of a human subject is possible only at very short range, e.g., a person waiting to cross the street. As we found in our indoor shootout, distances further than 5-10 feet provide little in the way of identifiable details.
  • Even at the shortest range possible, with a car just past the cameras, license plates were not readable. However, make, model, and color of vehicles is easily captured.
  • Performance varied widely at night. The Axis M3007 captured our human subject better at longer distances, but suffered from dropped frames and handled vehicle lights poorly. Panasonic handled lights best, but the human subject worse due to a dim image. Finally, the Vivotek FE8172V produced the brightest image, but noise and worse light handling made our subject difficult to spot and vehicles difficult to recognize.
  • At night, recognition even at short distances is difficult, due to noise and dimmer images. Detection of a subject at medium to long ranges (50'+) also becomes difficult. Vehicle information becomes very difficult to gather, due poor handling of headlights, noise, and dim images. Make and model were not possible to determine in most cases, and color was practically impossible to determine.

Taking the above into account, we believe the best use for panoramic cameras outdoors is situational awareness only. Users may find them desirable in conjunction with other cameras, to provide an overview of happenings in the scene, and provide the ability to follow subjects or vehicles as they move throughout the scene. We do not recommend using panoramic cameras if any level of identification is a requirement, aside from vehicle make and model in well lit scenes.

Daytime Comparisons

In order to test foot traffic common at intersections, we had our subject walk from approximately 100' to near the crosswalk signal, about 8' from the cameras. Starting with this overview image, we can immediately see differences in color and brightness among the three cameras. Some of this may be accounted for by switching white balance mode, but all were left on automatic for out tests.

The other notable difference is how much of the lens each camera uses. We can see the fuzzy edge of the fisheye in the M3007, while Panasonic crops this out. Vivotek does not crop it, but does not let the user pan up far enough to see it. This can have an effect on image quality, as resolution worsens the further you move towards the edge of the image, evident in the blurring in the Axis image. However, eliminating too much around the fisheye reduces the overall coverage of the camera. The FE8172V is an example of this, as this image is panned as far up as possible, but the image cuts off before reaching the horizon, visible in the other two images, despite all cameras being mounted at the same height and leveled.

At 100', the subject is visible, without any sort of fine details. Light shirt and (at best) blue jeans is all that can be made out.

Moving to about 50', at the crosswalk before crossing the street, slightly more detail is available on some cameras, though not much. At this point, we can also note differences in dewarping, with the subject appearing squat on the Axis M3007, and approximately proportionate on the other cameras. This could be an issue if trying to gauge height of a subject.

At 25', we can begin to make out facial features. At this point the subject may be recognizable if he's known, but strangers would be hard to identify with this resolution. The subject still appears wider using the Axis camera than others.

Finally, at 8', the subject is recognizable, with glasses and hair visible. A logo is almost visible on his shirt, as well.

Make and model of vehicles moving through the intersection were not difficult to determine, but even at the closest possible range (~20'), license plates were not discernable:

This clip shows traffic in motion for comparison. Users may download this video clip, as well. Note that both the M3007 and WV-SF458 dewarped on the camera, so the view may not be changed. The FE8172V features dewarping and immersive controls, however, so users may view the whole scene using this archive player download.

Nighttime Comparisons

Performance in the same test at night varied. The lighting in this scene varied between approximately 5-30 lux, depending on how close our target was to street and traffic lights, and discounting the effects of headlights. All cameras generally provided enough quality to monitor activities, though details of either our human subject or vehicles were difficult to detect. The Axis M3007 produced the clearest image with the truest color and least noise. The Panasonic WV-SF458 handled headlights and taillights best, but was overall the dimmest of our cameras, with the subject difficult to detect at range. Finally, though the FE8172V is the only true day/night camera in this test, and it provided bright images, it handled headlights and taillights poorly, and had substantial noise compared to the other two.

At 100', our subject is barely detectable, at best:

At 50', the subject is discernable on the M3007, though without any detail. He remains difficult to spot in the other two cameras, do to light levels and/or noise:

At 25', clothing can be at least partially determined using the M3007, and he is easily detectable.

Finally, at close range, clothing becomes easy to determine, and some facial detail is available.

Car color aside from "light" or "dark" was difficult to determine due to glare from headlights and taillights, making color a moot point. Make and model were practically impossible to determine, as well, due to noise, glare, or motion blur.

This video shows these effects in the nighttime traffic scene. Users may also download a 20-second video clip.

Test Details

We tested three cameras from our panoramic shootout:

  • Axis M3007-PV -- 5MP sensor, cropped to 800x600 view area, dewarped on camera
  • Panasonic WV-SF458: 3.1MP sensor, cropped to 1.3MP view area, dewarped on camera
  • Vivotek FE8172V: 5MP sensor, dewarped client side

All settings were left defaulted, except shutter speed, which was standardized at 1/30s. Recordings were at the camera's highest possible frame rate and resolution.

Comments (15)

Only IPVM Subscribers may comment. Login or Join.

Good test Ethan. Interesting seeing all the results of the big three we're seeing.

Nthan...great work! However assuming that the images were time synchronized, why are the images from the Vivotek camera always lagging behind the others?

Hi Michael, the cameras were synced to the same time server, and they're double checked before each test. There may be very slight float in the time (less than a second), but they generally show the same time. I suspect the lag we see is due to encoding latency, more than anything, as I can see it even in live video.


As I expected...I knew you would have them time synch'd! My comment was specifically directed to the encoding latency issue...and wondered why it's so noticeable w/ Vivotek and apparently nonexistent in the other 2?

Hey Michael, the Vivotek actually showed the least lag (all the way on the right in the embedded clips above), followed by Panasonic about a half a second (or less) behind it, then Axis about 1.5 seconds back.

I suspect it may have to do with how many streams the M3007 sends out, as we've noticed other issues because of this, too. I've already asked Axis for comment, and will circle back with whatever they tell us.

Ethan...yes, it was Axis...I was goijg from memory as I had closed the window long before I added the comment. It will be interesting to see what their response is...and it it's related to the number of streams, I wonder if shutting some of the added streams off would improve the performance?

Hello Ethan,

did u done any test on arecont vision 360/180 deg camera in indoor & outdoor application>

We have done a test of the 20MP 180 degree model.

We plan to test the upcoming 12MP 360 head to head against these models above.

Do you plan to test the 180 panoramics, such as those from scallop imaging? Would be interesting to see how they compared to 360 fisheye cameras, particularly with their ultra low light unit for night time.

Ben, we will do one general comparison between 180s and 360s to show fundamental tradeoffs but we are not going to 'shoot' them out head to head as it would not be apples to apples (dramatically different field of views).

As for Scallop Imaging, the issue we have is the max 1fps frame rate for full resolution. Regardless of its low light capability, at 1fps, that's a huge limitation. Our understanding is a new generation will come out (sometime) that will improve this and when that happens, we definitely test Scallop.

I would like to see how Immervision does in this comapriason. Are you planning to run include Immervision any time soon?

Jim, we included Immervision 1.3MP in the overall panoramic shootout, so you can see that comparison. Immervision image quality was in the middle of the pack there. As such, we would expect the same here.

More importantly, soon Immervision has their 5MP lens coming out and the first 3rd party minidome embedding their lenses. We plan to test both and would assume better performance from those new ones vs the old lens they have had out for years.

Right. I read that post with great interest some time ago. Awsome job by the way... I was also at the ISC when you did the live comparison. What i meant was that i wanted to see in in the 'street corner' version of the shootout. Should have explained myself better.

Also, that 5 mp Immervision lens was to have shipped under the Fuji name this month. I wanted a couple for a job I'm doing now, but they are not yet selling them. You are correct about the form factor issue. So i am sticking an Axis camera in a cylindrical light fixture (see below) that matches with other fixtures in the space. But after your shoot out, i am having second thoughts and thinking more about Arecont.

I am very interested in your test of the Brickcom camera (which imbeds the 5mp lens in a dome camera). I like the form factor and the fact that i don't have to worry about the VMS support issue; but i know nothing about the company. Tell me more....

Jim, thanks for the feedback and the diagram. Nice approach.

It's too bad that the 5MP lens keeps getting delayed. It's been going on for a very long time.

As for Bricksom, they are a Taiwanese manufacturer who first entered the surveillance market a few years ago. They are a subsidiary of Gemtek, a wireless broadband networking provider. Brickcom has a fairly standard line of surveillance cameras, with their key differentiator, not surprisingly being for wireless / cellular integrated options.

Immervision says they are going to embed their M mount lenses into many other minidomes over the next year so you can pick from Brickcom or later on, see who else Immervision adds.

Very interesting testing, thank you. I have implemented modeling panoramic cameras in VideoCAD8 and compared results of modeling with these tests.

Informative: 2
Funny: 1
Loading Related Reports