Camera Color Fidelity Shootout

FB
Figen Bekisli
•Published Sep 04, 2013 04:00 AM

The surveillance industry is obsessed with more and more pixels. But color fidelity - accurately rendering yellow as yellow or blue as blue - can be critical for identifying individuals or objects.

We decided to test 5 camera manufacturers - Axis, Avigilon, Bosch, Dahua and Sony - to see how faithful their cameras could reproduce color.

The Test

We used a ColorChecker (aka Macbeth chart) and shot out all cameras side by side simultaneously. Here's a preview of the results with the models blacked out.

This was a fairly simple low light indoor scene but still notice how much color fidelity varied across them.

Scenes

Moreover, we tested in a variety of conditions including different forms of outdoor street lighting to see how LED, mercury vapor, and sodium lights impacted performance.

We concluded with real world examples of a subject wearing different colored shirts.

Here's one example of a major manufacturer who made a range colors look basically the same:

Contrast to another major manufacturer who handled the same scene, same time dramatically better:

Who Won?

Here are our key findings from the test:

  • Avigilon's H3 1MP and Axis' M1114 had the most issues in our tests, with difficulty in nearly all scenes, especially difficulty differentiating between bright colors.
  • The Bosch NBN-733V and Sony SNC-VB600 were most accurate and consistent across scenes.
  • Low light conditions (~1 lx) generated less accurate colors for most of the cameras tested, with dark colors especially more difficult to discern.
  • Indoors, in full light, few problems were present, aside from Avigilon and Axis issues mentioned above.
  • Fluorescent and LED lights produced relatively true colors compared to mercury vapor and sodium lights.
  • Mercury vapor and sodium light created the most discoloration and differentiation issues, resulting in greenish and yellowish tints, respectively.
  • WDR improved color fidelity in all scenes. In some cases, colors became slightly more washed out with WDR on, however, they were generally easier to differentiate. Notably, WDR had little to no effect on color fidelity in the Sony VB600, which was true regardless of setting.
  • We found automatic white balance settings performed better in all scenes than using light-specific modes. 

Recommendations

Users seeking truest color in a wide variety of scenes should consider Bosch and Sony, as both were strong under the widest variety of light levels and types. Both the Avigilon H3 1MP and Axis M1114 had numerous problems faithfully representing colors, and should be avoided in these applications.

We recommend using automatic white balance as it results in best performance under most conditions. Specific modes and manual settings should be used with caution as they may reduce fidelity as light changes throughout the day. 

Overview Screencast [WATCH THIS!]

Take 5 minutes and let us show you all the key findings in this video screencast:

White Balance Settings

All images in this test are taken using automatic white balance settings. While a number of cameras offered multiple modes in order to compensate for different lighting types, we found automatic actually offered best performance. For example, in the case of Sony, while the camera offers modes specifically for mercury or sodium lighting, the camera essentially did not need them, staying true on their default automatic mode. 

Additionally, users could manually adjust white balance to compensate for harsh colored lighting such as sodium. However, unless this is turned on and off on a schedule, adjusting to compensate for yellow light at night would tint daytime video violet, as red and blue must be increased.

For these reasons, automatic white balance is recommended unless lighting problems are constant, 24/7, without change, in which case manual white balance could be attempted.

Indoor Lighting

We tested six cameras under three different indoor lighting conditions. These tests were performed under fluorescent office lighting, very common in commercial buildings.

Interior fluorescent, ~160 lx

Color fidelity in most cameras was fairly true. The Axis M1114 had the most severe problems, with yellow, light green and other lighter colors nearly indistinguishable. Additionally, light gray tones at the bottom row appeared white. Avigilon H3-B2 experienced similar problems, though to a lesser degree. Colors that had significant issues were highlighted on the charts below. The colors on the remaining four cameras were truer, with Axis Q1604 being more vibrant than the other three. 

30 lux

When the light was reduced to 30 lux, slight color changes occurred in all cameras. The colors were less vibrant but the distinguishability observed with full light was still present. Notably, the M1114 improves, with colors more discernable than in full light.

Low Light, ~1 lx

At 1 lux, low light conditions and noise greatly affect the color performance of the cameras. In the Avigilon H3-B2 camera, most darker colors appeared black (highlighted). Colors turned greenish in the Axis Q1604 and Dahua HF3101N, the latter being affected more drastically. Colors in the VB600 begin to wash out at this light level. Dark red appeared as brown in both the Axis Q1604 and Sony VB600. The M1114 improved yet again, with colors most true at this light level. The Bosch NBN-733V remained most true under all light conditions.

We also did some color tests using a person wearing different colored t-shirts under the same lighting conditions and FOV. This way we could see how well MacBeth Chart color results match with more real objects. Below are some darker color t-shirts which appear as black with Avigilon H3-B2, consistent with the MacBeth chart results. The results of the Bosch NBN-733V are also added for comparison of the worst and best performance in this scene.

Outdoor Daytime

In full sun, all cameras experienced issues. The Avigilon H3-B2 and Axis M1114 performed similarly to indoors, with lighter colors and gray tones washed out. Colors in the Q1604 were pale, with light grays hard to discern. For the NBN-733V, light green and yellow were difficult to distinguish. The VB600 displayed bluish green (top right) and cyan (bottom right) almost the same. 

Since bright green, yellow, and orange were among the most washed out colors in the Avigilon H3 and Axis M1114, we tested these colors in shirts on our subject. Though still washed out in places, these colors are easier to discern on our subject since reflected light illuminates folds and wrinkles of the shirt much less harshly than the bright direct sun overhead. The MacBeth chart is a single, flat surface, so is illuminated only by this harsh light, resulting in poorer fidelity.

Outdoor Lights

We also tested under three different outdoor lighting types, in order to see what effects each had on fidelity.

LED

Since LEDs produce bright white light, it intensified issues in the Avigilon H3-B2 and Axis M1114, which both had difficulty distinguishing many colors on the chart. The remaining four cameras handled the color fidelity well, similar to our indoor test.

These images show four of the most difficult colors from our test chart, (gray, bright green, yellow, and orange) on our subject for the cameras H3-B2, M1114 (two worst) and Sony VB600 (one of the best). Discerning one from the other is difficult under the bright white LED light for both H3-B2 and M1114 while VB600 doesn't have any issues.

Mercury Vapor

Mercury vapor lamps, another common outdoor choice, produce a bluish-green tint. This can be seen in the Q1604 and HF3101N, which experienced some discoloration with a more greenish tint. Other cameras were much the same as under LED lighting.

Light colors were washed out for Avigilon H3-B2 and Axis M1114 while darker colors were distinguishable in our t-shirt tests under mercury vapor. This result is consistent with the MacBeth Chart.

 

Sodium Vapor

Sodium vapor lights are notorious for having poor color rendition, due to the strongly orange light they produce. Under a sodium lamp, discoloration was significantly worse for Dahua HF3101N and the Axis Q1604. Although all cameras yielded worse results relative to other outdoor lighting conditions, Sony performed more satisfactorily in terms of both color fidelity and distinguishability.

The significant effect of the yellowish tint with Dahua HF3101N under sodium vapor light can be seen by comparing images of five different colored t-shirts below. Avigilon H3-B2, which showed significantly washed out colors, and the NBN-733, which remained true in this scene, are shown below for comparison.

WDR ON vs. OFF

Generally, color fidelity was better with WDR on. 

Indoor

First, in our indoor scene, the Q1604 improved with WDR turned on, with colors more saturated and truer.

Notably, Sony remains relatively unchanged, as it did under all lighting styles. Dark colors were slightly truer, though others look nearly identical.

This t-shirt example shows the differences in bright and dark colors in the Q1604. Much like the MacBeth chart shows, bright green is washed out with WDR off, while it is clearly visible when turned on. Darker colors, such as red in the example below, are not a problem.

Sodium Vapor

Under sodium vapor, the harshest lighting style in our test, WDR had the largest effect in the Q1604. Much of the test chart was desaturated when WDR was turned off. However, with WDR on, the yellow tint of the sodium vapor lighting is much more pronounced.

The comparison below demonstrates these issues using our subject in different colored t-shirts. With WDR off, bright green is extremely washed out, appearing white. However, turning it on, the entire scene becomes tinted yellow, including his skin tone, and the shirt looks much more yellow than green. Dark colors are much less of an issue, though the tint remains.

 

Test Parameters

For all cameras, we used the same camera settings.  Brightness, sharpness, and contrast were default. The exposure setting was 1/30 s and white balance was automatic. Below are the firmware versions we used in this test:

Comments are shown for subscribers only. Login or Join