If you optimize a camera's settings, how much can you improve its performance? An interesting and important question, this report provides concrete evidence based on our systematic experimentation.
Over the last two months, our camera shootouts have become our most popular series ever. In them, we use the camera's default settings. The advantage of this includes: (1) replicating an 'out of the box' setup, (2) reflecting common usage patterns (i.e., most users do minimal 'tweaking'), (3) easy for others to replicate and (4) ensures no 'tricks' might unfairly help or hinder manufacturers.
On the other hand, a number of people have protested our use of camera defaults. They assured us that if we 'tweaked' their cameras that the poor quality images that our tests documented would be rectified.
We were quite interested to see how much of a difference optimizations would make. We contacted all the manufacturers of the cameras used in our parking garage shootout (Axis, Arecont, Bosch, IQinVision, and Sony). They each provided recommendations to optimize their particular camera for a parking garage environment (specifically the parking garage in our test). This report compares the manufacturer recommended settings to camera defaults.
Below is an image from our testing scene showing the presence of an outdoor opening and artificial lighting. The cameras were mounted at ceiling level aimed directly down the middle of the driving lane to capture people and cars moving through the garage.
This is the same scene as in the original report. An important point to emphasize is that is a fairly wide scene - more relevant to real world surveillance, unlike the tiny FoVs common in manufacturer marketing images. Below is the overhead map depicting distances, the various subject locations, and pixels/foot values at several key locations:
To systematically spot differences, we examined performance differences across 15 different scenarios / comparisons:
- Midday Back Lighting (Far Vehicle) - All Cameras, Default vs Optimized
- Midday Back Lighting (Near Vehicle) - All Cameras, Default vs Optimized
- Midday Even Lighting (Far Human) - All Cameras, Default vs Optimized
- Midday Even Lighting (Near Human) - All Cameras, Default vs Optimized
- Nighttime Color (Far Vehicle) - All Cameras, Default vs Optimized
- Nighttime Color (Near Vehicle) - All Cameras, Default vs Optimized
- Nighttime Color (Far Human) - All Cameras, Default vs Optimized
- Nighttime Color (Near Human) - All Cameras, Default vs Optimized
- Arecont Default vs Optimized
- Axis Default vs Optimized
- Bosch Default vs Optimized
- Sony Default vs Optimized
- Exposure Region Impact (Far Field)
- Exposure Region Impact (Near Field)
- Setting Experiments
We prepared 15 comparisons slides like the one below to better understand and document the level of change from a camera with its default settings to an optimized one using the manufacturer's recommendations.
Even for ourselves, seeing and documenting differences across multiple cameras and multiple times of day showed the potential and limits of camera optimizations. Inside, the Pro section we break down the specifics.