The Indoor 7 Camera Shootout

By: Benros Emata, Published on May 28, 2011

Indoors, surveillance cameras are used all over. Given the generally small areas monitored and their shelter from the weather, getting good surveillance video is easier than outdoors. It raises the question: Does it even matter what type of surveillance cameras you use indoors? What type of practical differences or benefits does one camera provide over another?

We wanted to test this rigorously so we set up a 7 camera test kit to do a simultaneous shootout. In our kit we included cameras ranging from SD through 5MP from Arecont Vision, Avigilon, Axis, Bosch and Sony to see if more resolution helps, if D/N versus color only matters and if different manufacturers are 'better', etc.

We mounted our test kit in a corner at ceiling height as is common in real world installations.

We first tested during the day with even artifical lighting conditions to get a baseline test of an 'easy' scenario:

We tested with the shades open to see the impact of outdoor lighting through a window. Often businesses (like retailers) like to have the windows open to bring in natural light and to let prospective customers see in. We wanted to see what negative impact this might have on each camera:

Finally, we tested in the dark to see how cameras would perform after hours in case an intruder broke in. Since manufacturers use varying tricks at night, we did 2 tests here - one with manufacturer defaults and one with exposure normalized.

The overhead image and map below shows the coverage, areas, distances and pixels per foot at different testing positions. The human figure indicates the 3 spots where we performed analysis of image quality:

Analyzing Video Quality

We analyzed a series of scenes to see how performance varied across the cameras. We digitally zoomed each camera to the same level to show differences. See an example scene comparison from our parking lot test.

We conducted our analysis across 12 scenes. They are:

Get Notified of Video Surveillance Breaking News
Get Notified of Video Surveillance Breaking News

  • Even Artificial Lighting (Doorway)
  • Even Artificial Lighting (Far)
  • Even Artificial Lighting (Near)
  • Backlit/WDR via Open Window (Doorway)
  • Backlit/WDR via Open Window (Far)
  • Backlit/WDR via Open Window (Near)
  • Front-lit via Open Window (Doorway)
  • Front-lit via Open Window (Far)
  • Front-lit via Open Window (Near)
  • Low-Light (Default Settings)
  • Low-Light (Normalized Settings)
  • Low-Light Default vs Low-Light Normalized

Analyzing Bandwidth Consumption - NEW

Not only do we examine which cameras provide the best quality in each scene, we now break down the bandwidth consumption, showing who's the best and worse for the amount of network bandwidth and storage consumed.

Our analysis covers 7 metrics:

  • Even Artificial Lighting Bandwidth Consumption Per Camera
  • Backlight / WDR Bandwidth Consumption Per Camera
  • Front-Lit Bandwidth Consumption Per Camera
  • Low-Light (Default Settings) Bandwidth Consumption Per Camera
  • Low-Light (Normalized Settings) Bandwidth Consumption Per Camera
  • Average Bandwidth Consumption Per Scene
  • Average Bandwidth Consumption Per Camera Overall
 
  • In practically every scene tested, 720p appeared to provide a distinct edge over SD. At the wider FoVs tested, e.g. 31ft, it typically meant the difference between near absolute identification (720p) versus only a slight to moderate certainty of identification (SD).
  • Contrary to expectations, higher MP (e.g. 5MP) did not provide real material benefits over 720p cameras under test conditions. Often, even at the widest FoVs tested, the two categories rendered similar if not equal results.
  • SD approached near absolute certainties of subject identification only at more narrow FoVs and under more ideal lighting conditions (e.g., front-lit at 16ft or less FoVs).
  • Low-light tests revealed only a minority of cameras can be deployed with default settings to effectively monitor in ~0 lux environments. Most cameras will require exposure settings be optimized for a particular low-light application. When shutter speeds were normalized to 1/30s, SD D/N appeared to exhibit the best performance. 

In light of these findings we recommend the following:

  • Consider megapixel between 1MP to 2MP resolution. Higher megapixel may present only marginal gains in image quality, a diminished low-light sensitivity and a lower frame-rate.
  • SD may be considered for narrower FoV applications (e.g. ~16ft or less). SD will tend to provide slightly better low-light and bandwidth performance.
  • Consider positioning cameras 'downstream' from bright windows or light sources. If monitoring activity at a window is desired, however, consider a camera with capable WDR performance

Even Artificial Lighting (Doorway)

In this scenario cameras are digitally zoomed into a human subject ~28ft out, corresponding to a HFoV of ~26ft. Lighting is even and not particularly challenging (~200 lux).

Below is the Composite Snapshot (Download Full Video Clip):

The majority of MP cameras in the composite (i.e. 5MP, 720p Avigilon/Sony) provide unequivocally greater detail levels on the human subject than the SD cameras. In these cameras' images identification is highly certain. Still, SD provides some key investigative details such as hair-style, build, complexion, and gender. Note that both the 720p M1114 and 1.3MP AV1315DN provide lesser image qualities than the other two 720p cameras, despite having equal horizontal resolutions.

Even Artificial Lighting (Far)

In this scenario cameras are digitally zoomed into a human subject ~33ft out, corresponding to a HFoV of ~31ft. Lighting is even and not particularly challenging (~200 lux).

Below is the Composite Snapshot (Download Full Video Clip):

Results are similar to the 'doorway' counterpart, where three of five MPs (i.e. 5MP, 720p Avigilon, 720p Sony CH140) provide strong detail levels and strong odds of identification. SDs are considerably less detailed but still offer clues as to the subject's hair-style, build, complexion and gender. Note that in the SDs and two of the MPs  (1.3MP AV1315 and 720p Axis M1114) the presence of eye-glasses are not readily detectable.

Even Artificial Lighting (Near)

In this scenario cameras are digitally zoomed into a human subject ~17ft out, corresponding to a HFoV of ~16ft. Lighting is even and not particularly challenging (~200 lux).

Below is the Composite Snapshot (Download Full Video Clip):

In this 'near' composite all cameras provide fair to high levels of detail, and subsequent moderate to high certainty levels of identification. The same three MPs (5MP, 720p Avigilon and 720 Sony CH140) provide the greatest level of details once again. Note that the 720p M1114 provides a lesser image quality than the other two 720p cameras, despite having equal horizontal resolutions.

Back-lit/WDR via Open Window (Doorway)

In this scenario cameras are digitally zoomed into a human subject ~28ft out, corresponding to a HFoV of ~26ft. The open window creates a lighting challenge with moderate WDR properties. Roughly ~1000 lux is measured at the window, while ~300 lux is measured at center room.

Below is the Composite Snapshot (Download Full Video Clip):

In this backlit scene all cameras render the human subject's face somewhat under-exposed, leading to loss of contrast and overall clarity. The 720p Sony CH140 is the one exception which exposes the subject the best. Despite diminished picture qualities relative to even lit scenarios, the MPs (i.e. 5MP, 720p Avigilon and 720p Sony CH140) still provide moderate to strong odds of identification. Note that both the 720p M1114 and 1.3MP AV1315DN, despite having nearly twice the ppf, appear to perform only similarly to the SD Bosch NBN-498.

Back-lit/WDR via Open Window (Far)

In this scenario cameras are digitally zoomed into a human subject ~33ft out, corresponding to a HFoV of ~31ft. The open window creates a lighting challenge with moderate WDR properties. Roughly ~1000 lux is measured at the window, while ~300 lux is measured at center room.

Below is the Composite Snapshot (Download Full Video Clip):

The challenges of back-lighting and WDR are most evident in this composite. All cameras show severely diminished levels of detail compared to the same 'far' scene of the even lit scenario. The 5MP and 720p Avigilon/Sony provide the best level of details, but likelihood of identification has degraded to fair/moderate levels. A majority of the MPs however, still provide other key investigative details regarding clothes, gender, build, complexion and eye-glasses. Note the 720p (Sony CH140) and SD Bosch NBN-498's ability to provide more appropriate exposure levels on the external side of the window. This could be a potential benefit in the detection of suspicious activity occuring outside (e.g. potential assault, vandalism or break-in).

Back-lit/WDR via Open Window (Near)

In this scenario cameras are digitally zoomed into a human subject ~17ft out, corresponding to a HFoV of ~16ft. The open window creates a lighting challenge with moderate WDR properties. Roughly ~1000 lux is measured at the window, while ~300 lux is measured at center room.

Below is the Composite Snapshot (Download Full Video Clip):

Three of five MPs (i.e. 5MP, 720p Avigilon, and 720 Sony) provide strong enough detail levels for high certainty identification. The remaining MPs and both SDs are considerably 'blurry' and somewhat under-exposed.

Front-lit via Open Window (Doorway)

In this scenario cameras are digitally zoomed into a human subject ~28ft out, corresponding to a HFoV of ~26ft. The room is illuminated with natural light provided by the open window. Note that the open window has front-lit the room due to positioning cameras at the opposite end - 'down-stream' of light source. As a result lighting is relatively even (~300 lux) and does not present a challenge.

Below is the Composite Snapshot (Download Full Video Clip):

The 5MP, 720p Avigilon and 720p Sony CH140 provide the greatest level of details and fairly strong certainties of identification. The 1.3MP, 720p Axis M1114 and SDs provide similar levels of detail, which are not sufficient for strong odds of identification.

Front-lit via Open Window (Far)

In this scenario cameras are digitally zoomed into a human subject ~33ft out, corresponding to a HFoV of ~31ft. The room is illuminated with natural light provided by the open window. Note that the open window has front-lit the room due to positioning cameras at the opposite end - 'down-stream' of light source. As a result lighting is relatively even (~300 lux) and does not present a challenge.

Below is the Composite Snapshot (Download Full Video Clip):

None of the cameras appear to provide an exceptional level of detail in this composite. The 5MP, 720p Avigilon and 720p Sony provide the best level of details, but do not result in near or absolute certainty of identification.

Front-lit via Open Window (Near)

In this scenario cameras are digitally zoomed into a human subject ~17ft out, corresponding to a HFoV of ~16ft. The room is illuminated with natural light provided by the open window. Note that the open window has front-lit the room due to positioning cameras at the opposite end - 'down-stream' of light source. As a result lighting is relatively even (~300 lux) and does not present a challenge.

Below is the Composite Snapshot (Download Full Video Clip):

Strong detail levels are provided by all MP cameras (both higher MP and 720p), making identification highly likely. The one exception is the 720p Axis M1114 which is more comparable to the SD cameras in this composite. Despite a somewhat weaker picture quality, they still provide a reasonably good level of detail and chance at identification.

Low-light (Default Settings)

In this composite, cameras are left at their full field of view (no digital zoom). Light levels are measured ~0.1 lux in this scene. Note that all cameras are set to default settings.

Below is the Composite Snapshot (Download Full Video Clip):

With default settings applied, most cameras in the composite provide some measure of utility after low-light adaptation. The SD Bosch NBN-498 clearly provides the best overall picture quality amongst them. The exposure maximums of the 720p Sony CH140 and 720p Avigilon are among the lowest (1/30s), and, not surprisingly, exhibit considerably under-exposed images.

Low-light (Normalized)

In this composite, cameras are left at their full field of view (no digital zoom). Light levels are measured ~0.1 lux in this scene. Note that all cameras have been 'normalized' to a 1/30s max exposure time.

Below is the Composite Snapshot (Download Full Video Clip):

Interestingly, cameras on the top row (5MP Axis P1347, 720p Axis M1114, and SD Axis M1011), which all featured 1/6s max shutter speeds, have all degraded into essentially 'black' images, since normalizing to 1/30s. The SD Bosch NBN-498 continues to provide the best apparent illumination even after the normalization process. Note that the 720p Avigilon and 720p Sony CH140 have default max shutters of 1/30s, and as such underwent no changes in the normalization scenario.

Low-light Default vs. Normalized

In this comparison images from each of the seven (7) cameras are arranged side by side with default settings (left) vs. normalized settings (right). The light levels are the same for either side (~0.1 lux).

Below is the Composite Snapshot:

Bandwidth Analysis

In this section, we examine the comparative bandwidth consumption of each camera, explaining variances in results and key issues.

In the even lighting scene, bandwidth consumption is fairly similar as the scene has minimal motion and no lighting challenges. Keep this in mind when we contrast this to other scenes.

In the front-lit scene, we have the lowest overall bandwidth consumption of the all the scenes. Our hypothesis is that camera bandwidth consumption is minimized when the lighting is quite level. To the contrary, note the jump in Arecont's bandwidth. This will be a consistent (and increasing problem) for the Arecont.

In the backlight / WDR scene, all of the camera's bandwidth consumption remains relatively the same except for the Arecont which jumps even higher.

Here in the low light scene is where things get really interesting. The Arecont, Axis and Avigilon cameras all show steep jumps in bandwidth consumption (despite the fact that Arecont and Axis frame rate drops significantly). Key driving factor is the hard to compress visual noise from aggressive gain control. Since all of these cameras are VBR, the bitrate surges. While the Sony also displays significant visual noise, as it uses constant bit rate encoding, no change occurs. While the Bosch's rate control allows the bandwith to rise it does not because of strong low light performance with minimal noise.

In this final scene, we kept the low light levels the same but normalized all the cameras to a 1/30s exposure (note: the Avigilon and Sony default to 1/30s, all others were changed). The Arecont bandwidth consumption increases even higher (to scary levels) as well as the Axis SD (in color mode with high visible noise). Interestingly, though, the Axis MP cameras bandwidth drops to nearly zero as the scene is pitch black.

Now we can look at the average bandwidth levels across scenes. The pattern is obvious - low light bandwidth consumption is nearly triple that of daytime bandwidth consumption.

Finally, we can see the average bandwidth consumption by camera:

 

  • Arecont consumes the most - driven by its challenges in handling lighting variations.
  • Avigilon is the next highest - though the image quality was comparatively strong throughout
  • The SD Axis came in 3rd highest mainly do to challenges in handling challenging light conditions
  • Using the default 2Mb/s bandwidth level, Sony delivered very consistent bandwidth consumption.
  • The 5MP Axis was the second lowest. However, its max frame rate was only 40% of the other cameras (12 vs 30fps) so direct comparison is misleading.
  • The MP Axis camera's bandwidth consumption was helped by dropping their bandwidth consumption to near zero in the normalized test. On the other hand, the Avigilon and the Sony consumed more bandwidth but provided at least some details in its video (making them more usable than the MP Axis cameras).

 

Methodology

The following are the seven (7) cameras used in the 'Office Shootout':

  • Arecont AV1315DN (online $460) - 1.3MP D/N; 1/2.7" CMOS; F1.6; 0.1 Lux (BW)
  • Avigilon 1.0MP-HD-H264-B1 (MSRP $645) -720p D/N; 1/2.7" CMOS; F1.6; 0.4/0.04 Lux (Color/BW)
  • Axis M1011 (online $160) -  4CIF Color-only; 1/4" CMOS; Integrated F2.0; 1.0 Lux (Color)
  • Axis M1114 (online $475) - 720p D/N; 1/4" CMOS;
  • Axis P1347 (online $1350) - 5MP D/N; 1/2.5" CMOS; F1.6 Kowa;0.08 Lux (BW)
  • Bosch NBN-498 (online $700) - SD D/N; 1/3" CCD; F1.3 Bosch; 0.06 Lux (BW)
  • Sony CH140 (online $800) - 720p D/N; 1/3" CMOS; F1.2 Fujinon; 0.1 Lux (BW)

Each of the seven (7) cameras were simultaneously recorded to an Exacqvision VMS during five (5) key scenarios. All cameras were set to their defaults during testing, except during the low-light tests. During low-light tests D/N cameras were 'forced' into B/W to ensure consistency of comparison. Furthermore, cameras were normalized to 1/30s shutter speed for one of the tests. All camera lenses were adjusted to a uniform lens angle (~50 degrees) and recorded/analyzed at their - VMS allowed - maximum resolutions.

Here are the key default settings and firmware versions for each camera:

  • Arecont 1315DN - Automatic Exposure 'Balanced'; fw 65197
  • Avigilon 1.0MP - 1/30s maximum exposure; fw 1.0.4.4(3305)
  • Axis M1011 - 1/6s max exposure w/ AGC; fw 5.00.1
  • Axis M1114 - 1/6s max exposure w/ AGC; fw - 5.09
  • Axis P1347 - 1/6s max exposure w/ AGC; fw 5.20
  • Bosch NBN-498 -  1/60s shutter; SensUp 4x; AGC on '15'; fw 01500411
  • Sony CH140 - 1/30s shutter; View-DR/VE On; AGC 'Middle'; fw 1.26.01

Here are the five (5) key scenarios examined:

  • Even Artificial Lighting
  • Back-lit/WDR via Open Window
  • Front-lit via Open Window
  • Low-Light w/ Default Settings
  • Low-Light w/ Normalized Settings

All five (5) scenarios, employed the same choreagraphed activity. The activity involves, at the start, one human subject entering the room from a doorway (center room) and another subject positioned at the far corner of the room. Both subjects progress toward the center of the room and then toward nearer field of views until finally exiting scene.

 

 

4 reports cite this report:

Directory of Camera Shootout Series on Dec 27, 2011
The following directory lists all of our camera shootouts. These shootouts pit 4 - 8 different surveillance cameras in simultaneous tests on real...
Top 12 Reports for 2011 on Dec 25, 2011
Here are our top 12 reports for 2011, starting with the state of the surveillance industry: 2012 Video Surveillance Guide - Key trends reviewed...
Are You Smarter than a Security Trade Magazine? on Aug 25, 2011
If you have any IP video surveillance skills, you should easily spot at least 1 of 2 fundamental errors from a single page 'test report' published...
The Truth About IR Lighting and Bandwidth on Aug 13, 2011
One of the industry's top growing myths is that IR illumination massively reduces storage and bandwidth costs. No one has done more to perpetuate...
Comments : PRO Members only. Login. or Join.

Related Reports

Anyvision Facial Recognition Tested on Aug 21, 2019
Anyvision is aiming for $1 billion in revenue by 2022, backed by $74 million in funding. But does their performance live up to the hype they have...
Dahua 4K Camera Shootout on Aug 20, 2019
Dahua's new Pro Series 4K N85CL5Z claims to "deliver superior images in all lighting and environmental conditions", but how does this compare to...
Axis 4K Camera Shootout 2019 on Aug 14, 2019
Axis' 4K Q3518-LVE claims the "best video quality possible", with Lightfinder super low light performance, Axis' high end Forensic WDR, and...
Proactive CCTV "Only Affordable Video Archiving Solution" Profile on Aug 12, 2019
Proactive CCTV is claiming to offer "the only affordable video archiving solution on the market", reducing the storage typically required for H.265...
Axis Door Station A8207-VE Tested on Aug 07, 2019
Axis newest door station, the A8207-VE, claims to deliver "video surveillance, two-way communication, and access control" in a single device. But...
Hikvision 4K Camera Shootout on Aug 02, 2019
With their latest Smart Series 5 cameras, Hikvision is claiming cameras "fully loaded" with "state-of-the-art technology for high performance and...
Hanwha 4K Wisenet P Camera Tested on Jul 30, 2019
Hanwha claims "premium performance" and to "deliver the best in high resolution performance and efficiency" with their Wisenet P series 4K models,...
Bosch Starlight 8000i Cameras Tested on Jul 23, 2019
Bosch has released their new Flexidome IP Starlight 8000i cameras, claiming "exceptional detail even in extreme low-light situations." To see...
Avigilon ACC7 VMS Tested on Jul 22, 2019
Avigilon's Control Center 7 boldly claims it will "transform live video monitoring" with the new Focus of Attention "AI-enabled" interface. We...
History of Video Surveillance on Jul 19, 2019
The video surveillance market has changed significantly since 2000, going from VCRs to ab emerging AI cloud era.  The goal of this history is to...

Most Recent Industry Reports

TMA Apologizes to Amazon / Ring on Aug 23, 2019
Not only is Amazon / Ring making major incursions into the residential security market, the organization representing the biggest incumbents, The...
China Dahua Replaces Their Software With US Pepper on Aug 22, 2019
What does a US government banned company do to improve its security positioning in the US? Well, Dahua is unveiling a novel solution, partnering...
Security Integrators Outlook On Remaining Integrators In 2025 on Aug 22, 2019
The industry has changed substantially in the last decade, with the rise of IP cameras and the race to the bottom. Indeed, more changes may be...
First GDPR Facial Recognition Fine For Sweden School on Aug 22, 2019
A school in Sweden has been fined $20,000 for using facial recognition to keep attendance in what is Sweden's first GDPR fine. Notably, the fine is...
Anyvision Facial Recognition Tested on Aug 21, 2019
Anyvision is aiming for $1 billion in revenue by 2022, backed by $74 million in funding. But does their performance live up to the hype they have...
JCI Sues Wyze on Aug 21, 2019
The mega manufacturer / integrator JCI has sued the fast-growing $20 camera Seattle startup Wyze. Inside this note: Share the court...
Dahua 4K Camera Shootout on Aug 20, 2019
Dahua's new Pro Series 4K N85CL5Z claims to "deliver superior images in all lighting and environmental conditions", but how does this compare to...
ZK Teco Atlas Access Control Tested on Aug 20, 2019
Who needs access specialists? China-based ZKTeco claims its newest access panel 'makes it very easy for anyone to learn and install access control...
Uniview Beats Intel In Trademark Lawsuit on Aug 19, 2019
Uniview has won a long-running trademark lawsuit brought by Intel, with Beijing's highest court reversing an earlier Intel win, centered on...
Suprema Biometric Mass Leak Examined on Aug 19, 2019
While Suprema is rarely discussed even within the physical security market, the South Korean biometrics manufacturer made global news this past...

The world's leading video surveillance information source, IPVM provides the best reporting, testing and training for 10,000+ members globally. Dedicated to independent and objective information, we uniquely refuse any and all advertisements, sponsorship and consulting from manufacturers.

About | FAQ | Contact