The Indoor 7 Camera Shootout

Author: Benros Emata, Published on May 28, 2011

Indoors, surveillance cameras are used all over. Given the generally small areas monitored and their shelter from the weather, getting good surveillance video is easier than outdoors. It raises the question: Does it even matter what type of surveillance cameras you use indoors? What type of practical differences or benefits does one camera provide over another?

We wanted to test this rigorously so we set up a 7 camera test kit to do a simultaneous shootout. In our kit we included cameras ranging from SD through 5MP from Arecont Vision, Avigilon, Axis, Bosch and Sony to see if more resolution helps, if D/N versus color only matters and if different manufacturers are 'better', etc.

We mounted our test kit in a corner at ceiling height as is common in real world installations.

We first tested during the day with even artifical lighting conditions to get a baseline test of an 'easy' scenario:

We tested with the shades open to see the impact of outdoor lighting through a window. Often businesses (like retailers) like to have the windows open to bring in natural light and to let prospective customers see in. We wanted to see what negative impact this might have on each camera:

Finally, we tested in the dark to see how cameras would perform after hours in case an intruder broke in. Since manufacturers use varying tricks at night, we did 2 tests here - one with manufacturer defaults and one with exposure normalized.

The overhead image and map below shows the coverage, areas, distances and pixels per foot at different testing positions. The human figure indicates the 3 spots where we performed analysis of image quality:

Analyzing Video Quality

We analyzed a series of scenes to see how performance varied across the cameras. We digitally zoomed each camera to the same level to show differences. See an example scene comparison from our parking lot test.

We conducted our analysis across 12 scenes. They are:

Get Video Surveillance News In Your Inbox
Get Video Surveillance News In Your Inbox

  • Even Artificial Lighting (Doorway)
  • Even Artificial Lighting (Far)
  • Even Artificial Lighting (Near)
  • Backlit/WDR via Open Window (Doorway)
  • Backlit/WDR via Open Window (Far)
  • Backlit/WDR via Open Window (Near)
  • Front-lit via Open Window (Doorway)
  • Front-lit via Open Window (Far)
  • Front-lit via Open Window (Near)
  • Low-Light (Default Settings)
  • Low-Light (Normalized Settings)
  • Low-Light Default vs Low-Light Normalized

Analyzing Bandwidth Consumption - NEW

Not only do we examine which cameras provide the best quality in each scene, we now break down the bandwidth consumption, showing who's the best and worse for the amount of network bandwidth and storage consumed.

Our analysis covers 7 metrics:

  • Even Artificial Lighting Bandwidth Consumption Per Camera
  • Backlight / WDR Bandwidth Consumption Per Camera
  • Front-Lit Bandwidth Consumption Per Camera
  • Low-Light (Default Settings) Bandwidth Consumption Per Camera
  • Low-Light (Normalized Settings) Bandwidth Consumption Per Camera
  • Average Bandwidth Consumption Per Scene
  • Average Bandwidth Consumption Per Camera Overall

We found a suprising level of differences amongst cameras for image quality, bandwidth and overall performance. Contrary to common belief, more resolution did not always result in better image quality nor higher bandwidth. Inside the Pro section, we dig into the details and differences.

Key Findings and Recommendations

  • In practically every scene tested, 720p appeared to provide a distinct edge over SD. At the wider FoVs tested, e.g. 31ft, it typically meant the difference between near absolute identification (720p) versus only a slight to moderate certainty of identification (SD).
  • Contrary to expectations, higher MP (e.g. 5MP) did not provide real material benefits over 720p cameras under test conditions. Often, even at the widest FoVs tested, the two categories rendered similar if not equal results.
  • SD approached near absolute certainties of subject identification only at more narrow FoVs and under more ideal lighting conditions (e.g., front-lit at 16ft or less FoVs).
  • Low-light tests revealed only a minority of cameras can be deployed with default settings to effectively monitor in ~0 lux environments. Most cameras will require exposure settings be optimized for a particular low-light application. When shutter speeds were normalized to 1/30s, SD D/N appeared to exhibit the best performance.

In light of these findings we recommend the following:

  • Consider megapixel between 1MP to 2MP resolution. Higher megapixel may present only marginal gains in image quality, a diminished low-light sensitivity and a lower frame-rate.
  • SD may be considered for narrower FoV applications (e.g. ~16ft or less). SD will tend to provide slightly better low-light and bandwidth performance.
  • Consider positioning cameras 'downstream' from bright windows or light sources. If monitoring activity at a window is desired, however, consider a camera with capable WDR performance

Even Artificial Lighting (Doorway)

In this scenario cameras are digitally zoomed into a human subject ~28ft out, corresponding to a HFoV of ~26ft. Lighting is even and not particularly challenging (~200 lux).

Below is the Composite Snapshot (Download Full Video Clip):

The majority of MP cameras in the composite (i.e. 5MP, 720p Avigilon/Sony) provide unequivocally greater detail levels on the human subject than the SD cameras. In these cameras' images identification is highly certain. Still, SD provides some key investigative details such as hair-style, build, complexion, and gender. Note that both the 720p M1114 and 1.3MP AV1315DN provide lesser image qualities than the other two 720p cameras, despite having equal horizontal resolutions.

Even Artificial Lighting (Far)

In this scenario cameras are digitally zoomed into a human subject ~33ft out, corresponding to a HFoV of ~31ft. Lighting is even and not particularly challenging (~200 lux).

Below is the Composite Snapshot (Download Full Video Clip):

Results are similar to the 'doorway' counterpart, where three of five MPs (i.e. 5MP, 720p Avigilon, 720p Sony CH140) provide strong detail levels and strong odds of identification. SDs are considerably less detailed but still offer clues as to the subject's hair-style, build, complexion and gender. Note that in the SDs and two of the MPs (1.3MP AV1315 and 720p Axis M1114) the presence of eye-glasses are not readily detectable.

Even Artificial Lighting (Near)

In this scenario cameras are digitally zoomed into a human subject ~17ft out, corresponding to a HFoV of ~16ft. Lighting is even and not particularly challenging (~200 lux).

Below is the Composite Snapshot (Download Full Video Clip):

In this 'near' composite all cameras provide fair to high levels of detail, and subsequent moderate to high certainty levels of identification. The same three MPs (5MP, 720p Avigilon and 720 Sony CH140) provide the greatest level of details once again. Note that the 720p M1114 provides a lesser image quality than the other two 720p cameras, despite having equal horizontal resolutions.

Back-lit/WDR via Open Window (Doorway)

In this scenario cameras are digitally zoomed into a human subject ~28ft out, corresponding to a HFoV of ~26ft. The open window creates a lighting challenge with moderate WDR properties. Roughly ~1000 lux is measured at the window, while ~300 lux is measured at center room.

Below is the Composite Snapshot (Download Full Video Clip):

In this backlit scene all cameras render the human subject's face somewhat under-exposed, leading to loss of contrast and overall clarity. The 720p Sony CH140 is the one exception which exposes the subject the best. Despite diminished picture qualities relative to even lit scenarios, the MPs (i.e. 5MP, 720p Avigilon and 720p Sony CH140) still provide moderate to strong odds of identification. Note that both the 720p M1114 and 1.3MP AV1315DN, despite having nearly twice the ppf, appear to perform only similarly to the SD Bosch NBN-498.

Back-lit/WDR via Open Window (Far)

In this scenario cameras are digitally zoomed into a human subject ~33ft out, corresponding to a HFoV of ~31ft. The open window creates a lighting challenge with moderate WDR properties. Roughly ~1000 lux is measured at the window, while ~300 lux is measured at center room.

Below is the Composite Snapshot (Download Full Video Clip):

The challenges of back-lighting and WDR are most evident in this composite. All cameras show severely diminished levels of detail compared to the same 'far' scene of the even lit scenario. The 5MP and 720p Avigilon/Sony provide the best level of details, but likelihood of identification has degraded to fair/moderate levels. A majority of the MPs however, still provide other key investigative details regarding clothes, gender, build, complexion and eye-glasses. Note the 720p (Sony CH140) and SD Bosch NBN-498's ability to provide more appropriate exposure levels on the external side of the window. This could be a potential benefit in the detection of suspicious activity occuring outside (e.g. potential assault, vandalism or break-in).

Back-lit/WDR via Open Window (Near)

In this scenario cameras are digitally zoomed into a human subject ~17ft out, corresponding to a HFoV of ~16ft. The open window creates a lighting challenge with moderate WDR properties. Roughly ~1000 lux is measured at the window, while ~300 lux is measured at center room.

Below is the Composite Snapshot (Download Full Video Clip):

Three of five MPs (i.e. 5MP, 720p Avigilon, and 720 Sony) provide strong enough detail levels for high certainty identification. The remaining MPs and both SDs are considerably 'blurry' and somewhat under-exposed.

Front-lit via Open Window (Doorway)

In this scenario cameras are digitally zoomed into a human subject ~28ft out, corresponding to a HFoV of ~26ft. The room is illuminated with natural light provided by the open window. Note that the open window has front-lit the room due to positioning cameras at the opposite end - 'down-stream' of light source. As a result lighting is relatively even (~300 lux) and does not present a challenge.

Below is the Composite Snapshot (Download Full Video Clip):

The 5MP, 720p Avigilon and 720p Sony CH140 provide the greatest level of details and fairly strong certainties of identification. The 1.3MP, 720p Axis M1114 and SDs provide similar levels of detail, which are not sufficient for strong odds of identification.

Front-lit via Open Window (Far)

In this scenario cameras are digitally zoomed into a human subject ~33ft out, corresponding to a HFoV of ~31ft. The room is illuminated with natural light provided by the open window. Note that the open window has front-lit the room due to positioning cameras at the opposite end - 'down-stream' of light source. As a result lighting is relatively even (~300 lux) and does not present a challenge.

Below is the Composite Snapshot (Download Full Video Clip):

None of the cameras appear to provide an exceptional level of detail in this composite. The 5MP, 720p Avigilon and 720p Sony provide the best level of details, but do not result in near or absolute certainty of identification.

Front-lit via Open Window (Near)

In this scenario cameras are digitally zoomed into a human subject ~17ft out, corresponding to a HFoV of ~16ft. The room is illuminated with natural light provided by the open window. Note that the open window has front-lit the room due to positioning cameras at the opposite end - 'down-stream' of light source. As a result lighting is relatively even (~300 lux) and does not present a challenge.

Below is the Composite Snapshot (Download Full Video Clip):

Strong detail levels are provided by all MP cameras (both higher MP and 720p), making identification highly likely. The one exception is the 720p Axis M1114 which is more comparable to the SD cameras in this composite. Despite a somewhat weaker picture quality, they still provide a reasonably good level of detail and chance at identification.

Low-light (Default Settings)

In this composite, cameras are left at their full field of view (no digital zoom). Light levels are measured ~0.1 lux in this scene. Note that all cameras are set to default settings.

Below is the Composite Snapshot (Download Full Video Clip):

With default settings applied, most cameras in the composite provide some measure of utility after low-light adaptation. The SD Bosch NBN-498 clearly provides the best overall picture quality amongst them. The exposure maximums of the 720p Sony CH140 and 720p Avigilon are among the lowest (1/30s), and, not surprisingly, exhibit considerably under-exposed images.

Low-light (Normalized)

In this composite, cameras are left at their full field of view (no digital zoom). Light levels are measured ~0.1 lux in this scene. Note that all cameras have been 'normalized' to a 1/30s max exposure time.

Below is the Composite Snapshot (Download Full Video Clip):

Interestingly, cameras on the top row (5MP Axis P1347, 720p Axis M1114, and SD Axis M1011), which all featured 1/6s max shutter speeds, have all degraded into essentially 'black' images, since normalizing to 1/30s. The SD Bosch NBN-498 continues to provide the best apparent illumination even after the normalization process. Note that the 720p Avigilon and 720p Sony CH140 have default max shutters of 1/30s, and as such underwent no changes in the normalization scenario.

Low-light Default vs. Normalized

In this comparison images from each of the seven (7) cameras are arranged side by side with default settings (left) vs. normalized settings (right). The light levels are the same for either side (~0.1 lux).

Below is the Composite Snapshot:

Bandwidth Analysis

In this section, we examine the comparative bandwidth consumption of each camera, explaining variances in results and key issues.

In the even lighting scene, bandwidth consumption is fairly similar as the scene has minimal motion and no lighting challenges. Keep this in mind when we contrast this to other scenes.

In the front-lit scene, we have the lowest overall bandwidth consumption of the all the scenes. Our hypothesis is that camera bandwidth consumption is minimized when the lighting is quite level. To the contrary, note the jump in Arecont's bandwidth. This will be a consistent (and increasing problem) for the Arecont.

In the backlight / WDR scene, all of the camera's bandwidth consumption remains relatively the same except for the Arecont which jumps even higher.

Here in the low light scene is where things get really interesting. The Arecont, Axis and Avigilon cameras all show steep jumps in bandwidth consumption (despite the fact that Arecont and Axis frame rate drops significantly). Key driving factor is the hard to compress visual noise from aggressive gain control. Since all of these cameras are VBR, the bitrate surges. While the Sony also displays significant visual noise, as it uses constant bit rate encoding, no change occurs. While the Bosch's rate control allows the bandwith to rise it does not because of strong low light performance with minimal noise.

In this final scene, we kept the low light levels the same but normalized all the cameras to a 1/30s exposure (note: the Avigilon and Sony default to 1/30s, all others were changed). The Arecont bandwidth consumption increases even higher (to scary levels) as well as the Axis SD (in color mode with high visible noise). Interestingly, though, the Axis MP cameras bandwidth drops to nearly zero as the scene is pitch black.

Now we can look at the average bandwidth levels across scenes. The pattern is obvious - low light bandwidth consumption is nearly triple that of daytime bandwidth consumption.

Finally, we can see the average bandwidth consumption by camera:

  • Arecont consumes the most - driven by its challenges in handling lighting variations.
  • Avigilon is the next highest - though the image quality was comparatively strong throughout
  • The SD Axis came in 3rd highest mainly do to challenges in handling challenging light conditions
  • Using the default 2Mb/s bandwidth level, Sony delivered very consistent bandwidth consumption.
  • The 5MP Axis was the second lowest. However, its max frame rate was only 40% of the other cameras (12 vs 30fps) so direct comparison is misleading.
  • The MP Axis camera's bandwidth consumption was helped by dropping their bandwidth consumption to near zero in the normalized test. On the other hand, the Avigilon and the Sony consumed more bandwidth but provided at least some details in its video (making them more usable than the MP Axis cameras).

Methodology

The following are the seven (7) cameras used in the 'Office Shootout':

  • Arecont AV1315DN (online $460) - 1.3MP D/N; 1/2.7" CMOS; F1.6; 0.1 Lux (BW)
  • Avigilon 1.0MP-HD-H264-B1 (MSRP $645) -720p D/N; 1/2.7" CMOS; F1.6; 0.4/0.04 Lux (Color/BW)
  • Axis M1011 (online $160) - 4CIF Color-only; 1/4" CMOS; Integrated F2.0; 1.0 Lux (Color)
  • Axis M1114 (online $475) - 720p D/N; 1/4" CMOS;
  • Axis P1347 (online $1350) - 5MP D/N; 1/2.5" CMOS; F1.6 Kowa;0.08 Lux (BW)
  • Bosch NBN-498 (online $700) - SD D/N; 1/3" CCD; F1.3 Bosch; 0.06 Lux (BW)
  • Sony CH140 (online $800) - 720p D/N; 1/3" CMOS; F1.2 Fujinon; 0.1 Lux (BW)

Each of the seven (7) cameras were simultaneously recorded to an Exacqvision VMS during five (5) key scenarios. All cameras were set to their defaults during testing, except during the low-light tests. During low-light tests D/N cameras were 'forced' into B/W to ensure consistency of comparison. Furthermore, cameras were normalized to 1/30s shutter speed for one of the tests. All camera lenses were adjusted to a uniform lens angle (~50 degrees) and recorded/analyzed at their - VMS allowed - maximum resolutions.

Here are the key default settings and firmware versions for each camera:

  • Arecont 1315DN - Automatic Exposure 'Balanced'; fw 65197
  • Avigilon 1.0MP - 1/30s maximum exposure; fw 1.0.4.4(3305)
  • Axis M1011 - 1/6s max exposure w/ AGC; fw 5.00.1
  • Axis M1114 - 1/6s max exposure w/ AGC; fw - 5.09
  • Axis P1347 - 1/6s max exposure w/ AGC; fw 5.20
  • Bosch NBN-498 - 1/60s shutter; SensUp 4x; AGC on '15'; fw 01500411
  • Sony CH140 - 1/30s shutter; View-DR/VE On; AGC 'Middle'; fw 1.26.01

Here are the five (5) key scenarios examined:

  • Even Artificial Lighting
  • Back-lit/WDR via Open Window
  • Front-lit via Open Window
  • Low-Light w/ Default Settings
  • Low-Light w/ Normalized Settings

All five (5) scenarios, employed the same choreagraphed activity. The activity involves, at the start, one human subject entering the room from a doorway (center room) and another subject positioned at the far corner of the room. Both subjects progress toward the center of the room and then toward nearer field of views until finally exiting scene.

4 reports cite this report:

Directory of Camera Shootout Series on Dec 27, 2011
The following directory lists all of our camera shootouts. These shootouts pit 4 - 8 different surveillance cameras in simultaneous tests on real...
Top 12 Reports for 2011 on Dec 25, 2011
Here are our top 12 reports for 2011, starting with the state of the surveillance industry: 2012 Video Surveillance Guide - Key trends reviewed...
Are You Smarter than a Security Trade Magazine? on Aug 25, 2011
If you have any IP video surveillance skills, you should easily spot at least 1 of 2 fundamental errors from a single page 'test report' published...
The Truth About IR Lighting and Bandwidth on Aug 13, 2011
One of the industry's top growing myths is that IR illumination massively reduces storage and bandwidth costs. No one has done more to perpetuate...
Comments : PRO Members only. Login. or Join.

Related Reports

April 2018 IP Networking Course on Apr 19, 2018
This is the last chance to register for our IP Networking course. Register now. NEW - 2 sessions per class, 'day' and 'night' to give you double...
Axis Launches First Stereographic Panoramic Camera on Apr 05, 2018
Fisheye panoramics have not had much innovation in the past few years, with their multi-imager cousins getting most of the advances, e.g.,...
Dahua 4K Micro 4/3 Camera Tested (IPC-HF8835F) on Apr 03, 2018
Dahua is going big. Their new 4K HF8835F Micro Four Thirds sensor is one of the largest now on the market. Dahua is claming: Starlight...
Wireless Networking For Video Surveillance Guide on Mar 29, 2018
Wireless networking is a niche in video surveillance applications, but it can be a difficult one to understand with proper wireless design,...
Axis Z-Wave IP Camera Tested Poorly on Mar 20, 2018
Z-Wave is drawing notable interest for video surveillance use. In IPVM's initial coverage, 84% expressed interest in it, with nearly half being...
May 2018 Camera Course on Mar 16, 2018
Our next course starts on May 8th. Register now for the Spring 2018 Camera Course This is the only independent surveillance camera course, based...
Camera Form Factor Guide on Mar 16, 2018
When selecting surveillance cameras, users may choose from a number of different form factors, each with its own unique strengths and weaknesses,...
Cellular (4G / LTE / 5G) For Video Surveillance Guide on Mar 06, 2018
In this report, we explain using cellular for video surveillance including: 4G vs LTE vs 5G 4G standards 5G future Advantage: Placing cameras...
Next Gen 5MP / 6MP Camera Shootout (Axis vs Dahua vs Hanwha vs Hikvision) on Feb 28, 2018
Many manufacturers have released new generation 5MP / 6MP cameras that tout super low light, WDR and other features historically typical in 1080p...
Hikvision 6MP Camera Tested (DS-2CD4565-IZH) on Feb 27, 2018
In the next installment of our ongoing testing of 5MP/6MP cameras, we test the Hikvision DS-2CD4565F-IZH, a 'Smart Series' vandal dome combining...

Most Recent Industry Reports

Global Real-Time Video Surveillance - EarthNow on Apr 20, 2018
A new company, EarthNow, with backing from Bill Gates, Airbus and more, is claiming that: Users will be able to see places on Earth with a delay...
Dedicated Vs Converged Access Control Networks (Statistics) on Apr 20, 2018
Running one's access control system on a converged network, with one's computers and phones, can save money. On the other hand, hand, doing so can...
April 2018 IP Networking Course on Apr 19, 2018
This is the last chance to register for our IP Networking course. Register now. NEW - 2 sessions per class, 'day' and 'night' to give you double...
Rare Video Surveillance Fundraising - Verkada $15 Million on Apr 19, 2018
Fundraising in video surveillance (and the broader physical security market) has been poor recently. Highlights are few and far in between...
'Best In Show' Fails on Apr 19, 2018
ISC West's "Best In Show" has failed. For more than a decade, it has become increasingly irrelevant as the selections exhibit a cartoon level...
Security Camera Cleaning Frequency Statistics on Apr 18, 2018
150+ integrators told IPVM how often they clean cameras on customer's sites and why.  Inside we examine their answers and break down feedback...
Worst Access Control 2018 on Apr 18, 2018
Three access control providers stood out as providing the most problems for integrators. In this report, we analyze the answers to: "In the...
Axis VMD4 Analytics Tested on Apr 17, 2018
Axis is now on its 4th generation of video motion detection (VMD), which Axis calls "a free video analytics application." In this generation, Axis...
Arecont CEO And President Resign on Apr 17, 2018
This is good news for Arecont. Arecont's problems have been well known for years (e.g., most recently Worst Camera Manufacturers 2018 and starting...
Strong ISC West 2018, Says Manufacturers, GSX / ASIS Expected Weaker on Apr 17, 2018
Manufacturers say ISC West 2018 was strong, continuing the trend we have seen in 2017 results and 2016 results. However, those same 100...

The world's leading video surveillance information source, IPVM provides the best reporting, testing and training for 10,000+ members globally. Dedicated to independent and objective information, we uniquely refuse any and all advertisements, sponsorship and consulting from manufacturers.

About | FAQ | Contact