The Indoor 7 Camera Shootout

By: Benros Emata, Published on May 28, 2011

Indoors, surveillance cameras are used all over. Given the generally small areas monitored and their shelter from the weather, getting good surveillance video is easier than outdoors. It raises the question: Does it even matter what type of surveillance cameras you use indoors? What type of practical differences or benefits does one camera provide over another?

We wanted to test this rigorously so we set up a 7 camera test kit to do a simultaneous shootout. In our kit we included cameras ranging from SD through 5MP from Arecont Vision, Avigilon, Axis, Bosch and Sony to see if more resolution helps, if D/N versus color only matters and if different manufacturers are 'better', etc.

We mounted our test kit in a corner at ceiling height as is common in real world installations.

We first tested during the day with even artifical lighting conditions to get a baseline test of an 'easy' scenario:

We tested with the shades open to see the impact of outdoor lighting through a window. Often businesses (like retailers) like to have the windows open to bring in natural light and to let prospective customers see in. We wanted to see what negative impact this might have on each camera:

Finally, we tested in the dark to see how cameras would perform after hours in case an intruder broke in. Since manufacturers use varying tricks at night, we did 2 tests here - one with manufacturer defaults and one with exposure normalized.

The overhead image and map below shows the coverage, areas, distances and pixels per foot at different testing positions. The human figure indicates the 3 spots where we performed analysis of image quality:

[link no longer available]

Analyzing Video Quality

We analyzed a series of scenes to see how performance varied across the cameras. We digitally zoomed each camera to the same level to show differences. See an example scene comparison from our parking lot test.

We conducted our analysis across 12 scenes. They are:

Get Notified of Video Surveillance Breaking News
Get Notified of Video Surveillance Breaking News

  • Even Artificial Lighting (Doorway)
  • Even Artificial Lighting (Far)
  • Even Artificial Lighting (Near)
  • Backlit/WDR via Open Window (Doorway)
  • Backlit/WDR via Open Window (Far)
  • Backlit/WDR via Open Window (Near)
  • Front-lit via Open Window (Doorway)
  • Front-lit via Open Window (Far)
  • Front-lit via Open Window (Near)
  • Low-Light (Default Settings)
  • Low-Light (Normalized Settings)
  • Low-Light Default vs Low-Light Normalized

Analyzing Bandwidth Consumption - NEW

Not only do we examine which cameras provide the best quality in each scene, we now break down the bandwidth consumption, showing who's the best and worse for the amount of network bandwidth and storage consumed.

Our analysis covers 7 metrics:

  • Even Artificial Lighting Bandwidth Consumption Per Camera
  • Backlight / WDR Bandwidth Consumption Per Camera
  • Front-Lit Bandwidth Consumption Per Camera
  • Low-Light (Default Settings) Bandwidth Consumption Per Camera
  • Low-Light (Normalized Settings) Bandwidth Consumption Per Camera
  • Average Bandwidth Consumption Per Scene
  • Average Bandwidth Consumption Per Camera Overall
 
  • In practically every scene tested, 720p appeared to provide a distinct edge over SD. At the wider FoVs tested, e.g. 31ft, it typically meant the difference between near absolute identification (720p) versus only a slight to moderate certainty of identification (SD).
  • Contrary to expectations, higher MP (e.g. 5MP) did not provide real material benefits over 720p cameras under test conditions. Often, even at the widest FoVs tested, the two categories rendered similar if not equal results.
  • SD approached near absolute certainties of subject identification only at more narrow FoVs and under more ideal lighting conditions (e.g., front-lit at 16ft or less FoVs).
  • Low-light tests revealed only a minority of cameras can be deployed with default settings to effectively monitor in ~0 lux environments. Most cameras will require exposure settings be optimized for a particular low-light application. When shutter speeds were normalized to 1/30s, SD D/N appeared to exhibit the best performance. 

In light of these findings we recommend the following:

  • Consider megapixel between 1MP to 2MP resolution. Higher megapixel may present only marginal gains in image quality, a diminished low-light sensitivity and a lower frame-rate.
  • SD may be considered for narrower FoV applications (e.g. ~16ft or less). SD will tend to provide slightly better low-light and bandwidth performance.
  • Consider positioning cameras 'downstream' from bright windows or light sources. If monitoring activity at a window is desired, however, consider a camera with capable WDR performance

Even Artificial Lighting (Doorway)

In this scenario cameras are digitally zoomed into a human subject ~28ft out, corresponding to a HFoV of ~26ft. Lighting is even and not particularly challenging (~200 lux).

Below is the Composite Snapshot (Download Full Video Clip [link no longer available]):

[link no longer available]

The majority of MP cameras in the composite (i.e. 5MP, 720p Avigilon/Sony) provide unequivocally greater detail levels on the human subject than the SD cameras. In these cameras' images identification is highly certain. Still, SD provides some key investigative details such as hair-style, build, complexion, and gender. Note that both the 720p M1114 and 1.3MP AV1315DN provide lesser image qualities than the other two 720p cameras, despite having equal horizontal resolutions.

Even Artificial Lighting (Far)

In this scenario cameras are digitally zoomed into a human subject ~33ft out, corresponding to a HFoV of ~31ft. Lighting is even and not particularly challenging (~200 lux).

Below is the Composite Snapshot (Download Full Video Clip [link no longer available]):

[link no longer available]

Results are similar to the 'doorway' counterpart, where three of five MPs (i.e. 5MP, 720p Avigilon, 720p Sony CH140) provide strong detail levels and strong odds of identification. SDs are considerably less detailed but still offer clues as to the subject's hair-style, build, complexion and gender. Note that in the SDs and two of the MPs  (1.3MP AV1315 and 720p Axis M1114) the presence of eye-glasses are not readily detectable.

Even Artificial Lighting (Near)

In this scenario cameras are digitally zoomed into a human subject ~17ft out, corresponding to a HFoV of ~16ft. Lighting is even and not particularly challenging (~200 lux).

Below is the Composite Snapshot (Download Full Video Clip [link no longer available]):

[link no longer available]

In this 'near' composite all cameras provide fair to high levels of detail, and subsequent moderate to high certainty levels of identification. The same three MPs (5MP, 720p Avigilon and 720 Sony CH140) provide the greatest level of details once again. Note that the 720p M1114 provides a lesser image quality than the other two 720p cameras, despite having equal horizontal resolutions.

Back-lit/WDR via Open Window (Doorway)

In this scenario cameras are digitally zoomed into a human subject ~28ft out, corresponding to a HFoV of ~26ft. The open window creates a lighting challenge with moderate WDR properties. Roughly ~1000 lux is measured at the window, while ~300 lux is measured at center room.

Below is the Composite Snapshot (Download Full Video Clip [link no longer available]):

[link no longer available]

In this backlit scene all cameras render the human subject's face somewhat under-exposed, leading to loss of contrast and overall clarity. The 720p Sony CH140 is the one exception which exposes the subject the best. Despite diminished picture qualities relative to even lit scenarios, the MPs (i.e. 5MP, 720p Avigilon and 720p Sony CH140) still provide moderate to strong odds of identification. Note that both the 720p M1114 and 1.3MP AV1315DN, despite having nearly twice the ppf, appear to perform only similarly to the SD Bosch NBN-498.

Back-lit/WDR via Open Window (Far)

In this scenario cameras are digitally zoomed into a human subject ~33ft out, corresponding to a HFoV of ~31ft. The open window creates a lighting challenge with moderate WDR properties. Roughly ~1000 lux is measured at the window, while ~300 lux is measured at center room.

Below is the Composite Snapshot (Download Full Video Clip [link no longer available]):

[link no longer available]

The challenges of back-lighting and WDR are most evident in this composite. All cameras show severely diminished levels of detail compared to the same 'far' scene of the even lit scenario. The 5MP and 720p Avigilon/Sony provide the best level of details, but likelihood of identification has degraded to fair/moderate levels. A majority of the MPs however, still provide other key investigative details regarding clothes, gender, build, complexion and eye-glasses. Note the 720p (Sony CH140) and SD Bosch NBN-498's ability to provide more appropriate exposure levels on the external side of the window. This could be a potential benefit in the detection of suspicious activity occuring outside (e.g. potential assault, vandalism or break-in).

Back-lit/WDR via Open Window (Near)

In this scenario cameras are digitally zoomed into a human subject ~17ft out, corresponding to a HFoV of ~16ft. The open window creates a lighting challenge with moderate WDR properties. Roughly ~1000 lux is measured at the window, while ~300 lux is measured at center room.

Below is the Composite Snapshot (Download Full Video Clip [link no longer available]):

[link no longer available]

Three of five MPs (i.e. 5MP, 720p Avigilon, and 720 Sony) provide strong enough detail levels for high certainty identification. The remaining MPs and both SDs are considerably 'blurry' and somewhat under-exposed.

Front-lit via Open Window (Doorway)

In this scenario cameras are digitally zoomed into a human subject ~28ft out, corresponding to a HFoV of ~26ft. The room is illuminated with natural light provided by the open window. Note that the open window has front-lit the room due to positioning cameras at the opposite end - 'down-stream' of light source. As a result lighting is relatively even (~300 lux) and does not present a challenge.

Below is the Composite Snapshot (Download Full Video Clip [link no longer available]):

[link no longer available]

The 5MP, 720p Avigilon and 720p Sony CH140 provide the greatest level of details and fairly strong certainties of identification. The 1.3MP, 720p Axis M1114 and SDs provide similar levels of detail, which are not sufficient for strong odds of identification.

Front-lit via Open Window (Far)

In this scenario cameras are digitally zoomed into a human subject ~33ft out, corresponding to a HFoV of ~31ft. The room is illuminated with natural light provided by the open window. Note that the open window has front-lit the room due to positioning cameras at the opposite end - 'down-stream' of light source. As a result lighting is relatively even (~300 lux) and does not present a challenge.

Below is the Composite Snapshot (Download Full Video Clip [link no longer available]):

[link no longer available]

None of the cameras appear to provide an exceptional level of detail in this composite. The 5MP, 720p Avigilon and 720p Sony provide the best level of details, but do not result in near or absolute certainty of identification.

Front-lit via Open Window (Near)

In this scenario cameras are digitally zoomed into a human subject ~17ft out, corresponding to a HFoV of ~16ft. The room is illuminated with natural light provided by the open window. Note that the open window has front-lit the room due to positioning cameras at the opposite end - 'down-stream' of light source. As a result lighting is relatively even (~300 lux) and does not present a challenge.

Below is the Composite Snapshot (Download Full Video Clip [link no longer available]):

[link no longer available]

Strong detail levels are provided by all MP cameras (both higher MP and 720p), making identification highly likely. The one exception is the 720p Axis M1114 which is more comparable to the SD cameras in this composite. Despite a somewhat weaker picture quality, they still provide a reasonably good level of detail and chance at identification.

Low-light (Default Settings)

In this composite, cameras are left at their full field of view (no digital zoom). Light levels are measured ~0.1 lux in this scene. Note that all cameras are set to default settings.

Below is the Composite Snapshot (Download Full Video Clip [link no longer available]):

[link no longer available]

With default settings applied, most cameras in the composite provide some measure of utility after low-light adaptation. The SD Bosch NBN-498 clearly provides the best overall picture quality amongst them. The exposure maximums of the 720p Sony CH140 and 720p Avigilon are among the lowest (1/30s), and, not surprisingly, exhibit considerably under-exposed images.

Low-light (Normalized)

In this composite, cameras are left at their full field of view (no digital zoom). Light levels are measured ~0.1 lux in this scene. Note that all cameras have been 'normalized' to a 1/30s max exposure time.

Below is the Composite Snapshot (Download Full Video Clip [link no longer available]):

[link no longer available]

Interestingly, cameras on the top row (5MP Axis P1347, 720p Axis M1114, and SD Axis M1011), which all featured 1/6s max shutter speeds, have all degraded into essentially 'black' images, since normalizing to 1/30s. The SD Bosch NBN-498 continues to provide the best apparent illumination even after the normalization process. Note that the 720p Avigilon and 720p Sony CH140 have default max shutters of 1/30s, and as such underwent no changes in the normalization scenario.

Low-light Default vs. Normalized

In this comparison images from each of the seven (7) cameras are arranged side by side with default settings (left) vs. normalized settings (right). The light levels are the same for either side (~0.1 lux).

Below is the Composite Snapshot:

[link no longer available]

Bandwidth Analysis

In this section, we examine the comparative bandwidth consumption of each camera, explaining variances in results and key issues.

In the even lighting scene, bandwidth consumption is fairly similar as the scene has minimal motion and no lighting challenges. Keep this in mind when we contrast this to other scenes.

[link no longer available]

In the front-lit scene, we have the lowest overall bandwidth consumption of the all the scenes. Our hypothesis is that camera bandwidth consumption is minimized when the lighting is quite level. To the contrary, note the jump in Arecont's bandwidth. This will be a consistent (and increasing problem) for the Arecont.

[link no longer available]

In the backlight / WDR scene, all of the camera's bandwidth consumption remains relatively the same except for the Arecont which jumps even higher.

[link no longer available]

Here in the low light scene is where things get really interesting. The Arecont, Axis and Avigilon cameras all show steep jumps in bandwidth consumption (despite the fact that Arecont and Axis frame rate drops significantly). Key driving factor is the hard to compress visual noise from aggressive gain control. Since all of these cameras are VBR, the bitrate surges. While the Sony also displays significant visual noise, as it uses constant bit rate encoding, no change occurs. While the Bosch's rate control allows the bandwith to rise it does not because of strong low light performance with minimal noise.

[link no longer available]

In this final scene, we kept the low light levels the same but normalized all the cameras to a 1/30s exposure (note: the Avigilon and Sony default to 1/30s, all others were changed). The Arecont bandwidth consumption increases even higher (to scary levels) as well as the Axis SD (in color mode with high visible noise). Interestingly, though, the Axis MP cameras bandwidth drops to nearly zero as the scene is pitch black.

[link no longer available]

Now we can look at the average bandwidth levels across scenes. The pattern is obvious - low light bandwidth consumption is nearly triple that of daytime bandwidth consumption.

[link no longer available]

Finally, we can see the average bandwidth consumption by camera:

 

  • Arecont consumes the most - driven by its challenges in handling lighting variations.
  • Avigilon is the next highest - though the image quality was comparatively strong throughout
  • The SD Axis came in 3rd highest mainly do to challenges in handling challenging light conditions
  • Using the default 2Mb/s bandwidth level, Sony delivered very consistent bandwidth consumption.
  • The 5MP Axis was the second lowest. However, its max frame rate was only 40% of the other cameras (12 vs 30fps) so direct comparison is misleading.
  • The MP Axis camera's bandwidth consumption was helped by dropping their bandwidth consumption to near zero in the normalized test. On the other hand, the Avigilon and the Sony consumed more bandwidth but provided at least some details in its video (making them more usable than the MP Axis cameras).

 

[link no longer available]

Methodology

The following are the seven (7) cameras used in the 'Office Shootout':

  • Arecont AV1315DN (online $460) - 1.3MP D/N; 1/2.7" CMOS; F1.6; 0.1 Lux (BW)
  • Avigilon 1.0MP-HD-H264-B1 (MSRP $645) -720p D/N; 1/2.7" CMOS; F1.6; 0.4/0.04 Lux (Color/BW)
  • Axis M1011 (online $160) -  4CIF Color-only; 1/4" CMOS; Integrated F2.0; 1.0 Lux (Color)
  • Axis M1114 (online $475) - 720p D/N; 1/4" CMOS;
  • Axis P1347 (online $1350) - 5MP D/N; 1/2.5" CMOS; F1.6 Kowa;0.08 Lux (BW)
  • Bosch NBN-498 [link no longer available] (online $700) - SD D/N; 1/3" CCD; F1.3 Bosch; 0.06 Lux (BW)
  • Sony CH140 (online $800) - 720p D/N; 1/3" CMOS; F1.2 Fujinon; 0.1 Lux (BW)

Each of the seven (7) cameras were simultaneously recorded to an Exacqvision VMS during five (5) key scenarios. All cameras were set to their defaults during testing, except during the low-light tests. During low-light tests D/N cameras were 'forced' into B/W to ensure consistency of comparison. Furthermore, cameras were normalized to 1/30s shutter speed for one of the tests. All camera lenses were adjusted to a uniform lens angle (~50 degrees) and recorded/analyzed at their - VMS allowed - maximum resolutions.

Here are the key default settings and firmware versions for each camera:

  • Arecont 1315DN - Automatic Exposure 'Balanced'; fw 65197
  • Avigilon 1.0MP - 1/30s maximum exposure; fw 1.0.4.4(3305)
  • Axis M1011 - 1/6s max exposure w/ AGC; fw 5.00.1
  • Axis M1114 - 1/6s max exposure w/ AGC; fw - 5.09
  • Axis P1347 - 1/6s max exposure w/ AGC; fw 5.20
  • Bosch NBN-498 -  1/60s shutter; SensUp 4x; AGC on '15'; fw 01500411
  • Sony CH140 - 1/30s shutter; View-DR/VE On; AGC 'Middle'; fw 1.26.01

Here are the five (5) key scenarios examined:

  • Even Artificial Lighting
  • Back-lit/WDR via Open Window
  • Front-lit via Open Window
  • Low-Light w/ Default Settings
  • Low-Light w/ Normalized Settings

All five (5) scenarios, employed the same choreagraphed activity. The activity involves, at the start, one human subject entering the room from a doorway (center room) and another subject positioned at the far corner of the room. Both subjects progress toward the center of the room and then toward nearer field of views until finally exiting scene.

 

 

4 reports cite this report:

Directory of Camera Shootout Series on Dec 27, 2011
The following directory lists all of our camera shootouts. These shootouts...
Top 12 Reports for 2011 on Dec 25, 2011
Here are our top 12 reports for 2011, starting with the state of the...
Are You Smarter than a Security Trade Magazine? on Aug 25, 2011
If you have any IP video surveillance skills, you should easily spot at least...
The Truth About IR Lighting and Bandwidth on Aug 13, 2011
One of the industry's top growing myths is that IR illumination massively...
Comments : Members only. Login. or Join.

Related Reports

Seek Scan Thermal Temperature Screening System Tested on May 28, 2020
Now that IPVM has tested Dahua, Hikvision, and Sunell, we are returning to...
Remote Network Access for Video Surveillance Guide on Jul 27, 2020
Remotely accessing surveillance systems is key in 2020, with more and more...
Integrated IR Camera Shootout 2020 - Avigilon, Axis, Bosch, Dahua, Hanwha, Hikvision, Panasonic, Uniview, Vivotek on Jan 30, 2020
The best and worst cameras tested in this IPVM shootout showed major...
Directory of 201 "Fever" Camera Suppliers on Aug 04, 2020
This directory provides a list of "Fever" scanning thermal camera providers...
Video Surveillance Cameras 101 on Feb 25, 2020
Cameras come in many shapes, sizes and specifications. This 101 examines the...
Video Analytics 101 on Mar 16, 2020
This guide teaches the fundamentals of video surveillance...
Camera Course Summer 2020 - Last Chance on Jul 18, 2020
This is your last chance to register for the Summer 2020 Camera Course. This...
IronYun AI Analytics Tested on Feb 17, 2020
Taiwan / US startup IronYun has raised tens of millions for its "mission to...
Use Access Control Logs To Constrain Coronavirus on Apr 09, 2020
Access control users have included capabilities that are not commonly used...
YCombinator AI Startup Visual One Tested on Apr 02, 2020
Startup Visual One, backed by Silicon Valley's powerful Y Combinator, aims to...
USA ICI Elevated Skin Temperature Detectors Examined on Apr 06, 2020
Infrared Cameras, Inc. (ICI) is aiming to help slow the spread of COVID-19...
Cheap Camera Problems at Night on Feb 19, 2020
Cheap cameras generally have problems at night, despite the common perception...
Mobotix Thermal Detection Camera Tested on Jun 09, 2020
For years Mobotix has struggled but now sales are surging driven by Mobotix's...
Mobotix 7 Line Camera Tested on Mar 12, 2020
Mobotix is attempting a turn-around, struggling for years, then releasing the...
K3 Wall Mounted IR Temperature Gun Tested on Jun 12, 2020
Is this the solution to not spending thousands of dollars on fever...

Recent Reports

Genetec CEO Declares "We Don't Negotiate Payment With Patent Trolls" on Aug 11, 2020
Are patent trolls like terrorists? Genetec's CEO is coming out strongly...
Hanwha AI Analytics Camera Tested on Aug 11, 2020
Hanwha has released their Wisenet P AI camera, adding person and vehicle...
Alabama Schools Million Dollar Hikvision Fever Camera Deal on Aug 11, 2020
The Baldwin County, Alabama public schools purchased a $1 million, 144-camera...
Dahua Taunts Australian Government, Continues To Sell Illegal Fever Cameras on Aug 10, 2020
Dahua is effectively taunting the Australian government by continuing to sell...
HID Releases VertX Replacement Aero on Aug 10, 2020
HID is replacing two established and broadly supported types of access...
NDAA Compliant Video Surveillance Whitelist on Aug 10, 2020
This report aggregates video surveillance products that manufacturers have...
Telpo China Temperature Tablets Tested on Aug 10, 2020
The provider for overseas companies ranging from Canon Singapore to US'...
Dangerous Hikvision Fever Camera Showcased by Chilean City on Aug 07, 2020
Deploying a fever camera outdoors, in the rain, with no black body, is...
"Grand Slam" For Pelco's PE Firm, A Risk For Motorola on Aug 07, 2020
The word "Pelco" and "grand slam" have not been said together for many years....
FLIR Stock Falls, Admits 'Decelerating' Demand For Temperature Screening on Aug 07, 2020
Is the boom going to bust for temperature screening? FLIR disappointed...
VSaaS Will Hurt Integrators on Aug 06, 2020
VSaaS will hurt integrators, there is no question about that. How much...
Dogs For Coronavirus Screening Examined on Aug 06, 2020
While thermal temperature screening is the surveillance industry's most...
ADT Slides Back, Disappointing Results, Poor Commercial Performance on Aug 06, 2020
While ADT had an incredible start to the week, driven by the Google...
AHJ / Authority Having Jurisdiction Tutorial on Aug 06, 2020
One of the most powerful yet often underappreciated characters in all...
SIA Coaches Sellers on NDAA 889B Blacklist Workarounds on Aug 05, 2020
Last month SIA demanded that NDAA 899B "must be delayed". Now that they have...