IPVM Camera Testing Procedures

Author: John Honovich, Published on May 08, 2013

The foundation of IPVM's camera testing procedure is simultaneous, head to head comparisons, like so:

This is unique among surveillance industry publications.

Each camera is placed side by side, with the same FoV, connected to the same VMS and recorded at the same time and place. The setup looks like so:

We perform a few adjustments:

  • Firmware is upgraded to the most current version.
  • Minimum shutter speed is set to 1/30s to eliminate any slow shutter tricks.
  • Cameras are re-focused prior to testing.

However, unlike manufacturer demos, we do not 'tweak' cameras to optimize for individual scenes. Rather, we keep the same settings for all tests, to keep it fair, consistent and true to the most common real world uses.

We conduct tests in 3 fundamental areas:

  • Even light - this is the 'ideal' test when all cameras do the best.
  • Low light - measured at 0.2 lux
  • WDR - in an open door way back light

They look like so:

Get Video Surveillance News In Your Inbox
Get Video Surveillance News In Your Inbox

We typically run a series of pre-tests to get a feel for performance and identify any abnormalities. If the image quality looks excessively poor, we will re-check settings and/or request feedback from a manufacturer engineer. The multiple pre-test runs works out issues and increases the level of confidence that the test results accurately and fairly reflect performance.

Finally, we release test reports that are centered on a series of 5 - 15 image comparisons of multiple cameras head to head; again, like so:

The best way to understand relative performance is to properly setup and test simultaneously. Unlike looking at cameras individually, where it is easy to overestimate performance, the IPVM approach makes it clear which cameras have advantages in specific scenarios.

Bandwidth / Quantization / Compression

We typically measure bandwidth levels for each camera in each scenario. Some camera manufacturers attempt to game quantization / compression levels. We will measure those levels using a DRF analysis tool that allows us to understand if there is significant variances. This is especially important for manufacturers claiming to deliver super low bandwidth who typically do so by increasing compression levels but failing to disclose. We will report this and make adjustments in testing as appropriate.

Test Reports

See our directory of 160+ surveillance test reports. Recent favorites include our WDR Shootout, Panoramic Shootout and Axis vs Bosch Low Light Shootout.

Feel free to ask questions and we are happy to expand on this reference document.

Comments (11): PRO Members only. Login. or Join.

Related Reports

Glass Break Sensor Tutorial on Mar 21, 2017
Burglars often break glass windows to get into a house. Using glass break detectors in conjunction with alarm contacts is a good way to protect the...
Alarm Panic Switches Tutorial on Mar 16, 2017
Panic switches allow silently triggering an alarm system when it is otherwise disarmed. In this tutorial we explain and contrast the 7 most common...
Environmental Sensors For Burglar Alarm Tutorials on Mar 15, 2017
Intrusion detection systems can be used to alert users to environmental issues. By connecting sensors that monitor the heat, humidity, and...
Alarm Contacts Tutorial on Mar 07, 2017
Contacts are the devices that tell an alarm panel when an entryway is open or closed. The act of opening or closing an entryway, such as a door or...
Alarm Motion Detector Tutorial on Mar 06, 2017
Sensors designed to detect movement are a good way to protect large, open areas. In this tutorial, we examine motion detectors,...
TVI 4.0 Doubles HD Analog Bandwidth on Mar 05, 2017
HD analog's move up market continues. Starting a few years ago at just 720p, HD analog is now poised for 4K and beyond. Techpoint, the company...
Uniview (UNV) IP Cameras Tested on Feb 22, 2017
"We're #3," in China says Uniview (UNV). While the company significantly trails Hikvision and Dahua in total sales, one notable difference is that...
Glass Doors and Access Control Tutorial on Feb 22, 2017
The biggest challenge for many access control systems are glass doors. Here's what happens when a maglock is improperly installed to an existing...
Arecont Vision MicroDome Duo Tested on Feb 14, 2017
Arecont Vision is back with another multi-head camera, this time thinking smaller with the MicroDome Duo, a two-imager model, with the tagline...
Alarm Panels - Canned vs Integrated on Feb 07, 2017
Alarm control panels generally come in one of two types: Integrated alarm controllers, with controls and the keypad in the same...

Most Recent Industry Reports

Broken Browser Support for Video Surveillance on Mar 22, 2017
Modern web browsers have left the security industry behind. Current Chrome, Firefox, and Microsoft Edge browsers do not support NPAPI plugins,...
ADI Favorability Results on Mar 22, 2017
150 North American integrators provided feedback on 6 distributors, and why they do (or do not do) business with ADI. ADI is clearly a big name in...
1 Million Dahua Devices Exposed To Backdoor on Mar 22, 2017
Statistics show that 1 million Dahua devices are publicly exposed and vulnerable to the Dahua backdoor. Despite this, Dahua has downplayed the...
Hikvision Hires Crisis Communication Writer on Mar 21, 2017
Hikvision has hired a crisis communication writer as the company ramps up its efforts to deal with the 'crisis' it feels it is facing. 'Crisis...
Glass Break Sensor Tutorial on Mar 21, 2017
Burglars often break glass windows to get into a house. Using glass break detectors in conjunction with alarm contacts is a good way to protect the...
Unikey Smart Phone Access Control Platform on Mar 21, 2017
More and more people carry smart phones. Many think this could replace the conventional key or card for access control. However, using a phone...
VMS Update Automation Compared on Mar 20, 2017
Updating VMS software can be a tedious and time consuming processing, which historically has required users to access each machine, download update...
Anvil Traveler Threat Management Offering on Mar 20, 2017
A common component of enterprise security systems is traveler threat management, complementing the various physical security systems in place. One...
Simplisafe Warns Customers About Alarm Fines on Mar 17, 2017
Simplisafe markets themselves as a 'better way' than traditional professional alarm companies. However, in one key way, Simplisafe hides the...

The world's leading video surveillance information source, IPVM provides the best reporting, testing and training for 10,000+ members globally. Dedicated to independent and objective information, we uniquely refuse any and all advertisements, sponsorship and consulting from manufacturers.

About | FAQ | Contact