Optimized Camera Shootout - Upcoming and Released

By: John Honovich, Published on May 16, 2011

The response to our camera shootout series has been phenomenal with more than 10,000 reads in the last month and growing.

The main question or concern raised is our use of camera setting defaults. A number of members raised the point that results could be better (potentially significantly) better if the settings were tweaked for the environment.

[UPDATE: The Optimized Camera Shootout results are released.]

On the one hand, we want tests to be fair and representative of common real world use. Starting with the defaults shows what every camera delivers 'out of the box'. On the other hand, some deployments and some cameras may provide better results with a change of settings specific for the environment.

To test such a scenario, we are launching an 'Optimized Camera Shootout' for our Parking Garage scene. Here's how it is going to work:

  • We will email technical representatives of each manufacturer included in the Parking Garage shootout. They can review the complete video clips from the original test. Each manufacturer can then submit what software or hardware settings they would recommend for that scene.
  • By May 26th, all manufacturers must respond with their recommendations.
  • On May 31st, we will re-do the Parking Garage scene using the recommended settings for each manufacturer.
  • On June 6th, we will release the 'Optimized Camera Shootout' results (1)  demonstrating the differences between default and optimized settings for each manufacturer and (2) the relative performance of optimized settings across all manufacturers.

We are choosing the Parking Garage scene as the first 'optimization' test because the scene has significant lighting challenges that may respond well to optimization.

One important rule for this test is that the settings choosen cannot be changed during the course of the test. For instance, we will not adjust the aperture on a manual iris camera to adjust for day vs night conditions nor manually change exposure zones for different times of the day. All settings will be applied at the start of the test and will be used throughout. This is important to ensure that we are not 'gaming' the test. While such 'tricks' are common in sales demos, in the read world, settings are not changed manually during the day. Nor will it be in our tests.

Please let us know if you have any questions or feedback.

Comments : Members only. Login. or Join.

Related Reports

2020 Camera Book Released on Jan 10, 2020
This is the best, most comprehensive security camera training in the world, based on our unprecedented testing. Now, all IPVM PRO Members can get...
Milestone Machine Learning Camera Auto-Setting Examined on Mar 13, 2019
Milestone wants to improve image quality using Machine Learning to solve the problem of "a camera doesn't know what it is being used for",...
Startup Spectral Edge RGB+IR Claims "Unique Low Light Performance Tech" on Apr 03, 2019
This UK startup, born out of university research, is boldly proclaiming that they deliver "Unique low light performance tech for video surveillance...
The Embarrassing Story of ISC West's Best New IP Camera on Apr 24, 2019
A sad but simple situation: Only 2 companies paid SIA the thousands of dollars required to compete for the best new 'cameras IP' The judges...
Manufacturer Favorability Guide 2019 on Jun 12, 2019
The 259 page PDF guide may be downloaded inside by all IPVM members. It includes our manufacturer favorability rankings and individual...
False Verkada 'Unrivaled' Low Light Performance Claim Removed on Jun 12, 2019
Verkada falsely claimed that it delivered 'UNRIVALED LOW LIGHT PERFORMANCE' until IPVM questioned. In fact, Verkada's low light performance is...
Lens Iris Tutorial on Aug 08, 2019
Cameras, like humans, have irises, controlling the light which hits the imager and impacting image details. However, cameras have multiple types of...
Hikvision ColorVu is Smart Marketing on Oct 03, 2019
Hikvision ColorVu (see IPVM test results) is smart marketing, a lesson to be learned by competitors and a rising trend. Inside this note, we...
Biggest Low Light Problems 2019 on Nov 08, 2019
Over 150 integrators responded to our survey question: "What are the biggest problems you face getting good low-light images?" Inside, we share...
IPVM Opens 12,000 Sqft Testing Facility on Dec 16, 2019
IPVM is proud to announce the opening of the world's first video surveillance testing facility that will allow us to significantly expand our...

Most Recent Industry Reports

Hazardous & Explosion Proof Access Control Tutorial on Feb 27, 2020
Controlling access to hazardous environments requires equipment meeting specific ratings that certify they will not start fires or will not...
Motorola / Avigilon Drops ISC West on Feb 26, 2020
Motorola Solutions has pulled out of ISC West 2020 effective immediately, because of coronavirus concerns, IPVM has learned. This is done amidst...
Cancel or Not? Industry Split Over ISC West on Feb 26, 2020
The industry is split, polarized, over whether ISC West 2020 should run or be canceled. New IPVM survey results of 400+ respondents show heated...
Coronavirus Hits Sony, Bosch Says Switch on Feb 26, 2020
Sony's fall in video surveillance has been severe over the past decade. Now, they may be done. In this note, we examine Bosch's new...
Video Surveillance Cameras 101 on Feb 25, 2020
Cameras come in many shapes, sizes and specifications. This 101 examines the basics of cameras and features used in 2020. In this report, we...
Favorite Video Analytic Manufacturers 2020 on Feb 25, 2020
Video analytics is now as hot as ever, driven by the excitement of advancing deep learning offers. But what are actually integrator's...
Latest London Police Facial Recognition Suffers Serious Issues on Feb 24, 2020
On February 20, IPVM visited another live face rec deployment by London police, but this time the system was thwarted by technical problems and...
Masks Cause Major Facial Recognition Problems on Feb 24, 2020
Coronavirus is spurring an increase in the use of medical masks, which new IPVM test results show cause major problems for facial recognition...
Every VMS Will Become a VSaaS on Feb 21, 2020
VMS is ending. Soon every VMS will be a VSaaS. Competitive dynamics will be redrawn. What does this mean? VMS Historically...
Video Surveillance 101 Course - Last Chance on Feb 20, 2020
This is the last chance to join IPVM's first Video Surveillance 101 course, designed to help those new to the industry to quickly understand the...