How We Review Products

By: John Honovich, Published on Jul 14, 2011

IP Video Market Info has a specific style of reviewing products that often conflicts with common review practices. In this report, we explain how and why we review products the way we do.

Principles of IPVM Review

Let's start with the key principles of IPVM product reviews that differentiate our process:

  • Avoid any category benefits in reviews
  • Reviews must be comparative
  • Reviews must factor in cost prominently
  • Reviews must weigh feature's real world benefits
  • Reviews must recommend applications

Equally important, since our reviews are used by thousands around the world, we cannot assume what the specific use will be.

Avoid Category Benefits

Bad reviews heap praise on a product for features and benefits that any product in its category has. For instance, complimenting an IP camera's capability for viewing via a web browser is inane. Essentially all IP cameras do this and have for years. In the same way, complimenting a VMS for searching by date and time is ridiculous for the same reason.

Each product category has a core set of features that are common across the board. The presence of any of these features should be ignored as it provides no differentiating value.

Comparative Reviews

To determine how different a product is, it is essential to compare to other products in its category. At a minimum, at least 2 other products need to be selected to provide context for the product being reviewed. Saying something is 'good' or 'bad' without providing a comparison is meaningless and misleading. It might make slick marketing but it creates poor analysis.

One basic technique to providing comparisons is to pick one rival product with more feature sets and another rival with less feature sets. A more advanced technique would compare a rival with very similar feature sets, examining the subtle differences between the products.

Get Notified of Video Surveillance Breaking News
Get Notified of Video Surveillance Breaking News

Reviewing Cost

One of the most important elements in comparing products and performing reviews is assessing the cost of a product. Unfortunately, most product reviews ignore cost or, at best, mention it in the review's addendum.

Cost is a critical component. Reviews that say one product is the 'best' while ignoring the expense of the product does a dis-service to a reader.

Equally important, the review must relate the cost of the product to the value provided. A product that costs $100 less but is missing a critical feature may be a negative or at least a problem for a certain portion of the market.

Real World Benefits

All features are not beneficial and many 'features' are just marketing spin. A common technique by manufacturers is to create rigged feature comparisons where they take a list of their features and then rate everyone else accordingly.

Even among features that are worthwhile, some features are more valuable than another. Let's say 2 cameras were exactly the same, except one had WDR and D/N capability while the other had SNMP and WiFi support. The overwhelmingly majority of real world users care more about the former than the later.

The review should highlight the most important real world benefits that distinguish a product from its main competitors.

Recommended Applications

Finally, the review should recommend specific applications that given the cost, feature sets provided and competitive alternatives, that the product is most likely to be beneficial. Applications could be scenario based (e.g., low light, non-technical users) or market segment based (e.g., small box retailer, campuses). In a market as competitive and fragmented as video surveillance, a product is almost never best for everyone. The reviewer should clearly indicate both the best and worst applications.

Summary

With these guidelines in mind, a complete review should answer:

  • What products would you compare the reviewed product to?
  • What real world feature sets/benefits differentiate it from those products?
  • What is the price of the product and how does it compare to rivals?
  • Relative to its rivals, what are the best and worst applications to use this product?

Related Reports

Coronavirus Hits Sony, Bosch Says Switch on Feb 26, 2020
Sony's fall in video surveillance has been severe over the past decade. Now, they may be done. In this note, we examine Bosch's new...
Video Surveillance 101 Course - Last Chance on Feb 20, 2020
This is the last chance to join IPVM's first Video Surveillance 101 course, designed to help those new to the industry to quickly understand the...
2020 Camera Book Released on Jan 10, 2020
This is the best, most comprehensive security camera training in the world, based on our unprecedented testing. Now, all IPVM PRO Members can get...
2020 IP Networking Book Released on Jan 06, 2020
The new IP Networking Book 2020 is a 280 page in-depth guide that teaches you how IT and telecom technologies impact modern security systems,...
3 Weeks Later, Honeywell Still Cannot Say Whether They Are Vulnerable To Dahua Wiretapping [Now Admits] on Aug 27, 2019
The Dahua wiretapping vulnerability and Dahua's decision to delay disclosing it until IPVM inquired underscored problems with cybersecurity and...
Manufacturer Favorability Guide 2019 on Jun 12, 2019
The 259 page PDF guide may be downloaded inside by all IPVM members. It includes our manufacturer favorability rankings and individual...
Axis Will Not Block Resellers on Jun 10, 2019
While Axis generally has strong favorability amongst integrators, the biggest complaint is their channel model, which results in smaller integrator...
Closed Cloud Cameras Trashed on May 13, 2019
When you buy a camera, do you own it? Not anymore. In the world of closed cloud cameras, you may think you are buying a camera but all you are...
AI Video Tester Released on Apr 02, 2019
IPVM has released the world's first AI video tester that lets you see how various AI models (including from Amazon, Google, Microsoft and YOLO)...
Undercutting Partners, Arbitech Sells Millions In Unauthorized Axis on Mar 29, 2019
One of integrator's top complaints about Axis is poor margins. An enterprising, ethically questionable distributor is solving that. Inside this...

Most Recent Industry Reports

LIVE NOW "Fever Camera" Show on Jun 02, 2020
IPVM is excited for the world's first "Fever Camera" show, to be held today Tuesday, June 2nd and Wednesday the 3rd from 11am to 4pm EDT, giving...
Smart Entry Systems Presents Cloud Multi-Tenant Access Control on Jun 02, 2020
Smart Entry Systems presented Cloud Multi-Tenant Access Control at the May 2020 IPVM Startups show. Inside this report: A 30-minute video...
Genetec Drops Support for Dahua and Hikvision on Jun 01, 2020
Genetec has dropped support for Dahua and Hikvision, citing US blacklisting and ONVIF conformance blockage, the company informed partners in an...
Dotty "Hot Or Not" Elevated Body Temperature App Tested on Jun 01, 2020
What if you could take an existing phone or tablet and transform it into "fever camera"? That is what DottyAR is doing with their strangely named...
Optris "Fever Screening Systems" Examined on Jun 01, 2020
German manufacturer Optris has been building temperature measuring instruments for industrial manufacturing for over 15 years, and thermal cameras...
Fever Camera Sales From Integrators Surveyed on Jun 01, 2020
Fever cameras are the hottest trend in video surveillance currently but how much are integrators selling them? 220 integrators answered the...
Proxy Presents Mobile Credentials For BLE Devices and Access on May 29, 2020
Proxy presented Mobile Credentials For BLE Devices and Access at the May 2020 IPVM Startups show. Inside this report: A 30-minute video...
ISC West 2020 Moves To The Basement on May 29, 2020
The twice cancelled/postponed show will now not only be held in a different month (October) but on a different floor, moving down to the...
Integrators Avoiding Coronavirus Air Travel on May 29, 2020
IPVM asked integrators if air travel is part of their 2020 plans to see how significantly Coronavirus will impact future...
Viakoo Presents Cyber Hygiene for Cameras on May 28, 2020
Viakoo presented its 'Cyber Hygiene' and 'Service Assurance' products at the April 2020 IPVM New Products show. Inside this report: A...