How Should Video Management Software be Evaluated?

By: John Honovich, Published on Dec 27, 2009

In this report, we propose criteria to evaluate Video Management Software (VMS). Those selecting VMS software may find this useful. Moreover we plan to use this criteria for our upcoming testing series on VMS software (expands our ongoing IP Video Surveillance product testing program) .

We encourage comments, feedback and discussion on these criteria.

Goals of Our Evaluation

We believe VMS software evaluations should:

  • Focus on users rather than features: The criteria should examine how well the VMS software meets the goals of real users (conduct an investigation, review an alert) rather than simply list technical features.
  • Educate rather than rank: The evaluation should help users better understand how the software works rather than simply providing scores that obscure details and differences.
  • Show Usability: What is most hard to judge about VMS software from marketing material is their usability (see the widespread unhappiness in our recent discussion on VMS usability). All VMS software have dozens of features (or more) but which are the most complete, are the easiest to use, etc.
  • General rather than specific to an application: Since our results will be read by thousands of people around the world, we need to make it general enough so that many readers can make use of it. For a specific deployment, we would certainly supplement this with specific questions for the user's application.

The common 'trick' in evaluations designed by vendors is to take their data sheet and create a checklist that matches the VMS's proprietary implementation and to judge other systems by that criteria. By contrast it is critical that our evaluation is independent and user focused.

Criteria/Questions

We propose the following questions be used (with some notes on issues involved):

Basic Use (Live and Recorded Video)
  • How do I watch live video? (There are surprising variances in the complexity and options even in watching 1 or a few cameras)
  • What ways can I monitor many video feeds simultaneously?
  • How can I be notified and review alarms?
  • How do I search for recorded video? (The variances are tremendous amongst systems)
  • What ways can I search multiple video feeds?
  • How do you share images from video?
  • How do you share video clips from recorded video? 
  • How do I know if I am in live or recorded video mode? (This is often difficult to do in VMS software which is both sad and confusing)
  • How do you switch between live video, investigations and configuration? (Some systems require multiple applications, others are integrated into a single client, etc.)

Administration
  • How do I add/setup a camera?
  • How do you set up alarms to be displayed? (Some systems are very complex in setting up alarms)
  • How do you restrict access to the system? 
  • How do you restrict access to certain features or functionalities?
  • How can you setup the VMS to handle failures to servers/appliances?
  • How can you access reports or alerts on the health status of the servers/appliances/applications?
  • How can you upgrade the VMS software?

Usability
  • How does the VMS handle the higher resolution of megapixel cameras?
  • How does the VMS perform with low bandwidth connection (256 kb/s)? (Many VMS users are remote)
  • What, if any, VMS functionalities regulary take more than 10 seconds to load? (When software is unresponsive, users are less likely to use it and be more unsatisfied)
  • What happens when the VMS client cannot connect to a camera? (Since IP cameras can drop frequently, this can become a major source of frustration depending on how the VMS system handles it)
  • What options does the VMS provide to reduce storage consumption?
  • What help functionality does the VMS provide while using the client?
  • How similiar is the use of the VMS client to a DVR? (This is important to most security operators and guards who have limited technical training and prefer simple systems)
  • How similar is the configuration of the VMS server to a DVR? (This is important to traditional security integrators who often have limited IT expertise)
  • How simple is it for an untrained operator? (Many systems have impressive feature offerings but are extremely difficult for untrained operators to understand without formal training)
  • How powerful is it for the expert operator? (Some systems are quite limited in advanced functionality)

Scalability
  • What ways can I add/setup many cameras at the same time?
  • How do you do restrict access when using multiple servers/appliances? (This is a major differentiator amongst systems and a core requirement for larger, multi-site deployments)
  • How can I watch live video feeds from different appliances/servers?
  • How can I conduct searches from video feeds stored in different appliances/servers?

In this criteria, we omitted 3rd party support for cameras, security, business and storage systems. As a practical matter, we will unlikely be able to test these comprehensively. However, we will factor all of those into the overall report. Finally, pricing will certainly be included, allowing us to evaluate overall value comparing performance to price.

Our Tests vs Your Own Tests

We expect our tests to provide a low cost way to understand key comparative strengths and benefits of various VMS systems. Testing numerous VMS systems oneself is impractical for most, simply because of the cost and time involved (which can easily be thousands of dollars). We can provide this for a fraction of the cost. Moreover, because we focus solely on video surveillance, we can identify key issues and comparative concerns easily.

We do recommend that end users and integrators tests the systems themselves that they plan to roll out. Our tests can help you determine what the 'short list' of vendors should be and provide basis for your own testing for your finalists. 

Feedback

1 report cite this report:

Readers Respond: 134 Votes, 25 Comments Examined on Feb 14, 2010
Over 100 people voted and more than 25 left detailed comments to our recent...

Related Reports

Convergint Coronavirus Cuts on Mar 25, 2020
One of the world's largest security integrators, Convergint, has made a major...
Directory of 180 "Fever" Camera Suppliers on Jul 09, 2020
This directory provides a list of "Fever" scanning thermal camera providers...
Converged vs Dedicated Networks For Surveillance Tutorial on Feb 12, 2020
Use the existing network or deploy a new one? This is a critical choice in...
Use Access Control Logs To Constrain Coronavirus on Apr 09, 2020
Access control users have included capabilities that are not commonly used...
Low Voltage Nation Wants to "Help You Carve Out A Fulfilling Career" Interviewed on Jul 06, 2020
It is difficult to make your way in this industry as there is little formal...
Injes Tiny Temperature Terminal Tested on Jul 17, 2020
While temperature terminals have trended bigger, the Injes DFace801 is...
Avigilon Open Analytics Tested on Apr 16, 2020
After years of effectively closed analytics, Avigilon decided in late 2018 to...
Spectron IR Thermal Fever Screening System Examined on Apr 14, 2020
Most are quick to avoid "fever screening" and "medical" labels, but...
Disaster Relief Loans For Integrators Examined on Mar 23, 2020
Integrators are being impacted by the coronavirus. Now, the US government is...
Disruptive Free Lead Generation Added To IPVM on May 15, 2020
IPVM has added lead generation for sellers, for free, disrupting the...
FDA Gives Guidance on 'Coronavirus' Thermal Fever Detection Systems on Mar 30, 2020
The US FDA has given IPVM guidance on the use of thermal fever detection...
London Live Police Face Recognition Visited on Feb 13, 2020
London police have officially begun using live facial recognition in select...
Meditherm Medical Thermal Cameras Examined on May 07, 2020
US-based Meditherm emphasized to IPVM its 20+ year development history and...
Help Security End Users Facing Coronavirus Improve Remote Access on Mar 24, 2020
Many end-users and integrators are struggling with the impact of coronavirus...
Facial Recognition 101 on Mar 18, 2020
Facial recognition interest, use and fear is increasing. This guide aims to...

Recent Reports

Google Invests in ADT, ADT Stock Soars on Aug 03, 2020
Google has announced a $450 million investment in the Florida-based security...
US Startup Fever Inspect Examined on Aug 03, 2020
Undoubtedly late to fever cameras, this US company, Fever Inspect, led by a...
Motorola Solutions Acquires Pelco on Aug 03, 2020
Motorola Solutions has acquired Pelco, pledging to bring blue back and make...
False: Verkada: "If You Want To Remote View Your Cameras You Need To Punch Holes In Your Firewall" on Jul 31, 2020
Verkada falsely declared to “3,000+ customers”, “300 school districts”, and...
US GSA Explains NDAA 889 Part B Blacklisting on Jul 31, 2020
With the 'Blacklist Clause' going into effect August 13 that bans the US...
Access Control Online Show July 2020 - On-Demand Recording of 45+ Manufacturers Presentations on Jul 30, 2020
The show featured 48 Access Control presentations, all now recorded and...
Face Detection Shootout - Dahua, Hanwha, Hikvision, Uniview, Vivotek on Jul 30, 2020
Face detection analytics are available from a number of manufactures...
Sunell is The First China Manufacturer to Market NDAA Compliance on Jul 30, 2020
Most China manufacturers are going to be impacted by the NDAA 'Blacklist...
Ink Labs Relabels China YCX Fever Camera And Steals Dahua's Marketing on Jul 30, 2020
A US company marketed a 'thermal temperature scanner' as its own, selling...
Genetec and Dahua-Backed Intelbras Split Examined on Jul 29, 2020
China is the cause of the breakup between Canada's and Brazil's largest video...
This YouTuber is Now Selling ThermoHealth Temperature Screening on Jul 29, 2020
An enterprising 20-year old is mass marketing medical devices on Facebook and...
Hikvision Returns To Growth Driven By Overseas Fever Cameras on Jul 29, 2020
While Hikvision's revenue fell in Q1 2020, it rebounded in Q2 attributed to...
Brazil's Biggest Domestic Surveillance Company Intelbras Profile on Jul 29, 2020
While Intelbras is not widely known outside of Latin America, Intelbras is a...
The Kiosk Market Pivots To Temperature Screening (Interviewed) on Jul 28, 2020
Video surveillance is not the only market that has pivoted to medical device...
Integrator Acquisitions 'A Good Market' During COVID-19, Says Greybeards on Jul 28, 2020
Industry broker Ron Davis of the "Greybeards" says that the integrator and...