How Should Video Management Software be Evaluated?

Author: John Honovich, Published on Dec 27, 2009

In this report, we propose criteria to evaluate Video Management Software (VMS). Those selecting VMS software may find this useful. Moreover we plan to use this criteria for our upcoming testing series on VMS software (expands our ongoing IP Video Surveillance product testing program) .

We encourage comments, feedback and discussion on these criteria.

Goals of Our Evaluation

We believe VMS software evaluations should:

  • Focus on users rather than features: The criteria should examine how well the VMS software meets the goals of real users (conduct an investigation, review an alert) rather than simply list technical features.
  • Educate rather than rank: The evaluation should help users better understand how the software works rather than simply providing scores that obscure details and differences.
  • Show Usability: What is most hard to judge about VMS software from marketing material is their usability (see the widespread unhappiness in our recent discussion on VMS usability). All VMS software have dozens of features (or more) but which are the most complete, are the easiest to use, etc.
  • General rather than specific to an application: Since our results will be read by thousands of people around the world, we need to make it general enough so that many readers can make use of it. For a specific deployment, we would certainly supplement this with specific questions for the user's application.

The common 'trick' in evaluations designed by vendors is to take their data sheet and create a checklist that matches the VMS's proprietary implementation and to judge other systems by that criteria. By contrast it is critical that our evaluation is independent and user focused.


We propose the following questions be used (with some notes on issues involved):

Basic Use (Live and Recorded Video)
  • How do I watch live video? (There are surprising variances in the complexity and options even in watching 1 or a few cameras)
  • What ways can I monitor many video feeds simultaneously?
  • How can I be notified and review alarms?
  • How do I search for recorded video? (The variances are tremendous amongst systems)
  • What ways can I search multiple video feeds?
  • How do you share images from video?
  • How do you share video clips from recorded video?
  • How do I know if I am in live or recorded video mode? (This is often difficult to do in VMS software which is both sad and confusing)
  • How do you switch between live video, investigations and configuration? (Some systems require multiple applications, others are integrated into a single client, etc.)

  • How do I add/setup a camera?
  • How do you set up alarms to be displayed? (Some systems are very complex in setting up alarms)
  • How do you restrict access to the system?
  • How do you restrict access to certain features or functionalities?
  • How can you setup the VMS to handle failures to servers/appliances?
  • How can you access reports or alerts on the health status of the servers/appliances/applications?
  • How can you upgrade the VMS software?

  • How does the VMS handle the higher resolution of megapixel cameras?
  • How does the VMS perform with low bandwidth connection (256 kb/s)? (Many VMS users are remote)
  • What, if any, VMS functionalities regulary take more than 10 seconds to load? (When software is unresponsive, users are less likely to use it and be more unsatisfied)
  • What happens when the VMS client cannot connect to a camera? (Since IP cameras can drop frequently, this can become a major source of frustration depending on how the VMS system handles it)
  • What options does the VMS provide to reduce storage consumption?
  • What help functionality does the VMS provide while using the client?
  • How similiar is the use of the VMS client to a DVR? (This is important to most security operators and guards who have limited technical training and prefer simple systems)
  • How similar is the configuration of the VMS server to a DVR? (This is important to traditional security integrators who often have limited IT expertise)
  • How simple is it for an untrained operator? (Many systems have impressive feature offerings but are extremely difficult for untrained operators to understand without formal training)
  • How powerful is it for the expert operator? (Some systems are quite limited in advanced functionality)

  • What ways can I add/setup many cameras at the same time?
  • How do you do restrict access when using multiple servers/appliances? (This is a major differentiator amongst systems and a core requirement for larger, multi-site deployments)
  • How can I watch live video feeds from different appliances/servers?
  • How can I conduct searches from video feeds stored in different appliances/servers?

In this criteria, we omitted 3rd party support for cameras, security, business and storage systems. As a practical matter, we will unlikely be able to test these comprehensively. However, we will factor all of those into the overall report. Finally, pricing will certainly be included, allowing us to evaluate overall value comparing performance to price.

Our Tests vs Your Own Tests

We expect our tests to provide a low cost way to understand key comparative strengths and benefits of various VMS systems. Testing numerous VMS systems oneself is impractical for most, simply because of the cost and time involved (which can easily be thousands of dollars). We can provide this for a fraction of the cost. Moreover, because we focus solely on video surveillance, we can identify key issues and comparative concerns easily.

We do recommend that end users and integrators tests the systems themselves that they plan to roll out. Our tests can help you determine what the 'short list' of vendors should be and provide basis for your own testing for your finalists.


We are happy to elaborate on proposed criteria, add additional criteria or simply discuss the key issues involved in evaluating VMS software.

1 report cite this report:

Readers Respond: 134 Votes, 25 Comments Examined on Feb 14, 2010
Over 100 people voted and more than 25 left detailed comments to our recent question/discussion. In this post, we review and respond to the...

Related Reports on VMS

Axis Video Revenue Down on Oct 20, 2016
An important milestone. Axis revenue for video products is down year over year. But Axis is now focusing on 'diversification'. In this report, we...
Worst Access Control 2016 on Oct 19, 2016
Two access control providers stood out as being the worst to work with for integrators. In this report, we analyze the answers to: "In the past...
Mobotix Founder Out, Company Embracing Open Platform on Oct 13, 2016
The founder of Mobotix, Dr. Ralf Hinkel, is out of Mobotix, just 7 months after the company was acquired by Konica-Minolta. With his departure,...
IndigoVision Going Head-To-Head Against Milestone, Genetec on Oct 12, 2016
Cameras are commodities, the money is in VMS. This is the new philosophy of IndigoVision, long known as a closed end-to-end systems provider. The...
VMS Installer Sizes Getting Out of Control on Oct 11, 2016
Huge VMS file sizes increase installation size and can cause problems in the field.  As VMSes become more mature and add features their file size...
Axis Employees Get $100 Bonus For Beating Out Avigilon on Oct 11, 2016
Axis offered its employees a $100 bonus for beating out Avigilon's VMS, according to an Axis communication obtained by IPVM. Axis...
How And When ONVIF Will Support H.265 on Oct 07, 2016
ONVIF Profile S does not support H.265. And Profile S is what literally every ONVIF device supports today for video. So what happens to H.265...
Axis Zipstream 2 Tested on Oct 03, 2016
Smart codecs are one of the big trends in the industry now. Axis was the first manufacturer to launch a smart codec, releasing Zipstream in Spring...
Hiring Spree At Aimetis 6 Months After Being Acquired on Sep 28, 2016
Aimetis was acquired in April 2016, and is now expanding almost all of their departments, hiring employees from Axis and other industry...
Axis Secretly Paid Anixter Sales People To Push Axis NVRs on Sep 26, 2016
Internal Axis communication shows how Axis paid Anixter and Tri-Ed sales people with secret bonuses to push Axis NVRs. In this report, we examine...

Most Recent Industry Reports

"WTF?!?!? Who is Brian Karas?!?" Exclaims Knightscope on Oct 21, 2016
Knightscope co-founder Stacy Stephens emailed us: He may not have intended to send it us and he probably can figure out who Brian Karas is,...
Security Consultants Speak Episode 1 - Protus3 on Oct 21, 2016
This is a first of a series of conversations with security consultants. If you are a security consultant that wants to talk and can share frank...
Sony and Samsung Breaking VBR on Oct 21, 2016
For years, users have known variable bitrate (VBR) as one thing only: bandwidth varies, compression stays the same. This is not an accident but an...
Pelco Matrix End Of Life - End Of An Era on Oct 20, 2016
Pelco Matrix switchers, once the pinnacle of large SD analog installations, are now literally impossible to build. The Matrix products were not...
Axis Video Revenue Down on Oct 20, 2016
An important milestone. Axis revenue for video products is down year over year. But Axis is now focusing on 'diversification'. In this report, we...
Worst Access Control 2016 on Oct 19, 2016
Two access control providers stood out as being the worst to work with for integrators. In this report, we analyze the answers to: "In the past...
Hacked DVRs Surge To 400,000 on Oct 19, 2016
The global internet is under attack from record breaking botnets. And it is getting worse, Mirai doubled in size in the last month. Shamefully,...
Longse Rips Off Hikvision on Oct 19, 2016
Longse is on the attack, and now they are targeting Hikvision. Evidently not content just to rip off Milestone and Video Insight, Longse has now...
China "Unswerving Leadership Over State-Owned Enterprises" Like Hikvision on Oct 18, 2016
The PR agency of the Chinese government declared: President Xi Jinping stressed the Communist Party of China's (CPC) unswerving leadership over...
Move Aside Cisco, Axis Has A Network Switch For Integrators on Oct 18, 2016
Cisco is a common choice for network switches, including in our Favorite Network Switches survey, but now Axis is releasing a 16 port PoE+ switch...

The world's leading video surveillance information source, IPVM provides the best reporting, testing and training for 10,000+ members globally. Dedicated to independent and objective information, we uniquely refuse any and all advertisements, sponsorship and consulting from manufacturers.

About | FAQ | Contact