Criteria for Evaluating VMS Systems

Author: John Honovich, Published on May 24, 2011

For our VMS Competitive Comparison, we identified 9 key differentiators to contrast the abilities of Video Management Systems. In this update, we explain how we ranked each differentiator.

Inside the competitive comparison itself, we provide a 23 minute webcast that explains each one. This update provides a written summary of the criteria.

Levels

We assigned 6 levels for each differentiator:

  • None: System does not support
  • Weak: System supports but is of very limited use
  • Moderately Weak: System supports but has major constraints
  • Moderate: Basic functionality or support but not outstanding compared to competitors
  • Moderately Strong: High degree of functionality but lacking compared to a few competitors
  • Strong: On the highest level of functionality compared to competitors

Metrics for Each Differentiator

We used the following metrics to judge each differentiator:

Pricing

The strongest rating was provided to those systems with the lowest up-front license costs (including base, per channel and add-on license charges). Secondary consideration was given to maintenance charges. No consideration was provided to value or features per price. While this is important, rating value is relative to the features and functionalities one needs. We recommend users compare price to features needed to determine the value for one's own application.

Simplicity

The strongest rating was provided to systems that (a) made commonly used functionalities easy to access (like selecting cameras to view video, adding a camera, exporting a video, etc.), (b) had one or minimal number of client applications and (c) that had 'intelligent' defaults or wizards that simplified setup or technical knowledge required to start running a system.

Get Video Surveillance News In Your Inbox
Get Video Surveillance News In Your Inbox

3rd Party IP Camera Support

The strongest rating was provided to systems that supported the most total number of 3rd party manufacturers. Supporting of 'standards' was not a factor as industry support is too limited (We will likely change the weight on standards later in 2010 as support increases). Supporting of larger camera manufacturers was not a practical matter as all VMS systems that supported many 3rd party cameras supported the largest manufacturers (e.g., Axis).

3rd Party System Support

The strongest rating was provided to systems that supported the most number of access control, PSIM systems, PoS systems and video keyboards/matrixes. Additional weight was added for VMS systems that supported large providers (e.g., supporting Lenel is more valuable than almost all other access control systems).

Alarm/Event Management

Rating was based on systems that had integrated alarm management inside of their VMS client software. Additional strength was provided to systems that allowed prioritizing alerts, forward alerts, acknowledging alerts and reviewing response procedures for alerts.

Analytics

Strongest rating was provided to systems that supported the broadest number of 3rd party video analytic systems. Weight was detracted from systems that we believed supported more 'motion detection+' rather than 'intelligent video'.

Enterprise Management

Strongest rating was provided to systems that provided centralized account management, configuration and live monitoring across multiple servers. Specifically, strong systems eliminate the need to update passwords manually for multiple servers and allow batch updates to configurations of multiple cameras.

Redundancy

The strongest rating was provided to systems that supported redundancy for their (A) management/enterprise server and (B) recording of specific video channels. No weight was provided to supporting RAID/redundant storage as all VMS systems can support that through COTS hardware.

Video Distribution

The strongest rating was provided for systems that allowed video to be shared or pushed from one client workstation to another over a network. We did not provide any weight to supporting multiple monitors from a single PC (this is very common for systems).

Accuracy of Levels

We believe that most 'ratings' are accurate within 2 levels. For example, we may rate something 'weak' but another informed user might the performance 'moderately weak'.
 
While we have tested these features, we can certainly be slightly off on some details (though we are confident that we are not significantly off - e.g., rating something a strong when really it should be moderately weak, etc.). Secondly, exact ratings from one level to the next is subject to some level of interpretation (e.g., is a system really 'moderately strong' or 'strong').

Discussion

Please use the comments to ask questions or provide suggestions on the criteria selected. 

Please do not comment on specific manufacturer's performance on this thread. I will transfer such comments to the comparison thread as that is the appropriate forum for such discussion.

The Comparison

Read the VMS Competitive Comparison now.

Comments : PRO Members only. Login. or Join.

Related Reports on VMS

IP Camera - 15 Year Shootout on May 22, 2017
How far have IP cameras come? We bought and tested 4 cameras across the past 15 years to understand how much and where performance has...
Hanwha 9MP Fisheye Camera Tested (PNF-9010R) on May 09, 2017
12MP sensor fisheye panoramic cameras are becoming increasingly common. We have tested Hikvision's DS-2CD63C2F as well as Panasonic's SFV481 4K...
Avigilon Discontinuing Rialto Analytics Line on Apr 27, 2017
Avigilon is informing dealers/partners that the legacy VideoIQ Rialto products have been discontinued, recommending the newer ACC ES Analytics...
Eagle Eye Exec On Mountain Of Servers - VSaaS Growth Analysis on Apr 25, 2017
Eagle Eye VP of Operations, Hans Kahler, posted a picture of himself sitting on top of a shipment of new servers, as a testament to the companies...
PureTech Video Analytics Examined on Apr 21, 2017
PureTech's analytics were chosen for a US border protection system (see related post), which the company claims no other analytics vendor was able...
Axis Network Horn Tested on Apr 18, 2017
We bought and tested the Axis network horn C3003-E, examining setup and calibration, event audio, VMS integration, and sound pressure levels...
Aqueti 100MP Mantis Camera Profile on Apr 14, 2017
One of the original gigapixel camera startups, Aqueti, which we first covered in 2012, is back. This time, they have partnered with NVIDIA,...
VMS Vs NVR Usage Statistics on Apr 07, 2017
What is used more often? VMS software or NVR appliances? 160 integrators told us what they most commonly use and why. [premium_content] Key...
Panasonic Consolidates Video Insight on Apr 04, 2017
Panasonic has consolidated VideoInsight, the VMS provider it acquired in 2015. In this note, we examine Panasonic's goals and its potential...
Milestone / Lenel Resell Partnership on Apr 03, 2017
Lenel has never found success in video management. Nearly a decade ago Lenel OEMed an OEM of Milestone. Now, in an equally uncommon move,...

Most Recent Industry Reports

Avigilon New COO James Henderson Profile on May 23, 2017
It has been nearly 2 years since the infamous Bryan Schmode 'resigned' as Avigilon COO. Now, Avigilon once again has a COO, promoting James...
Aura's 'Invisible Ripple' Next Gen Intrusion Detection Tested on May 22, 2017
Aura Home is a startup intrusion detection system, but it claims new, high-tech sensing that monitors the 'invisible ripples' movement creates,...
Pelco Shutting Down Clovis Line, Laying Off 200 on May 22, 2017
Pelco's Clovis facility once turned out some of the industry's most popular products. Now, the facility is mostly building "obsolete" equipment,...
IP Camera - 15 Year Shootout on May 22, 2017
How far have IP cameras come? We bought and tested 4 cameras across the past 15 years to understand how much and where performance has...
Remote Video Monitoring Providers Directory on May 19, 2017
Remote video monitoring can help integrators generate RMR plus end users lower their security costs and/or improve response to critical...
Axis Criticizes OEMs: "When You Buy An Axis Camera, An Axis Camera Is What You Get!" on May 19, 2017
When you buy a Honeywell camera, you likely get a Hikvision, Dahua or some other company's product. The same goes for easily 100 different...
Hackable 125kHz Access Control Migration Guide on May 19, 2017
Despite being one of the most popular credentials, 125 kHz credentials are easily copied and insecure as we showed in our test results, video...
Forget The Backdoor, "ALL HIKVISION PRODUCTS" On Sale on May 18, 2017
Less than 2 weeks after the Hikvision Backdoor was confirmed, Hikvision has launched a sale "ON ALL HIKVISION PRODUCTS". In this note, we examine...
Amazon Techs Installing IP Cameras Tested on May 18, 2017
In 2015, Amazon started offering video surveillance installation. Now, Amazon has made it a lot easier, with automatic add-on options and...

The world's leading video surveillance information source, IPVM provides the best reporting, testing and training for 10,000+ members globally. Dedicated to independent and objective information, we uniquely refuse any and all advertisements, sponsorship and consulting from manufacturers.

About | FAQ | Contact