VMS Testing Plan 2013/2014

Author: John Honovich, Published on Jul 23, 2013

We are embarking on a major new VMS testing program where we will perform in-depth tests both on features and products never done anywhere before. To lead that, Sarit Williams has joined IPVM. She previously led QA for a VMS developer. Now, she will be solely focused on testing and researching VMSes.

This note explains our testing approach and timetable for the next year.

Previously ** ****

*** **** ***** ***** ** *** ******* ** ************* **** - ****, ********* **** **** * ***** *** test ******* *** * *** *********** **********.

Step * - ******* *********** / *********

**** **** ******, ** *** ******** **** ******* *********** / shootouts ***** ** **** ** ******** *** ***** *** ***** them ****. *** ********, ** **** ******* *********-*********************, ***** ** **** * *****, ********* *** *** ***** details ** *** **** ********* ***** ********.

*** ******* ** **** ** ** ******* *********** ********. ** one ******** *** **** ******* ******** *** ********** *** ***** work ** * ******* ** ******* *****. *** ********, ****** talk ***** ********* *** **** **** ** ****** ******* **? What ***** ** ****** ** *****? **** ******* *****? **** initial ******** ** ** ******* * ********** ** ********** ***** specifics.

** ********** ***** * ***** ******* ******* *********** / *********. Update ******** ****: ******* ****:

VMSes ******

**********, ** **** ** **** ** ** ** *****. *******, for *** ******* / ***********, ** *** ******** ** * to **** ** **********. (*** **** ***** ** ****, *** longer **** ****** *****, *** ***** **** **** ** ********). We ******* * ** ****** ** **** ******** ***** ****** the ***** ********** ** *******.

***** *** * ** ******** * ******** ****** ***** ** usage *** ****** ********:

  • ******* *** ********* (*** '*** *****')
  • ******** *** ***** (*** "** *** ******')

**** **** *** **** **** *** *** '****'. ******, **** aspects *** ************ *******. *******, ** **** ***** ********* **** have ***** ****** (******* ** ** ** *** ** ** beat ****).

****** ****, ** **** ****** * ****** ** (** *** we **** **** ******* ***** *** ************ *** ** ***** also *** ***** ******** *** * *** ****** **** ****).

Step * - ********** *** *****

***** ** ******** *** (******* ***** **) ******* ***** - likely **** **** ** ********, ** **** ****** ** ********** VMS *****, ***** *** ****** **** ***** ** **** * single ************. ** **** *** *** ******* ***** ** * baseline *** ********** (*.*., ** **** ** **** *******, ** can ***, "*******'* ********* ** ******/***** **** *********** ******* **** have/lack ******** ******** *, * *** * ********** ****** ** the ********* **********, ***.).

***** *** ******* ****** ** ***** ***** ******* *** ** will **** ** ** **, *** **** ******* * ** ** *** *** ******* *****.

Step * - ***** **** ****

****** *** ****, ** **** ** ******* ** *** *** IPVM *** *********** ********** **** ****** **** ******* ** ******* how ***** ***** **, ******* ********** *** ***** **** ***** the ******** ********** *** *** **** ***** ******* ****** **** their ********* / ***** ****.

Going *******

*********, **** ** **** *** *********. ***** **** *, ** will ******** ** ******, **** **** ******** ******* ** **** as *** *** *******. ****, ******, ****** **** ** ******** of *** ******* ******** *** *** **** ****. *** ** know **** ********* ** ******** *** ****.

Comments (13)

Will relative costs to the end user, including SMA's and license renewals, be included? Total affordability, over a certain number of years, should be a major benchmark item, not just product feature sets. If you were doing automobiles, Rolls Royce or Ferrari might have the highest score, but be totally inappropriate for the majority of buyers due to cost factors.

Hi Ted,

Yes, we absolutely will cover total costs and compare to overall features/value delivered. The focus to start is understanding and documenting how the VMSes work, since this takes the most time. Once we do that, we can start comparing to the costs to better understand the relative value.

Welcome Sarit, I am looking forward to the reports! I am curious as to the level of “training” that these reviews will contain in the future? I don’t mean full how to type set up guides for each VMS or anything that in depth. Just some basics and maybe some tips/tricks discovered through using the VMS.

Thank you Ross! I will certainly continue to include training tips/tricks discovered in each VMS (like in the multistream and exporting reports). Additionally, we will be preparing new VMS 101 training to help train members on fundamentals and main differentiators; I hope to see you there.

What happens as feature sets change and evolve? Most certainly all vendors will have changes and upgrades in time for ISC. And where does Avigilon stand with IPVM continuing research on their products? Does IPVM foresee issues with obtaining new cameras or software updates from them considering the recent and ongoing problems with them?

I'll leave the first couple questions to Sarit, but as far as Avigilon: we have sources for Avigilon product and support has actually been answering questions for us. We also have sent questions and copies of posts their way, as we do with all manufacturers when doing tests or updates, but they have chosen not to respond.

So in short, we don't see it as a problem, and we welcome their feedback on technical issues, as always, if they choose to give it.

Would be interesting to also redo the integrator surveys again that you performed a couple years back.

Hi Thomas, We are going to redo them. However, it has not even been 2 years yet (e.g., favorite VMSes) so we are waiting to give some time for things to actually change. This fall would looks like an appropriate time.

Great Job ,

Can you include support issues relevant to the vms?

Can we send in suggestions on some weve used in past?

look forward to this report .

Christopher,

Thanks so much. Please feel free to email any info to sarit@ipvm.com; any information you have would be greatly appreciated. I can tell you that as I'm setting up these environments and clients and going through this process as an end user would I have lots to write about (errors during install, crashes, activations, etc) and those subjects are certainly on the list.

This is great and I look forward to the analysis. Would it be possible to see table listing all features in a side-by-side comparison? Each feature set would also need to be explained as well. I understand that such a schedule may be difficult, but it would be pretty easy to follow.

Sarit, John,

This is exactly what we think is required for BMS benchmarking. As some time back I raised this in this forum seeking the stress testing approaches as well as quantitatively measuring the features/performance.

I am looking forward the step1 and also will be happy to contribute in the process.

Thanks, Krish

Thank you Krish and Jim for the suggestions- duly noted! We will continue to jot down suggestions and recommendations as we expand our baseline tests.

Login to read this IPVM report.
Why do I need to log in?
IPVM conducts unique testing and research funded by member's payments enabling us to offer the most independent, accurate and in-depth information.

Related Reports

Software Only VMS vs NVR Appliances on May 23, 2018
Should you buy your own PC/server and load VMS software on it or get a turnkey appliance (both hardware and software, e.g., NVR, Hybrid DVR) from a...
Dahua Intrusion Analytics And VMD Tested (Poorly) on May 21, 2018
Dahua ships basic analytics on practically all their cameras, ranging from low cost to high end. To see how these analytics work in real world...
Best Manufacturer Technical Support 2018 on May 21, 2018
While 5 manufacturers made the worst technical support 2018 list, only 3 stood out as providing the best technical support to 190+ integrators in...
Stealth / UCIT - Remote Video Monitoring Provider Profile on May 18, 2018
Can 2 remote video monitoring companies, Stealth Monitoring from the US and UCIT from Canada combine to impact the market and compete in a changing...
Cybersecurity for IP Video Surveillance Guide on May 18, 2018
Keeping surveillance networks secure can be a daunting task, but there are several methods that can greatly reduce risk, especially when used in...
Dahua 12MP Fisheye Camera Tested (NK8BR4) on May 16, 2018
Continuing our coverage of 12MP sensor fisheye cameras, we bought and tested the Dahua NK8BR4, examining: Default vs. Optimized...
Worst Manufacturer Technical Support 2018 on May 16, 2018
5 manufacturers stood out as providing the worst technical support to 190+ integrators in new IPVM results. These integrators answered: In the...
Axis 12MP Stereographic Camera Tested (M3058-LVE) on May 10, 2018
Axis has released the M3058-PLVE, a 12MP sensor, stereographic panoramic camera and Axis' first with integrated IR claiming images "sharp to the...
N3N "Truly Open Visualization Platform For Physical Security" Profile on May 09, 2018
A Cisco-backed startup, N3N, claims to deliver "the world’s first and only truly open visualization platform for physical security." The company...
S2 Access Control / 'Unified Security Management' Profile on May 08, 2018
In our 13th access control company profile, we examine S2 Security's Netbox platform: Unified Security Management Platform positioning Core...

Most Recent Industry Reports

Software Only VMS vs NVR Appliances on May 23, 2018
Should you buy your own PC/server and load VMS software on it or get a turnkey appliance (both hardware and software, e.g., NVR, Hybrid DVR) from a...
Buy Arecont: Top Bid $10 Million Cash on May 22, 2018
Last year, Arecont had a deal for a purchase price of $170 million (see Failed Arecont China Acquisition). This year, Arecont has a deal for a...
Installing Box Cameras Indoors Tutorial on May 22, 2018
This tutorial starts our physical installation for video surveillance series, starting with Box Cameras, one of the oldest and most basic types....
The Hikvision Smart Classroom Behavior Management System on May 22, 2018
Hikvision's rapidly growing offering of analytics, which we most recently examined with Hikvision's ethnic minority analytics, is now going into...
Dahua Intrusion Analytics And VMD Tested on May 21, 2018
Dahua ships basic analytics on practically all their cameras, ranging from low cost to high end. To see how these analytics work in real world...
Exacq Improving Technical Support, Responding To Integrator Complaints on May 21, 2018
Exacq had been a long-term favorite of integrators, but since their 2014 Tyco acquisition, Exacq has fallen in IPVM integrator studies (though...
Best Manufacturer Technical Support 2018 on May 21, 2018
While 5 manufacturers made the worst technical support 2018 list, only 3 stood out as providing the best technical support to 190+ integrators in...
Stealth / UCIT - Remote Video Monitoring Provider Profile on May 18, 2018
Can 2 remote video monitoring companies, Stealth Monitoring from the US and UCIT from Canada combine to impact the market and compete in a changing...
Cybersecurity for IP Video Surveillance Guide on May 18, 2018
Keeping surveillance networks secure can be a daunting task, but there are several methods that can greatly reduce risk, especially when used in...
Forced Entry / Duress Access Tutorial on May 17, 2018
Even though access control normally keeps people safe, tragedies have revealed a significant issue. If users are forced to unlock doors for...

The world's leading video surveillance information source, IPVM provides the best reporting, testing and training for 10,000+ members globally. Dedicated to independent and objective information, we uniquely refuse any and all advertisements, sponsorship and consulting from manufacturers.

About | FAQ | Contact