VMS Testing Plan 2013/2014

Author: John Honovich, Published on Jul 23, 2013

We are embarking on a major new VMS testing program where we will perform in-depth tests both on features and products never done anywhere before. To lead that, Sarit Williams has joined IPVM. She previously led QA for a VMS developer. Now, she will be solely focused on testing and researching VMSes.

This note explains our testing approach and timetable for the next year.

Previously ** ****

*** **** ***** ***** ** *** ******* ** ************* **** - ****, ********* **** **** * ***** *** test ******* *** * *** *********** **********.

Step * - ******* *********** / *********

**** **** ******, ** *** ******** **** ******* *********** / shootouts ***** ** **** ** ******** *** ***** *** ***** them ****. *** ********, ** **** ******* *********-*********************, ***** ** **** * *****, ********* *** *** ***** details ** *** **** ********* ***** ********.

*** ******* ** **** ** ** ******* *********** ********. ** one ******** *** **** ******* ******** *** ********** *** ***** work ** * ******* ** ******* *****. *** ********, ****** talk ***** ********* *** **** **** ** ****** ******* **? What ***** ** ****** ** *****? **** ******* *****? **** initial ******** ** ** ******* * ********** ** ********** ***** specifics.

** ********** ***** * ***** ******* ******* *********** / *********. Update ******** ****: ******* ****:

VMSes ******

**********, ** **** ** **** ** ** ** *****. *******, for *** ******* / ***********, ** *** ******** ** * to **** ** **********. (*** **** ***** ** ****, *** longer **** ****** *****, *** ***** **** **** ** ********). We ******* * ** ****** ** **** ******** ***** ****** the ***** ********** ** *******.

***** *** * ** ******** * ******** ****** ***** ** usage *** ****** ********:

  • ******* *** ********* (*** '*** *****')
  • ******** *** ***** (*** "** *** ******')

**** **** *** **** **** *** *** '****'. ******, **** aspects *** ************ *******. *******, ** **** ***** ********* **** have ***** ****** (******* ** ** ** *** ** ** beat ****).

****** ****, ** **** ****** * ****** ** (** *** we **** **** ******* ***** *** ************ *** ** ***** also *** ***** ******** *** * *** ****** **** ****).

Step * - ********** *** *****

***** ** ******** *** (******* ***** **) ******* ***** - likely **** **** ** ********, ** **** ****** ** ********** VMS *****, ***** *** ****** **** ***** ** **** * single ************. ** **** *** *** ******* ***** ** * baseline *** ********** (*.*., ** **** ** **** *******, ** can ***, "*******'* ********* ** ******/***** **** *********** ******* **** have/lack ******** ******** *, * *** * ********** ****** ** the ********* **********, ***.).

***** *** ******* ****** ** ***** ***** ******* *** ** will **** ** ** **, *** **** ******* * ** ** *** *** ******* *****.

Step * - ***** **** ****

****** *** ****, ** **** ** ******* ** *** *** IPVM *** *********** ********** **** ****** **** ******* ** ******* how ***** ***** **, ******* ********** *** ***** **** ***** the ******** ********** *** *** **** ***** ******* ****** **** their ********* / ***** ****.

Going *******

*********, **** ** **** *** *********. ***** **** *, ** will ******** ** ******, **** **** ******** ******* ** **** as *** *** *******. ****, ******, ****** **** ** ******** of *** ******* ******** *** *** **** ****. *** ** know **** ********* ** ******** *** ****.

Comments (13)

Will relative costs to the end user, including SMA's and license renewals, be included? Total affordability, over a certain number of years, should be a major benchmark item, not just product feature sets. If you were doing automobiles, Rolls Royce or Ferrari might have the highest score, but be totally inappropriate for the majority of buyers due to cost factors.

Hi Ted,

Yes, we absolutely will cover total costs and compare to overall features/value delivered. The focus to start is understanding and documenting how the VMSes work, since this takes the most time. Once we do that, we can start comparing to the costs to better understand the relative value.

Welcome Sarit, I am looking forward to the reports! I am curious as to the level of “training” that these reviews will contain in the future? I don’t mean full how to type set up guides for each VMS or anything that in depth. Just some basics and maybe some tips/tricks discovered through using the VMS.

Thank you Ross! I will certainly continue to include training tips/tricks discovered in each VMS (like in the multistream and exporting reports). Additionally, we will be preparing new VMS 101 training to help train members on fundamentals and main differentiators; I hope to see you there.

What happens as feature sets change and evolve? Most certainly all vendors will have changes and upgrades in time for ISC. And where does Avigilon stand with IPVM continuing research on their products? Does IPVM foresee issues with obtaining new cameras or software updates from them considering the recent and ongoing problems with them?

I'll leave the first couple questions to Sarit, but as far as Avigilon: we have sources for Avigilon product and support has actually been answering questions for us. We also have sent questions and copies of posts their way, as we do with all manufacturers when doing tests or updates, but they have chosen not to respond.

So in short, we don't see it as a problem, and we welcome their feedback on technical issues, as always, if they choose to give it.

Would be interesting to also redo the integrator surveys again that you performed a couple years back.

Hi Thomas, We are going to redo them. However, it has not even been 2 years yet (e.g., favorite VMSes) so we are waiting to give some time for things to actually change. This fall would looks like an appropriate time.

Great Job ,

Can you include support issues relevant to the vms?

Can we send in suggestions on some weve used in past?

look forward to this report .

Christopher,

Thanks so much. Please feel free to email any info to sarit@ipvm.com; any information you have would be greatly appreciated. I can tell you that as I'm setting up these environments and clients and going through this process as an end user would I have lots to write about (errors during install, crashes, activations, etc) and those subjects are certainly on the list.

This is great and I look forward to the analysis. Would it be possible to see table listing all features in a side-by-side comparison? Each feature set would also need to be explained as well. I understand that such a schedule may be difficult, but it would be pretty easy to follow.

Sarit, John,

This is exactly what we think is required for BMS benchmarking. As some time back I raised this in this forum seeking the stress testing approaches as well as quantitatively measuring the features/performance.

I am looking forward the step1 and also will be happy to contribute in the process.

Thanks, Krish

Thank you Krish and Jim for the suggestions- duly noted! We will continue to jot down suggestions and recommendations as we expand our baseline tests.

Login to read this IPVM report.
Why do I need to log in?
IPVM conducts unique testing and research funded by member's payments enabling us to offer the most independent, accurate and in-depth information.

Related Reports

4MP Camera Shootout - Axis, Dahua, DW, Hanwha, Hikvision, Uniview, Vivotek on Sep 24, 2018
4MP usage continues to climb, especially for lowe cost fixed lens models. To see who was best, we bought and tested seven 4MP models from Axis,...
Alexa Guard Expands Amazon's Security Offerings, Boosts ADT's Stock on Sep 21, 2018
Amazon is expanding their security offerings yet again, this time with Alexa Guard that delivers security audio analytics and a virtual "Fake...
Central Stations Face Off Against NFPA On Fire Monitoring on Sep 18, 2018
Central stations are facing off against the NFPA over what they call anti-competitive language in NFPA 72, the standard that covers fire alarms....
European Mega Security Firm Verisure Pushing Security Fog on Sep 17, 2018
The European mega security firm Verisure (Securitas Direct), with a reported 2 million customers, is pushing security fog, as shown in this BBC...
VMS Export Shootout - Avigilon, Dahua, Exacq, Genetec, Hikvision, Milestone on Sep 13, 2018
When crimes, accidents or problems occur, exporting video from one's video surveillance system is critical to proving incidents. But who does it...
Dahua Low-Cost 4MP Camera Tested (N44CL52) on Sep 10, 2018
4MP use continues to increase, especially in low-cost models, according to integrators in our 2018 Resolution Usage Statistics. We bought Dahua's...
Stanley Security Acquires 3xLogic, Kushner Becomes Product President on Sep 10, 2018
Stanley Security acquired 3xLogic a few months ago. However, the company has still not officially publicly announced it, leading many to wonder...
Dell Launches IoT for Surveillance on Sep 05, 2018
Historically, Dell has been a PC and server provider (e.g., "Dude, you're getting a Dell") and widely used for surveillance storage. However, in...
JCI / Napco Integration Battle on Aug 30, 2018
JCI and Napco are firing salvos at each other over integration issues which both sides blame on the other. The bigger problem is that central...
Directory Of 110+ Video Management Software (VMS) Suppliers on Aug 30, 2018
This directory provides a list of Video Management Software providers to help you see and research what options are available. Listing...

Most Recent Industry Reports

Alexa Guard Expands Amazon's Security Offerings, Boosts ADT's Stock on Sep 21, 2018
Amazon is expanding their security offerings yet again, this time with Alexa Guard that delivers security audio analytics and a virtual "Fake...
UTC, Owner of Lenel, Acquires S2 on Sep 20, 2018
UTC now owns two of the biggest access control providers, one of integrator's most hated access control platforms, Lenel, and one of their...
BluePoint Aims To Bring Life-Safety Mind-Set To Police Pull Stations on Sep 20, 2018
Fire alarm pull stations are commonplace but police ones are not. A self-funded startup, BluePoint Alert Solutions is aiming to make police pull...
SIA Plays Dumb On OEMs And Hikua Ban on Sep 20, 2018
OEMs widely pretend to be 'manufacturers', deceiving their customers and putting them at risk for cybersecurity attacks and, soon, violation of US...
Axis Vs. Hikvision IR PTZ Shootout on Sep 20, 2018
Hikvision has their high-end dual-sensor DarkfighterX. Axis has their high-end concealed IR Q6125-LE. Which is better? We bought both and tested...
Avigilon Announces AI-Powered H5 Camera Development on Sep 19, 2018
Avigilon will be showcasing "next-generation AI" at next week's ASIS GSX. In an atypical move, the company is not actually releasing these...
Favorite Request-to-Exit (RTE) Manufacturers 2018 on Sep 19, 2018
Request To Exit devices like motion sensors and lock releasing push-buttons are a part of almost every access install, but who makes the equipment...
25% China Tariffs Finalized For 2019, 10% Start Now, Includes Select Video Surveillance on Sep 18, 2018
A surprise move: In July, when the most recent tariff round was first announced, the tariffs were only scheduled for 10%. However, now, the US...
Central Stations Face Off Against NFPA On Fire Monitoring on Sep 18, 2018
Central stations are facing off against the NFPA over what they call anti-competitive language in NFPA 72, the standard that covers fire alarms....
Hikvision USA Starts Layoffs on Sep 18, 2018
Hikvision USA has started layoffs, just weeks after the US government ban was passed into law. Inside this note, we examine: The important...

The world's leading video surveillance information source, IPVM provides the best reporting, testing and training for 10,000+ members globally. Dedicated to independent and objective information, we uniquely refuse any and all advertisements, sponsorship and consulting from manufacturers.

About | FAQ | Contact