VMS Testing Plan 2013/2014

Author: John Honovich, Published on Jul 23, 2013

We are embarking on a major new VMS testing program where we will perform in-depth tests both on features and products never done anywhere before. To lead that, Sarit Williams has joined IPVM. She previously led QA for a VMS developer. Now, she will be solely focused on testing and researching VMSes.

This note explains our testing approach and timetable for the next year.

Previously ** ****

*** **** ***** ***** ** *** ******* ** ************* **** - ****, ********* **** **** * ***** *** test ******* *** * *** *********** **********.

Step * - ******* *********** / *********

**** **** ******, ** *** ******** **** ******* *********** / shootouts ***** ** **** ** ******** *** ***** *** ***** them ****. *** ********, ** **** ******* *********-*********************, ***** ** **** * *****, ********* *** *** ***** details ** *** **** ********* ***** ********.

*** ******* ** **** ** ** ******* *********** ********. ** one ******** *** **** ******* ******** *** ********** *** ***** work ** * ******* ** ******* *****. *** ********, ****** talk ***** ********* *** **** **** ** ****** ******* **? What ***** ** ****** ** *****? **** ******* *****? **** initial ******** ** ** ******* * ********** ** ********** ***** specifics.

** ********** ***** * ***** ******* ******* *********** / *********. Update ******** ****: ******* ****:

VMSes ******

**********, ** **** ** **** ** ** ** *****. *******, for *** ******* / ***********, ** *** ******** ** * to **** ** **********. (*** **** ***** ** ****, *** longer **** ****** *****, *** ***** **** **** ** ********). We ******* * ** ****** ** **** ******** ***** ****** the ***** ********** ** *******.

***** *** * ** ******** * ******** ****** ***** ** usage *** ****** ********:

  • ******* *** ********* (*** '*** *****')
  • ******** *** ***** (*** "** *** ******')

**** **** *** **** **** *** *** '****'. ******, **** aspects *** ************ *******. *******, ** **** ***** ********* **** have ***** ****** (******* ** ** ** *** ** ** beat ****).

****** ****, ** **** ****** * ****** ** (** *** we **** **** ******* ***** *** ************ *** ** ***** also *** ***** ******** *** * *** ****** **** ****).

Step * - ********** *** *****

***** ** ******** *** (******* ***** **) ******* ***** - likely **** **** ** ********, ** **** ****** ** ********** VMS *****, ***** *** ****** **** ***** ** **** * single ************. ** **** *** *** ******* ***** ** * baseline *** ********** (*.*., ** **** ** **** *******, ** can ***, "*******'* ********* ** ******/***** **** *********** ******* **** have/lack ******** ******** *, * *** * ********** ****** ** the ********* **********, ***.).

***** *** ******* ****** ** ***** ***** ******* *** ** will **** ** ** **, *** **** ******* * ** ** *** *** ******* *****.

Step * - ***** **** ****

****** *** ****, ** **** ** ******* ** *** *** IPVM *** *********** ********** **** ****** **** ******* ** ******* how ***** ***** **, ******* ********** *** ***** **** ***** the ******** ********** *** *** **** ***** ******* ****** **** their ********* / ***** ****.

Going *******

*********, **** ** **** *** *********. ***** **** *, ** will ******** ** ******, **** **** ******** ******* ** **** as *** *** *******. ****, ******, ****** **** ** ******** of *** ******* ******** *** *** **** ****. *** ** know **** ********* ** ******** *** ****.

Comments (13)

Will relative costs to the end user, including SMA's and license renewals, be included? Total affordability, over a certain number of years, should be a major benchmark item, not just product feature sets. If you were doing automobiles, Rolls Royce or Ferrari might have the highest score, but be totally inappropriate for the majority of buyers due to cost factors.

Hi Ted,

Yes, we absolutely will cover total costs and compare to overall features/value delivered. The focus to start is understanding and documenting how the VMSes work, since this takes the most time. Once we do that, we can start comparing to the costs to better understand the relative value.

Welcome Sarit, I am looking forward to the reports! I am curious as to the level of “training” that these reviews will contain in the future? I don’t mean full how to type set up guides for each VMS or anything that in depth. Just some basics and maybe some tips/tricks discovered through using the VMS.

Thank you Ross! I will certainly continue to include training tips/tricks discovered in each VMS (like in the multistream and exporting reports). Additionally, we will be preparing new VMS 101 training to help train members on fundamentals and main differentiators; I hope to see you there.

What happens as feature sets change and evolve? Most certainly all vendors will have changes and upgrades in time for ISC. And where does Avigilon stand with IPVM continuing research on their products? Does IPVM foresee issues with obtaining new cameras or software updates from them considering the recent and ongoing problems with them?

I'll leave the first couple questions to Sarit, but as far as Avigilon: we have sources for Avigilon product and support has actually been answering questions for us. We also have sent questions and copies of posts their way, as we do with all manufacturers when doing tests or updates, but they have chosen not to respond.

So in short, we don't see it as a problem, and we welcome their feedback on technical issues, as always, if they choose to give it.

Would be interesting to also redo the integrator surveys again that you performed a couple years back.

Hi Thomas, We are going to redo them. However, it has not even been 2 years yet (e.g., favorite VMSes) so we are waiting to give some time for things to actually change. This fall would looks like an appropriate time.

Great Job ,

Can you include support issues relevant to the vms?

Can we send in suggestions on some weve used in past?

look forward to this report .

Christopher,

Thanks so much. Please feel free to email any info to sarit@ipvm.com; any information you have would be greatly appreciated. I can tell you that as I'm setting up these environments and clients and going through this process as an end user would I have lots to write about (errors during install, crashes, activations, etc) and those subjects are certainly on the list.

This is great and I look forward to the analysis. Would it be possible to see table listing all features in a side-by-side comparison? Each feature set would also need to be explained as well. I understand that such a schedule may be difficult, but it would be pretty easy to follow.

Sarit, John,

This is exactly what we think is required for BMS benchmarking. As some time back I raised this in this forum seeking the stress testing approaches as well as quantitatively measuring the features/performance.

I am looking forward the step1 and also will be happy to contribute in the process.

Thanks, Krish

Thank you Krish and Jim for the suggestions- duly noted! We will continue to jot down suggestions and recommendations as we expand our baseline tests.

Login to read this IPVM report.
Why do I need to log in?
IPVM conducts unique testing and research funded by member's payments enabling us to offer the most independent, accurate and in-depth information.

Related Reports

Magos Radar Company Profile on Nov 12, 2018
Magos America General Manager Yaron Zussman admits when he first came across Magos, he asked himself: "What's innovative about radar?" Be that as...
Axis 2N Intercom Tested on Nov 08, 2018
Axis expanded its video intercom business buying Czech-based 2N in 2016. Despite competing against owner Axis' intercoms, 2N recently registered as...
Dahua Dual Imager Dome Camera Tested (HDBW4231FN-E2-M) on Nov 07, 2018
Dahua has introduced a dual-imager dome model, the HDBW4231FN-E2-M, with two independently positionable sensors including integrated IR, not found...
Avigilon Opens Up Analytics And Cameras on Nov 06, 2018
Avigilon is opening up. The company historically famous for advocating its own end-to-end solutions and making it harder for 3rd parties to...
Worst Products on Nov 03, 2018
Security integrators periodically report on their favorite and worst products to IPVM. These are known integrators who IPVM pays to answer surveys....
Solar-Powered, Smart-Phone-Based Access Kit (VIZPin) Examined on Nov 02, 2018
Cloud-based access control company VIZPin is releasing a solar-powered and smart phone based access control system for gates and other remote...
Unisight Company Profile on Nov 01, 2018
Hikvision's largest US OEM, LTS has started to carry Unisight, whose products (shown below) look a lot like Hikvision's rival Dahua: Who is...
Directory of Access Control and Intrusion Startups on Oct 30, 2018
This directory catalogs access control and intrusion detection startups IPVM has found noteworthy. The directory is provided for your review to...
VMS Camera Management Shootout - Avigilon, Dahua, Exacq, Genetec, Hanwha, Hikvision, Milestone, Network Optix on Oct 29, 2018
Camera setup, configuration and maintenance are the most common tasks when managing a surveillance system. Who does it best and worst? Who offers...
Axxon Face Search Tested on Oct 26, 2018
AxxonSoft has brought facial recognition to their Axxon Next VMS for free with the simply named Face Search, claiming to allow users to find...

Most Recent Industry Reports

Integrator Credit Card Alternative Divvy on Nov 13, 2018
Most security integrators are small businesses but large enough that they have various employees that need to be able to expense various charges as...
Directory of Video Intercoms on Nov 13, 2018
Video Intercoms, also known as Video Door-Phones or Video Entry Systems, have been growing in the past decade as more and more IP camera...
Beware Amazon Go Store Hype (Tested) on Nov 13, 2018
IPVM's trip to and testing of Amazon Go's San Francisco store shows a number of significant operational and economic issues that undermine the...
Magos Radar Company Profile on Nov 12, 2018
Magos America General Manager Yaron Zussman admits when he first came across Magos, he asked himself: "What's innovative about radar?" Be that as...
Genetec Privacy Protector Tested on Nov 12, 2018
Genetec has built Kiwi Security's Privacy Protector into Security Center, an analytic which anonymizes individuals in cameras' fields of view...
Chinese Government Increases Hikvision Ownership on Nov 12, 2018
The Chinese government - Hikvision's controlling shareholder - is increasing its ownership of the video surveillance giant amid sharp stock price...
Axis: "No One Wants To Buy A Camera" on Nov 09, 2018
Axis has, in its own description, made a bold declaration: The industry is changing so rapidly that the following statement might seem bold but...
Video Surveillance Hard Drive Size Statistics 2018 on Nov 08, 2018
What is the most common hard drive size for video surveillance? 150+ integrators answered: What size hard drive do you most commonly use? What...
Axis 2N Intercom Tested on Nov 08, 2018
Axis expanded its video intercom business buying Czech-based 2N in 2016. Despite competing against owner Axis' intercoms, 2N recently registered as...
Haven Targets School Security with Lockdown Lineup on Nov 08, 2018
Haven, a US startup founded in 2014 as a residential-focused company, has now raised funding and is offering a lineup of commercial grade locks for...

The world's leading video surveillance information source, IPVM provides the best reporting, testing and training for 10,000+ members globally. Dedicated to independent and objective information, we uniquely refuse any and all advertisements, sponsorship and consulting from manufacturers.

About | FAQ | Contact