Upcoming VMS Comparison Research / Testing

I want us to do some fundamental research to better understand how specific VMS features work. I am throwing this out to gather feedback from members on what they think and what they would like to see.

Here's a few examples:

  • Video export - What's the process of video exporting? What options are available? How is video played back once exported?
  • Camera discovery / addition - How does camera discovery work? What options does it provide? How about manual addition?
  • Digital zoom - How do you enable digital zoom? How do you digitally zoom, pan and tilt? How do you return to the 'full' view?

I imagine there being a few dozen of these, including how do I review an alert, how do I search for video at a specific time, how do I create a view, etc.

My rationale is that I want a clear understanding of tradeoffs - are there major variances? is someone doing something really wrong? is someone missing something fundamental? etc.

VMSes to Use

The plan is to use 5 VMSes (in alphabetical order):

  • Avigilon
  • Exacq
  • Genetec
  • Milestone
  • OnSSI

We'd like to use VMSes that are broadly used so that readers can maximize value from looking at their own (or likely alternatives). On the other hand, the VMS market is deeply fragmented and there are numerous other VMSes that are widely used.

We certainly plan to test other VMSes in the future and, when we do so, we can compare them to this initial group test as a baseline (e.g., Seetec's camera discovery is better/worse/different than those 5).

Publication Schedule

My idea is to publish one such comparison each week - Week 1, video export, Week 2 camera discovery, Week 3 - motion based search, Week 4 - recording setup, etc.

Let me know if you have feedback or questions.


I think the list is very good since those are the market leaders. If there was space for another one then I would love to see an indepth review of Axxon, their 16 channel free version but also the enterprise version.

We can do tests of individual VMSes independent of this. My main concern is keeping the resulting reports readable.

John,

what about some one box budget NVR reviews for us that do small projects? I understand that not many exist that support third party cameras (other than the ones from NAS companies like QNAP, e.t.c) but even the ones from huge companies like Hikvision, Dahia, e.t.c would be interesting - at least for me :)

Harris, we did a Hikvision DVR last year and have plans to do Dahua in the next few months.

John,

I think the biggest question is who are you testing for - installers or end users? Obviously, there will be a huge difference in what questions to ask and features to compare. Take your examples - Digital Zoom would be a feature that would definitely be of interest to end users, while Video Export could apply to both installers and end users (installers would need to set up the export and possibly provide storage). On the other hand, Camera Discovery, while it may apply to sophisticated end users, is more an installer concern.

The Phase 1 evaluations I wrote for Avigilon, Dallmeier and Endura, and posted here (and three others I can't release yet) were based primarily on end user needs, with a few management questions thrown in. Phase 2 evaluations, except for Dallmeier's, go much deeper into both end user operations and technical detail from a management, upgrade and installation perspective.

These reports are not meant to rate or rank VMSes. Their goal is to understand how certain features are implemented and how the implementations differ.

An installer may be more interested in reading the camera discovery report while an end user may care more about digital zoom. Each report will be self contained.

In the future, my goal is to use this research to better analzye and compare products. At that point, we can make judgements about what installers or end users care more about.

John,

How about throwing in another VMS other than both Milestone & ONSSI? Doesn't Milestone still write the backend for both & ONSSI just the front end (Occularis)? Or have I been away from these for too long and now ONSSI actually writes their own recording engine as well?

That would give room for another VMS...

Thanks!

To the best of my knowledge, the backend is still the same, but the front end is different, and often substantially so. I am open to suggesting for an alternative VMS to include.

Alarms! How do the VMS's deal with alarms and especially, how do they present them.

We used to have an VMS which had a simple Alarm window in which the latest alarm was presented.
Now that we switched to Milestone I have to pull out a bunch of tricks in order to create the same thing.

Ruling. Is the VMS able to create certain rules. For example. Move a PTZ back to its home position after not being moved for 15 mins. Ofcourse some camera's have this build-in, but not all.

3rd party add-ons. How many are availible.

John,

From our evaluation specs:

  • How are camera salvos handled: How many pre-made salvos? Can the salvos intermingle 16:9 and 4:3 cameras? Can custom salvos be easily programmed? How flexible are the salvos? Can cameras be played back in synch? Can some cameras be synched while others are not?
  • How are camera displays handled? Virtual matrix vs. Monitor wall? Soft vs. hard decoders?
  • How are analog PTZs addressed? Is addressing flexible (any PTZ or group of PTZs attached to any encoder's port or limited to only the cameras attached encoded by that encoder)?
  • How are PTZs controlled? Mouse? External joystick/keyboard? On-screen joystick?
  • Camera callup methods: Drag and drop? Double-click? Keyboard?
  • Live/Playback switching: Separate programs or windows (like Genetec Omnicast)?
  • How is a playback timeline selected and controlled? Timeline bar? Step back/forward in increments (s, m, h, d)? Manual entry of date/time?
  • Method of designating camera(s) and date/times for clip creation: Manual entry of camera/date/start time/end time? Mark beginning/end during playback on timeline?
  • How is playback direction and speed selected or entered?
  • How are bookmarks entered and used? Can they be "global" (make a bookmark and call other cameras to that exact time)? Can they contain metadata (typed in explanation)?
  • Are clips auto-named or does the user have to create their own filename? If auto-named, can the user enter metadata and can the clips' metadata be searched for text?
  • Where are clips saved? Is there one specific location or does the system use Windows Explorer?
  • Does the system allow pixel searches? How is the area of interest designated?
  • What format are clips saved in? Proprietary or standard (.avi, etc.)?
  • Fault notification.
  • User rights management: Internal or via Active Directory and Windows Rights?
  • POS Integration: Internal or via external program?

Carl has a nice list posted and it includes a lot of elements from the Client side of things. The Client support must be evaluated as part of this effort.

One of my pet peeves is the method to add cameras that are NOT automatically found and must be entered manually.

The evaluation should be how easy it is to add 20 cameras manually.

Another thing to evaluate is how the 'new' cameras can be added before the VMS has official support. Many times this boils down to how setting up ONVIF controls are handled.

Lastly, evaluate how well using RTSP streams are supported. Some VMSs ONLY use this method.

The things I'm curious about are "tips and tricks" that most software incorporate that people don't usually know about, either because they're not obvious, or because people just don't take the time.

I think of a few that Vigil has that I'll sometimes show to the more savvy users when I'm training them, such as

  • Smart Search - mask areas within a playback image and have the system re-search for motion in those areas
  • Markers - select a sub-section of a playback timeline, then "zoom" on that area for finer scrub control, and for exporting just that section rather than the entire timeline
  • Exporting multiple cameras at once

None of these are hidden, but they're also not in-your-face, so unless they have time to sit and play, users don't generally find them... invariably though, users that I do show them to love them.

Also, things like "recommended" operations - again using Vigil as an example: when exporting video, one can select multiple destinations (local disk, DVD-R, flash drive, etc.), and the "Exports" folder on the local drive is generally selected by default. I always tell users to leave that checked, even if they're exporting to optical, because then you keep a copy of it on the machine.

Vigil also has the option to export to generic AVI as well as its own "Authentic Video" (proprietary MJPEG codec) - I always *reinforce* that they should export as Authentic, as it embeds all sorts of critical metadata as well as authenticity checking, which could be critical if it's to be used as evidenced. I've had several occasions where users have exported as AVI and then ask where the time display is (answer: it's not there in AVI). I'm sure most DVRs/NVRs have similar options; it would be good to see that included.

Anyway, don't know if those kinds of things fall within the scope of your tests - things like Smart Search and Markers in particular, are features I use regularly and if I was looking at other VMSes, I'd want to know if they had something similar. It would be nice to know about other such "neat" features that are in other systems.

Another one: Keyboard shortcuts. Can key combinations be used to control functions like playback direction and speed; initiate clip creation; create bookmarks, etc?

By the way, what happened to the "bullets" feature when posting here? I used bullets in my post and I believe Matt did too. Do "bullets require Java or Flash? Bullets appear everywhere except on my office computer. Oh well....

^Good one, Carl!

Was this ever completed? IS there a specific comparison between exacq and genetec?