VMS Remote Monitoring Tested

By: Sarit Williams, Published on Aug 21, 2013

Remote monitoring of VMS systems is often critical. Large organizations have campuses, offices or branches scattered across a country, with a need to access that surveillance video. Indeed, all organization, large and small, frequently want to watch video from their homes or while on the road.

Limited bandwidth remains a key problem, with remote locations often only providing 1Mb/s upstream to send video out. Worse, many locations (e.g., banks and retailers) want to constrain how much bandwidth video takes up to ensure transactions are always promptly processed.

At the same time, megapixel adoption has surged, becoming the de facto standard for new IP video surveillance systems, raising the question of how well remote monitoring will work with increased strain.

The Test

We decided to test performance of remotely monitored VMS systems. We used a tool to throttle bandwidth to 3 levels:

  • 56kb/s: Worst case scenario for legacy systems
  • 256kb/s: Challenging but common scenario with limited bandwidth
  • 1.5Mb/s: "Good" bandwidth for remote monitoring

We then tested the following VMSes: Avigilon, Exacq, Genetec, Milestone, Network Optix/Digital Watchdog and VideoInsight.

We started with thick clients and then tested web clients. (Note: a future report will be dedicated to mobile applications).

The Key Questions

This report examines and answers the following:

  • How good or bad was performance at 56kb/s, 256kb/s and 1.5Mb/s?
  • How did performance vary between a single and 4 camera view?
  • How usable was live monitoring vs investigations?
  • What difference in performance occurred across cameras and resolutions?
  • What optimization techniques were available to improve performance?
  • How did manufacturer performance vary?

Key Findings

Get Notified of Video Surveillance Breaking News
Get Notified of Video Surveillance Breaking News

Here are the key lessons from our testing:

  • Higher definition cameras clearly and significantly increased load times over SD ones. 3MP took about twice as long on average as SD.
  • Going from 56Kb/s to 256Kb/s made a dramatic difference in usability in almost all thick clients. Going to 1.5Mb/s made all VMS thick client almost as responsive and fast as having unthrottled connections.
  • Thick clients were far less usable than web clients, largely because web clients offered better options to reduce bandwidth demand (primarily transcoding).
  • Even among web clients using transcoding, performance differences were sizeable with Exacq loading the fastest and providing the most granular options as well as ability to transcode for search as well.
  • Among the thick clients, Genetec Security Center exhibited serious problems maintaining a connection at 56kb/s and 256kb/s. Client connections kept dropping not giving cameras a chance to load. Milestone had similar issues.
  • While multi-streaming with a secondary low stream is billed to improve remote monitoring, in our tests, many VMS's low stream took almost as long to load as the high stream with negligible benefits in real-time performance.

Throttling Bandwidth Tool

This short video shows the tool (WAN Connection Emulator) we used to throttle bandwidth and simulate various network conditions. We highly recommend it for anyone interested in understanding remote monitoring problems.

Thick Client Performance

The lowest bandwidth used, 56kb/s, was the most stressful for the VMSes in both a single camera view and especially in 4-camera layout. However, marked improvement can be seen when bandwidth increased to 256kb/s and 1.5Mb/s.

Single Camera Performance

  • 56kb/s - Performance at 56kb/s across all VMSes was very slow. In a single camera view, the HD streams took as long as 100 seconds to load, with 30 - 40 seconds average. This can seem like an eternity when waiting for video to display. Cameras dropped constantly and or flickered and showed stale images 10 seconds old or longer. The lowest was when using a low quality secondary stream defaulted was 10 seconds.
  • 256kb/s - At 256kb/s there was at least 50% and no dropped cameras, though occasional flickering still occurred.
  • 1.5Mb/s- At 1.5Mb/s performance was greatly improved when compared to prior restrictions.  Flickering and dropped cameras were rare and overall improvement was dramatically better than 56kb/s.

4 Camera View Performance

  • 56kb/s – Layout view at 56kb/s was almost non-existent. As soon as one camera appeared, another disappeared maxing the streaming to one camera at a time. Cameras dropped constantly and or flickered and showed stale images 20 seconds old or longer. Time to load a layout with two or more at one time took 102 seconds for the longest and 26 seconds for the quickest one where multistreaming was built in (HD Witness/DW Spectrum).
  • At 256kb/s, there was a major improvement, though flickering and dropped cameras still occurred.
  • At 1.5Mb/s performance was greatly improved when compared to prior restrictions.  Flickering and dropped cameras was rare and overall improvement was ~90% when compared to 56kb/s

The chart below shows the difference between views and bandwidth available (in seconds):

 

Optimization Techniques Web Clients

Overall, web clients offered significant better usability and quicker load time that thick clients, as shown in this graph:

There are 3 main techniques VMSes offer to improve performance at very low bandwidth.

  • Transcoding
  • Multi-streaming
  • Resolution (Image size) and quality (compression) reductions  

Web Client / Transcoding

This option showed the most improvement in overall performance. Although all VMSes, (except Video Insight) employ this method in their thin client. However, transcoding implementation effectiveness differed significantly.

Exacq’s implementation of Transcoding was the most notable, allowing the user to dynamically change size and quality of the stream for individual cameras, layouts and playback thus allowing some streaming, albeit at a lower quality instead of no streaming or significantly delayed as in the other VMSes.

In contrast, the majority of VMSes used transcoding on the backend automatically, transcoding the H.264 stream to a JPEG or MJPEG keeping quality and size the same- not taking into account reducing the image size or quality or let the end user choose a lower stream.

Some thin clients are stripped of major functionality and admin level management features, but for the purpose of viewing critical video remotely and/or investigations they suffice.

Here is an overview video:

Multistreaming

Touted as a major enhancement, multistreaming showed slight improvement in single camera view (typically half the resolution as full), with upward savings of 5-10 seconds per camera (versus 25 - 30 at full). However, there was a collective improvement in single camera load time when a layout was used, and the cameras low stream was configured to use the lowest resolution and lowest quality.  However, once a camera came up the other one disappeared and so on- even with lower streams.

Moreover, how the low stream is configured and whether the user can select it remotely hugely impacts the benefit of having such a feature. In, Avigilon the second stream is added automatically, and the user cannot control its settings relying on the VMS to choose. [Update: Network Optix added controls in their new 2.0 release.]

In Milestone’s Web Client the default stream is used (multistreaming in Corporate only). If low stream is set as the default it affects local users as well- not just Remote.

Exacq’s implementation in the Web Client allows a user to “configure” low streams dynamically affecting only their session, not local users.  On the other side of the spectrum Video Insight doubles the users number of cameras in the server list and expects the end user to know which has the lower stream (based on visual trial/error and good naming conventions).

Here is a video:

Resolution/Compression Affect

As we dropped stream resolution from 3MP down to QCIF for a given camera, load times radically decreased. This should not be surprising as bitrate is a key driver of load time. The tests confirmed the impact higher resolution has:

 

VMS Specific Issues

A few other VMS specific issues are worth noting:

  • Genetec: Genetec exhibited unstable connectivity between thick client/server; dropping connection every 2-6 minutes due to low bandwidth of 56k and 256k making it difficult for cameras to even try loading. At 1.5Mb, the cameras wasted time in loading an image in Remote stream due to fetching entity details and a stock photo that appears before the first image is displayed. 
  • Milestone: Milestone's thick Client exhibited similar stability issues: dropping the server/client communication as soon as bandwidth was throttled to the lower 56k and 256k speed rates. Additionally, the web client in IE10 had difficulty trying to load the image. Unlike IE, Chrome was able to load the initial image.
  • Avigilon: Avigilon's Web Client constantly crashed in IE10 upon Activex installation attempts; the only browser supported for remote viewing. Tech Support is investigating the issues.
  • Exacq: Exacq's Web Client performance at the lowest bandwidth showed the best implementation and consideration of remote users. They offered two types of web clients: Advanced (reviewed in Transcoding video) and Simple to accommodate even 56k.  The Simple Web Client option delivers JPEG images without any advanced UI options or playback to quickly view streaming video while allowing users to select the Transcoding size and quality as well.
  • Network Optix/Digital Watchdog: As one VMS where low streams are built in and added automatically it does not allow the user to choose the configuration of the low stream (fps/res/quality) and camera loading at lower bandwidth resulted in warped images initially, though the VMS was fluid and responsive at the lowest bandwidth save camera images. Network Optix also transcodes recorded video. [UPDATE: The newest version 2.0, released the same day as this report, adds in controls for the secondary stream.]
  • Video Insight: Video Insight's Web Client does not support transcoding and serves the same image as their thick client which showed consistent increased load time.  Furthermore, at lower bandwidth rates (56k and 256k) interacting with the either client (web or thick) via digital zoom or switching cameras/layouts results in a frozen, unresponsive client.

VMS Versions Tested

  • Genetec version Server/Client 5.2.1045.11, Web Client 3.1
  • Milestone Enterprise version 8.1a and Corporate server version 6.0a/Client 8.0a, Web Client 2.5.
  • Avigilon Server/Client/Web Client version 5.0.0.52
  • Exacq Server/Client/Web Client version 5.6.8.45600
  • Network Optix/Digital Watchdog Server/Client/Web Client version 1.51.3031 
  • Video Insight Server/Client/Web Client version 5.5.18.3

1 report cite this report:

Milestone Acquired by Canon on Jun 13, 2014
Wow. Industry insiders have been speculating about Milestone being acquired for weeks. But the outcome is surprising. Milestone has been...
Comments (33) : PRO Members only. Login. or Join.

Related Reports

ONVIF Video Surveillance Tutorial on Jan 29, 2019
ONVIF is well known within the surveillance industry as an interface to connect IP cameras and VMS systems. However, new users may find it...
Verkada Cloud VMS/Cameras Tested on May 02, 2019
Verkada is arguably the most ambitious video surveillance startup in many years. The company is developing their own cameras, their own VMS, their...
Milestone XProtect 2019 R1 Tested on May 15, 2019
For the past few years, Milestone has released quarterly software updates XProtect VMS platform. What is new and how much impact do the updates...
H.265 Usage Statistics on Apr 19, 2019
H.265 has been available in IP cameras for more than 5 years and, in the past few years, the number of manufacturers supporting this codec has...
Average Frame Rate Video Surveillance 2019 on May 23, 2019
What is the average frame rated used in video surveillance systems? In IPVM's 2011 statistics, the average was 6-8fps increasing to ~10fps in...
Verkada Video Quality Problems Tested on May 23, 2019
Verkada suffers from numerous video quality problems, not found in commercial IP cameras, new IPVM testing of Verkada vs Axis and Hikvision...
Smart CODEC Usage Statistics 2019 on Jun 03, 2019
Smart codecs are now nearly a standard feature in IP cameras, but our statistics show integrator adoption has not increased at the same rate. In...
IndigoVision Control Center VMS Tested on May 30, 2019
IPVM's last test of IndigoVision's VMS was in 2010, which found enterprise VMS features and a simple client interface. but no 3rd party camera...
Exacq Remote Cloud Access Tested on Jun 20, 2019
Remote cloud access has been missing from most VMSes (including Exacq and Milestone). Now, Exacq, after releasing Cloud Drive Storage earlier in...
Milestone XProtect 2019 R3 'Centralized Search' Tested on Oct 30, 2019
Milestone has had problems over the last few years releasing significant new software. Now, in XProtect 2019 R3, Milestone is touting "one search...

Most Recent Industry Reports

Brivo Business Profile 2020 on Jan 27, 2020
Brivo has been doing cloud access for more than 20 years. Is the 2020s the decade that cloud access becomes the norm? CEO Steve Van Till recently...
Favorite VMS / NVR Manufacturers 2020 on Jan 27, 2020
In 2018, a new winner emerged and a former top choice declined. Now, there is a new #1, a new top 5 finisher and 2 major VMSes in decline. Our...
"Hikvision Football Arena" Lithuania Causes Controversy on Jan 24, 2020
Controversy has arisen in Lithuania over Hikvision becoming a soccer team's top sponsor and gaining naming rights to their arena, with one local MP...
Axis and Genetec Drop IFSEC 2020 on Jan 23, 2020
Two of the best-known video surveillance manufacturers are dropping IFSEC International 2020, joining Milestone who dropped IFSEC in 2019. The...
Multipoint Door Lock Tutorial on Jan 23, 2020
Despite widespread use, locked doors are notoriously weak at stopping entry, and thousands can be misspent on locks that leave doors quite...
Avigilon Shifts Cloud Strategy - Merges Blue and ACC on Jan 23, 2020
Avigilon is shifting its cloud strategy, phasing out its Blue web-managed surveillance platform as a stand-alone brand and merging it with its ACC...
Verkada Paying $100 For Referrals Just To Demo on Jan 22, 2020
Some companies pay for referrals when the referral becomes a customer. Verkada is taking it to the next level - paying $100 referrals fees simply...
Camera Analytics Shootout 2020 - Avigilon, Axis, Bosch, Dahua, Hanwha, Hikvision, Uniview, Vivotek on Jan 22, 2020
Analytics are hot again, thanks to a slew of AI-powered cameras, but whose analytics really work? And how do these new smart cameras compare to top...
Intersec 2020 Final Show Report on Jan 21, 2020
IPVM spent all 3 days at the Intersec 2020 show interviewing various companies and finding key trends. We cover: Middle East Enterprise...
Vehicle & Long Range Access Reader Tutorial on Jan 21, 2020
One of the classic challenges for access control are parking lots and garages, where the user's credential is far from the reader. With modern...