Preliminary IPVM VMS Load Testing Results

We started preliminary testing of VMS server load. Here’s what we have found so far and what we plan to do next.

We used a high end laptop (quad core i7 / 16GB RAM) and a low end mini PC (dual core AMD / 4GB RAM). Full tech specs at the bottom.

For this preliminary testing, we tested Exacq for recording and viewing of up to 12 cameras. We added 1 camera at a time and then recorded system status (CPU consumption, RAM, etc.).

Here’s what we found for recording:

  • CPU usage rose linearly with increased cameras. RAM usage / impact was minimal.
  • For the high end laptop, while CPU usage rose, it was so trivial that even with 12 cameras, CPU usage did not go higher than 6%.
  • For the mini PC, CPU usage relative to cameras was a lot higher (not surprising) but RAM usage was not impacted.

By comparison, for viewing:

  • CPU usage was far more impacted by viewing than recording.
  • RAM usage was not a factor.
  • On the high end laptop, CPU usage of simultaneous viewing / recording was more than double that of recording alone.
  • On the low end mini PC, CPU usage nearly maxed out at just 5 cameras.

Things we did not test:

  • Server side motion detection as Exacq does not support it. Obviously, this is something that could have a major impact and variance across VMSes.
  • Advanced options on live video display.
  • Other VMSes.
  • More cameras.

Below are the tech specs of the two machines used for our testing.

Test Laptop:

Operating System: Windows 8 Pro 64-bit (6.2, Build 9200) (9200.win8_gdr.130531-1504)
Language: English (Regional Setting: English)
System Manufacturer: Hewlett-Packard
System Model: HP ENVY dv6 Notebook PC
BIOS: F.27
Processor: Intel(R) Core(TM) i7-3630QM CPU @ 2.40GHz (8 CPUs), ~2.4GHz
Memory: 16384MB RAM
Available OS Memory: 16274MB RAM
Page File: 2580MB used, 30077MB available
Windows Dir: C:\Windows
DirectX Version: DirectX 11
DX Setup Parameters: Not found
User DPI Setting: Using System DPI
System DPI Setting: 120 DPI (125 percent)
DWM DPI Scaling: Disabled
DxDiag Version: 6.02.9200.16384 64bit Unicode

Graphics Card Specs:

Card name: Intel(R) HD Graphics 4000
Manufacturer: Intel Corporation
Chip type: Intel(R) HD Graphics Family
DAC type: Internal
Device Type: Full Device
Display Memory: 1664 MB
Dedicated Memory: 32 MB
Shared Memory: 1632 MB

Mini PC:

Operating System: Windows 7 Professional 64-bit (6.1, Build 7601) Service Pack 1 (7601.win7sp1_gdr.130828-1532)
Language: English (Regional Setting: English)
System Manufacturer: AMD
System Model: Brazos
BIOS: Phoenix BIOS SC-T v2.1
Processor: AMD G-T56N Processor (2 CPUs), ~1.6GHz
Memory: 4096MB RAM
Available OS Memory: 3688MB RAM
Page File: 1947MB used, 5427MB available
Windows Dir: C:\windows
DirectX Version: DirectX 11
DX Setup Parameters: Not found
User DPI Setting: Using System DPI
System DPI Setting: 96 DPI (100 percent)
DWM DPI Scaling: Disabled
DxDiag Version: 6.01.7601.17514 32bit Unicode

Graphics Card Specs:

Card name: AMD Radeon HD 6320 Graphics
Manufacturer: Advanced Micro Devices, Inc.
Chip type: AMD Radeon HD 6320 Graphics (0x9806)
DAC type: Internal DAC(400MHz)
Display Memory: 1960 MB
Dedicated Memory: 372 MB
Shared Memory: 1588 MB

As a side note to this I would like to bring up issues related to VMS software limitations. Such as only some VMS softwares limiting camera counts to 32,64,97 and 128. If you have a machine that has high enough specs to handle it and no bottlenecks in throughput then 32,64 and even 97 is not a respectable limit.

Hi Derek,

interesting start! Could you tell us some info (type of CODEC, resolution used, compression level, etc.) about the 12 video streams being sent to the VMS servers?

I would also be interested in knowing more on how the disk performs by increasing the number of cameras with regard to:

- Disk responsiveness (ms)

- Disk usage (avg disk queue length)

- Disk throughput (Bytes/sec)

Are you aware of the performance counters that Windows OSes provide?

Tiago, I am using PerfMon to count and collect data of the system for every new camera added. I'm using H.264 for every camera, and I used 10 720p and 2 1080p cameras. I've defaulted every camera used for testing, and the only change made was bitrates, which is set to VBR with no cap, or VBR with highest possible cap. So, camera compression was left at defaults for each camera.

For disk performance, as more cameras were added, there were more reads/writes per second to the disk. As the test continues, we can incorporate more information about the disk and other system processes as we progress.

Are you viewing directly from the PC's or using a remote client to connect for your viewing tests?

Nathan, for the client side test, we are viewing the PC directly using the VMS's client, and as for the server side test, we are not viewing (video) directly on the PC, but instead, we are using Perfmon to collect system performance data.

This is a good start Derek. Part of my job is to do VMS perfformance testing on the hardware we design and manufacture and your observatioins are in line with what I see regularly.

To complete your efforts, as Tiago asked, you do need to observe the HDD metrics he asked for and compare them to the data coming into the system from the cameras. This means the VMS is set up for continuous recording. If the camera Receive Bytes/sec and the HDD WriteBytes/sec are virtually the are in good shape usually. The Queue length is also an important metric on a server because if it does not return to zero, then the VMS is exposed to losing data. The HDD throughput will typically be the bottleneck in larger camera counts where the combined camera data is around 250+ Mbits/sec. The smaller camera count systems will tend to stress the CPU resources first.

Nathan's request about the viewing is important to answer. My observations are that LOCAL viewing is a CPU hog for ALL VMSs and one can find statements from most VMS vendors that doing local viewing on the recording server is not recommended. Doing viewing on a REMOTE "thick" client is best for a Server as long as one does not force the Server into a transcoding operation, which will impact server performance quite a bit.

VMS Web support has this transcoding someplace.... and depending on where it is (Server or dedicated Web server)... there will be an effect to pay attention to.

One way to stress the HDD path with H264 is to use cameras that allow for CBR...and set the cam CBR to the maximum value allowed. I know SONY and Samsung cams allow for this.

If you get in to testing the recording server WITHOUT doing local viewing, then I suggest using MJPEG streams as they will use up more bandwidth with fewer cams than H264 will use.