Subscriber Discussion

Hikvision 30+ Cameras On NVR - Apps And Client Really Slow Down And CPU Usage Jumps - Any Other Experiences Or Tests?

MC
Michael Cogbill
Nov 03, 2017

I have a several of projects with Hikvision NVRs (DS-9600 series) with 31 cameras, 34 camera and one at 44 cameras. Mostly Hikvision cameras but also have a few FLIR thermals (FC series) and/or Arecont surrounds (AV-12176). I have noticed that if you get over 20 to 25 cameras, the APPS and Client really slow down and CPU usage jumps.  If you open a second screen it get much worse.   In these projects I have used an i7-4 core 64bit CPU, 16 GB Ram, and a Matrox C680 Graphic card for the client PC. I managed to get two screens open with about 30 cameras but as soon as you get a lot of activity the bit rate naturally jumps, the CPU usage hits the 100% mark and the PC often locks up. Running windows 10 or Windows 7 but don't see a lot of difference between OSes. Also, had a little mini-pc (Intel X5-Z8300 64-bit, 4GB Ram, Intel HD Graphics, Win10) that we put behind a TV for display in a staff area with a wireless keyboard/mouse for control. The iVMS-4200 Client quickly crashed the unit when I put up a full 32 view layout.

Alternatively, I tested a Digital Watchdog Spectrum NVR with its client software installed in the same projects to compare. Client performance with three displays (16 cameras each) was much better. CPU usage hovered around the 40% level.  Noticed it went to about 70% CPU load with 6 display open and considerable slow down/delay of live action but did not crash. Much better performance.  Also had no problem running this client SW on the mini-pc (30 - 40% CPU load).

Am I missing something fundamental here in my spec/designs? What should one expect from the client SW, workstations and multi-display configurations?  Had IPVM done any testing along these lines?

MC
Michael Cogbill
Nov 03, 2017

P.S. Also note that all cameras are running on a isolated GB network with their own network switches. We connect this to NIC 1 of the NVR and use NIC 2 to bridge to the main house network (for remote access and viewing). Client workstation sits on the isolated network as well.  Our network usage was low - max 20%. No delays or problems with the images showing on the NVR monitor outputs.

Avatar
Joseph Parker
Nov 03, 2017
We've attempted to do so twice with NVMS-7000, the LTS rebrand. Had all the same problems you mentioned, on both windows 7 and server 2012. The software is a dog, barely functions as a viewer, bogs down as a server, and the best part is the storage server would randomly crash and not send error reports, so no recordings! Fun! I converted both to Spectrum at a loss and moved on, but the first one I probably dumped 50 man hours into trying to fix. Buying the 55 licenses day 1 for that system out of my own pocket would have been a huge cost savings. I'd also add that we are also a MSP, and even with our combined staff expertise we could not make it work with any reliability at all.
(2)
UI
Undisclosed Integrator #1
Nov 03, 2017

 We had very similar experience with a 9000 series 64 Channel NVR and about 40 cameras.  PC's running the IVMS-4200 and connecting to the NVR would red line when attempting to display live video.   We attempted to force Stream 2 by default and it helped some, but overall, not very good.  I did notice and posted in another thread about using 4200 to direct connect to 16 HIK IP cameras and the CPU performance was outstanding.  Somehow bypassing the server and pulling the streams from the cameras directly seems to perform better.

U
Undisclosed #2
Nov 03, 2017

Somehow bypassing the server and pulling the streams from the cameras directly seems to perform better.

That's to be expected, since the stream would typically come through the server, which is busy writing to disk most of the time.

U
Undisclosed #2
Nov 03, 2017

It does on first thought sound like a problem of the VMS software, its viewing functionality to be exact. I haven't used iVMS, but others I've used have handled >100 cams viewed at the same time, and most certainly it shouldn't crash even if it was a bit slow with all of it going on.

Does the PC you run the viewer on have latest updates (graphics drivers etc.)?

MC
Michael Cogbill
Nov 03, 2017

Drivers are all updated and all windows updates as well.

Avatar
Jon Dillabaugh
Nov 03, 2017
Pro Focus LLC

Why not just use DW Spectrum as a viewing client for free, if all you need is live view? Obviously, this won't help if you also need playback ability.

I have never tried installing that many cameras on an NVR, because any system we sell of that size always gets a properly sized server running Spectrum. I actually would never consider selling a single NVR with that many cameras.

(1)
(1)
U
Undisclosed #2
Nov 03, 2017

You may also want to check Resource Monitor (resmon.exe) and disk I/O on the server, assuming you're recording also. What kind of storage are you using? And image rate and resolution?

 

Edit: Oops, this thing probably didn't run Windows :)

UI
Undisclosed Integrator #3
Nov 03, 2017

Never noticed this issue. I have plenty of systems with 32 cameras on 9632, 60 cameras on 9664,110 cameras on 96128, 225 cameras on 96256 and many other configurations with stitchers and 360 fisheyes and have not experienced those issues. We do use decoders for the screen outputs on many projects. As for iVMS 4200, we have used it, several projects using the aux screen functions (4 screens) maxed out and had no problems.

 

LT
Larry Tracy
Nov 05, 2017

I have many of the same NVRs out there and haven't had any issues at all

UI
Undisclosed Integrator #4
Nov 03, 2017

We have had similar issues with the iVMS-4200 Software (Client Software). Jon's recommendation is probably the best if you want a solid live-viewing VMS Solution. 

(1)
Avatar
Ethan Ace
Nov 03, 2017

We're testing this now, just got 32 cameras connected and ready to go on a 9632. 

Can anyone share what resolution, codec, framerate, etc., they were using when issues occurred? Or total throughput?

We'll be measuring on our own but would like to recreate conditions for some of your issues, as well.

(1)
MC
Michael Cogbill
Nov 04, 2017

Mostly had 2 and 3MP cameras streaming with H.264 and frame rate set to 8 fps. Also had 3 arecont av-12176 omni surrounds plus 19 2MP Hikvision set to record at 8 FPS...

Undisclosed #3 - what were the PC specs and graphics card you used for running the 4 auxes of the iVMS-4200?

UI
Undisclosed Integrator #3
Nov 05, 2017

Dont recall but I can assure you they were low ball simple machine with a Nvidia 4 channel graphics card nothing expensive and works like a charm. You are aware that you have (4) independent screen views with iVMS-4200, set up any way you like, right? Just create the view you desire, hit aux and move the screen to the desired monitor. It is not the fanciest and it has never been proclaimed to be but it works fine, I have NEVER once had a freeze-up or other issue like what has been mentioned.

(1)
UI
Undisclosed Integrator #5
Nov 05, 2017

I have a site with 3 buildings, each of which have 32 2MP cameras all connected via a dedicated gig network back to a Dell with radion graphics and 16gb ram. All hikvision switches and cameras, dual monitors.

There were some issues that may or may not have been worked out. 

The customer is only viewing 32 high traffic cameras at a time via ivms4200, 16 cameras on each screen from throughout the premises based on the time of day and activity. 

During the installation we attempted viewing all 32 from a single recorder and the whole thing would freeze and reboot. The NVR, not the Dell. This happened multiple times from 2 of the recorders. 

I think I was lucky in this situation that the customer was not interested in viewing 96 cameras at a time. 

PC has handled everything ok, hovering in the 20-40% cpu load range. But the NVRs were definitely struggling with the load of recording, live view via hdmi, as well as client

MC
Marty Calhoun
Nov 05, 2017
IPVMU Certified

Odd indeed, you sure of all your settings? It is not the way this equipment works, it can easily handle that payload and much more. I have many as in 100+ systems using 9632 recorders with HDMI out on an had no 'overclocking' type issues whatsoever, ever. and 2mp cameras, sounds fishy to me. We have several loaded with 5mp cameras without problems.  Have you spoken with your local technical representative? Or your RSM? Not the ADI counter guy but Hikvision technical support? Please contact the right folks, check your settings before blackballing the system or wasting your money on another solution that you do not need to do.

(1)
MC
Michael Cogbill
Nov 06, 2017

For my part I did get the technical representative from Hikvision to one of my projects.  This particular project had one 9632, one 9616, 3 Flir FC series, 1 Arecont Surround (AV-12176), and 29 Hikvision 2MP cameras (bullets and 2 PTZs).  All Cameras on a separate 1 Gig network connected to NIC1 of the NVRs.  Client PC running iVMS-4200 was Win10, i7-7700, 16GB Ram, Matrox graphics 4K C680 6 output card.. No problem with the NVRs. However, when we pushed 16 channels each to two 58" Monitors and 4 channels to a 27" third monitor, CPU was usage ran consistently in the RED (80%+) but network usage was low 20 to 25%. Over time (I suspect when the bit rates increased due to activity) it would lock up the client PC.  Hikvision said I needed a more powerful Client PC to run the 3 monitors with iVMS 4200. The Client PC more than meets evern the performance spec recommended by Hikvision for the client software. 

If you have some advanced settings please share...

(1)
(1)
MC
Marty Calhoun
Nov 06, 2017
IPVMU Certified

Not sure but seems like it might be that you are depending on the computer processor to do all the work and do not have a decent graphic card driving the monitors with a separate processor on that card? When you run larger monitors it does take more to push them. Dump the 6-Output card concept, never have had luck going that route, 4 port only, but thats just me.

Why dont you forget computer and go with a 6916? That puts all the power on the decoder and ZERO on iVMS-4200 and I guarantee it will run like a champ. Hikvision has a new generation of decoders on the market that are super sweet indeed.Forget the 6400 series, 6900 series are in a class by them selves. I used (6) of them on a recent job, replaced computers, and HDMI hassles completely. Get ahold of one, same way of running things just differing on the HDMI side of things, gives you way more power.

Hope this help

 

U
Undisclosed #6
Nov 07, 2017
"When you run larger monitors it does take more to push them" Why?
Avatar
Jon Dillabaugh
Nov 07, 2017
Pro Focus LLC

Because gravity...

(1)
U
Undisclosed #6
Nov 07, 2017
Ok :) Do you agree? I don't
Avatar
Jon Dillabaugh
Nov 07, 2017
Pro Focus LLC

Size of display has nothing to do with resource usage, but a heavier object is surely harder to “push”...

U
Undisclosed #6
Nov 07, 2017
Ok :) Do you agree? I don't
MG
Michael Gombos
Feb 20, 2018

Do the new decoders support playback, or are they live view only?

DR
Dennis Ruban
Nov 06, 2017

I've noticed "hardware decoding preferred" in iVMS settings. Does anybody know what types of GPU's are supported? Any test about this topic?

I tried to google and found nothing.

Avatar
Aitor Fernandez
Nov 13, 2017

Try to use the setting "auto change stream type". This way when you are watching many cameras the computer will be pulling the substream from the camera, and when you double click one of them to bring it full screen it will change to the main stream.

Even if you have 4K monitors, if you watch 4 cameras you will only be able to see them at 1920x1080 at most. If you are trying to watch 36 cameras on one single monitor, even 4K, at most each space on the 6x6 grid gets 640x360 pixels, so it is useless to have the computer decode 36 main streams, which will bring the processor to its knees.

I have used a computer with 4 monitors and up to 128 cameras, which is the maximum iVMS-4200 supports, without any problems, when the main streams and sub streams on the cameras are correctly configured.

(1)
New discussion

Ask questions and get answers to your physical security questions from IPVM team members and fellow subscribers.

Newest discussions