All Of A Sudden Exacvision Server Running Very Slow, The Stream Is Less Then 1kbps

I inhertied an exacqvision (v5.x) from someone, all of a sudden the cameras are running very choppy and the stream on the bottom is less then 1kbs it once in a while will jump higher, but it mainly stays low. The NIC is a gigabit NIC, and when I view a different NVR from it, it runs perfectly smooth. I did a network speed test and it is also fast. It is only the exacvision that is running slow. RAM and CPU are both less then 20% utliziation.

I even disabled all cameras besides one and it is still slow

Any ideas?

Thanks


Does this unit have the Remote Client Bandwidth Throttling enabled under the Configure System Network tab?

If not that, what camera brand are connected to this unit? I had a similar situation with Pelco IP cameras and there motion detection setting hammering the Exacq software slowing it down and sometimes crashing it.

I just checked and it is not checked off. I have a few brands hooked up to it (hikivsion, geovision and one other brand), but I disabled all but one and I am having the exact same problem.

Thanks

A few things to check:

Determine whether the issue is from the camera to the NVR, or from the NVR to the client.

  • Are you running the client locally on the server, or is the client running on your own workstation? Go to the Network tab of the Configure System page and ensure that a bandwidth limit has not been set for remote clients. The setting should say OFF.
  • Go to the camera's web page and check if the camera stream is better there. If not, then maybe reboot the camera.

I am running the client directly on the server, and the setting is off.

When I go to the cameras web page everything is running perfectly smooth.

I am an IT tech by training, and I cant find anything wrong on the computer (besides maybe settings on the NVR).

Thanks

This is an odd problem. Since you said it happened all of the sudden, i bet it's something easy to resolve, but I'm not sure what it it is.

You might also check the stream settings. Is the FPS, resolution and quality what you expect it to be for a high quality, smooth video?

Otherwise I'd recommend calling exacqVision Tech Support. They can get a remote session set up and take a look. If you are on v5.x, you are on a version at least 2 years (8 releases) old.

When I am testing in the lab and start to see funny stuttering I will include, in addition to the CPU/RAM/NIC you mentioned....

The Storage itself.... look at the Queue length in Resource monitor.... > number of disks in array is not good....and you want to see it goto zero even if it is a smaller number because that means the system was able to clean out the pipe.

The Switch has a play here.... you may need to restart it as well if it is confused about where packets need to go.

A single camera acting odd can imply that it needs the reboot like Ryan suggested.

Look in the exacq logs to see if you have video throttling errors. This was corrected in later releases. This could be disks with bad sectors. Stop the system, and run a disk check on the storage drive. Repair bad sectors and remove all of the locked files in the year date day tree on storage disks. I occasionally see this, especially with units that are not protected by ups for brownouts (why do integrators not provide ups for servers?).

Update version. If is too outdated, I believe exacq still charges 250 to make it up to date.

I couldn't agree more...

...with units that are not protected by ups for brownouts (why do integrators not provide ups for servers?)....

Our sales folks always inform the customers that a UPS is strongly suggested to help protect their system data. It is even higlighted on our Quick Start Guides.

We see far too many instances where power 'misbehaved' at a site and took out a RAID array or even the OS drive.

Sometimes a sites DOES have a UPS...but they don't set up the communication part to let the servers know that something bad is happening soon. This type of installation only helps with brownouts/spikes...not true power outages.

The issue has been resolved. After much digging around, I found in the logs that the system was throttling the cameras due to issues writing the data to the hard drives. Being that the NVR can only make use of 4GB of RAM (even the 64bit version) it also didnt have enough RAM to cache it.

I replaced the hard drives with new ones and now it works.

Thanks for all the help

1, thanks for following up!

What hard drives were their originally and what did you change to? Curious to see what made the difference?

I don't remember the exact model number, but the first set of drives were 3TB Seagate NVR drives. I replaced them with 6TB Western Digital Purple drives.

The WD Purple are decent drives for low count arrays. Low count means 8 or less drives. If you have more than 8 of these drives in a chassis...then it will cause other HDD errors you dont want to get.

Keep an eye out on the storage because you just doubled your available space and might not be back at the boundary where the throttling was happening.

A decent rule of thumb is to stay within 80% of the available array space...the 20% is your safe factor where the application is 'cleaning house of the old video' and the new data has to go somewhere.

I saw the severe limitation of 4GB RAM in my lab performance testing which is why our machines start out at 8GB. Boost the RAM in the system if you can.