Subscriber Discussion

Server Hardware/Milestone Question

JH
Jay Hobdy
Jul 01, 2017
IPVMU Certified

So I am sitting here remotely trying to install Milestone LPR onto a clients server and it is taking forever. I am an hour in (part of that is I probably forgot to go back and look) and still on the VMS install, not including the LPR plug in or server.

 

So I started researching if the server had the right specs. I should also note the server hard drive is 100% full. Not sure if this contributes to slowness. The server is also running Unifi software with 25 cameras

 

The server is a Intel(R) Xeon(R) CPU E3-1245 v3 @ at 3392 MHz (x 8)

 

 

Milestone calculator for 2 LPR cameras suggests

Intel Xeon E5-2603 v3
 
8 GB RAM
 
2 Gigabit NICs
 
Windows Server 2012 R2 x64 Standard/Data Center
 
 
 
 
Does this seem right? Is the clients server undersized?
 
 
JH
John Honovich
Jul 01, 2017
IPVM

As for the taking forever to download part, related: VMS Installer Sizes Getting Out of Control.

I'll defer to others on the server sizing.

(2)
JH
Jay Hobdy
Jul 01, 2017
IPVMU Certified

I have 10 of these sites we setup in the last couple months, and Milestone does take a few minutes to install, but this particular server is absurd.

 

The client's IT department just put this server back in action so it is going to be hard for me to tell them this server stinks. I have other servers I can bounce around pretty quickly.

 

 

 

 

Avatar
Josh Hendricks
Jul 01, 2017
Milestone Systems

I don't think our server calculator currently provides specs for add-ons like LPR. Our presales team are much better at the sizing game than I am, but I'd probably add ~4-6 cameras to the spec for every LPR camera used. That should keep you in the ballpark given LPR will try to decode at a rate of 4fps by default.

I'm not sure how you generated the spec for 2 camera LPR, but the CPU you have is ~87% higher specced at 9581 passmark benchmark score compared to the specced CPU.

Its a bit of a budget proc these days, but would probably would do LPR on two cameras just fine. But you will definitely have problems if the OS disk is full, and I can't say what the impact of running a 25 camera Unifi package on the same server will be, but it will definitely eat at your available CPU time.

If the installation was running slowly, I'd wager you might be recording to the OS drive. With that much IO, there wouldn't be a lot of overhead available for extracting the installer package or installing components. The installation would probably be much faster if the Unifi software could be stopped for the duration of the install.

Also if you're recording to C: (in anyone's VMS) it would be good to add some disk specifically for recording so that you're not spinning up your OS disk for surveillance at all.

(1)
JH
Jay Hobdy
Jul 01, 2017
IPVMU Certified

One problem we realized.... we were using H264 on our LPR's and now I realize the issue with I/Key frames and the need to use MJPEG. So today I am logging into all the servers and changing cameras to MJPEG but I can only choose 1 3 or 5 FPS on MJPEG. Milestone states to use 4. Should I do 3 or 5? I chose 5.

 

 

Now on the slow server I just got

 

 

 

E drive is partitioned for Milestone

 

I figured having the drive full was slowing things down 

Avatar
Josh Hendricks
Jul 01, 2017
Milestone Systems

I don't recall off the top of my head if the SQL installation will also be done on the E: drive if you choose to install to a different drive. I should hope so, and I'm fairly sure it does.

But with a 100% full OS drive you can't be surprised by any failures. Windows itself will struggle to stay stable and operational at that point.

I'd recommend clearing some space on the C drive and giving it another go. And I suspect stopping the existing VMS will speed things up greatly.

With a 100% full 3TB OS drive, that's going to be a major reason behind the slow installation process. Any time you have high IO activity on your OS drive, you're going to find everything (including simply launching Windows components like Control Panel) are going to be much slower than normal.

(1)
(1)
Avatar
Jared Tarter
Jul 03, 2017
Milestone Systems

Hi Jay,

In regards to the settings for LPR, the default value of the number of frames that our LPR software analyzes per second is 4fps.  However, that value is adjustable and doesn't mean the camera has to be set to 4fps.  The camera can be set to anything higher than that value.  If the LPR is for a situation where the car has to drive vary slowly (e.g., entering a parking lot or parking garage) or even come to a complete stop (e.g., a parking lot/garage where they have to wait for the arm to raise), then you could easily get away with analyzing fewer frames per second because the car will be in the FOV longer.

You will have to do some "trial and error" because LPR definitely isn't a simple plug-n-play technology.  It takes some tweaking and tuning.  I would recommend reference the LPR second in the administrators manual for whatever XProtect software you are installing.  There is a lot of information in there covering angle of view, pixels on target, shutter speed, etc.

For the SQL Express error you got, it is a fairly generic error from SQL that can mean a number of things from permissions issues, to a setting in SQL that is preventing our software from doing what it needs to do (this could be the case if SQL Express was already installed by a different application and you are trying to reuse that instance), etc.  Installing one VMS on a machine with another VMS almost always causes problems.  If you haven't solved this already, I would suggest opening a case with tech support so they can take a look in the logs to see if there is further information surrounding that error.

Also, as Josh mentioned, the OS drive being full can cause very weird behavior in Windows and can definitely cause things to run incredibly slow.

(3)
JH
Jay Hobdy
Jul 05, 2017
IPVMU Certified

Thanks Josh and Jared.

 

I discussed this with the client and they agreed with everything you mentioned, the Unifi running on same drive while trying to install Milestone, the drive being full, etc.

 

They are going to take the server back and install a SSD drive for the OS.

 

As far as the LPR, we changed the FPS on all cameras, and will be taking new snapshots this weekend and seeing how it goes. Definitely not a plug and play item... Thanks 

Avatar
Mike Dotson
Jul 05, 2017
Formerly of Seneca • IPVMU Certified

Jay,

From a hardware performance perspective for any VMS, you do NOT want to create a partition on your OS physical drive due to the contention of multiple services wanting to get to a single physical drive.  

The chart shows a total inbound network value of appx 400Mbits, which ..if it is mostly the camera streams.... is in the realm of 'dont do it'.

The SSD will help there, however if you are recording to the SSD as a result of partitioning, be very aware of the 'max writes' or other wear level specs on the SSD.

They really do wear out and quit working.   A workaround that will delay this is to use a much larger SSD so that the 'self healing' that enterprise SSDs perform has room to reprovision itself.

If this server can be redone with a OS drive and a separate physical drive for the video streams, you will be better off.

 

(1)
JH
Jay Hobdy
Jul 30, 2017
IPVMU Certified

Just wanted to update.

 

Client installed a SSD drive for the OS. All VMS records to a different physical drive.

 

Absolute night and day difference. Now when I click on something, it actually opens.

 

When I discussed this with the client, they knew it was an issue, it just took me whining for them to resolve.

 

Milestone loaded like a champ after that.

 

Could there be an issue with 2 VMS writing to the same physical drive? The drive is partitioned but I have to assume if Ubiquiti is writing 25 cameras to the drive, and milestone jumps in and try to write, they could bump heads. Or is Windows smarter than that?

Avatar
Josh Hendricks
Jul 30, 2017
Milestone Systems

The two VMSs writing to the same physical disk(s) is only a problem from a performance and hardware longevity perspective. If they're both recording 25 cameras it's similar to one VMS recording 50.

With caveats of course. Different VMSs will have different read/write strategies. Even if both use reasonably efficient strategies individually, they may interfere with each other enough that the resulting IO becomes much more random than if a single VMS was managing the same number of cameras.

 Edit: This is just an educated guess based on limited knowledge. For example, if you had one VMS which can buffer to RAM, and media files are preallocated, it would likely handle 50 cameras more efficiently than if 25 of those cameras were on a different VMS without RAM-based prebuffers or with a different file allocation strategy as the software with lower efficiency will (probably) effectively limit the performance of the other sharing the same disk(s).

Avatar
Mike Dotson
Aug 25, 2017
Formerly of Seneca • IPVMU Certified

We came across another situation from one of our customers where they had two VMSs pointing to the same storage array (Drive Letter) recently.

One of the VMSs assumes that it 'owns' the array, preallocates all the space then marks it 'Read Only'.   The other VMS was also pointed to the same storage, and of course could not write to the array.

(2)
JH
Jay Hobdy
Aug 25, 2017
IPVMU Certified

we actually have one physical drive for storage (besides the SSD) but it is partitioned into 2 drives. I wonder if that would help here?

Avatar
Mike Dotson
Aug 25, 2017
Formerly of Seneca • IPVMU Certified

It would be a workaround to the issue at the expense of performance as both VMSs would be Writing/Reading from one physical drive at the same time.

If the bandwidth needed for the VMS is low (50Mbits for each VMS) ...then this would possibly be OK performance wise.

New discussion

Ask questions and get answers to your physical security questions from IPVM team members and fellow subscribers.

Newest discussions