Subscriber Discussion

Hard Drive Confusion

UI
Undisclosed Integrator #1
Jan 31, 2017

Can someone help clarify a big hard drive confusion for me? How do I calculate if my setup requires 7200 RPM drives, 15k RPM, or SSD drives / RAID?

Let's say on a 100 camera example, with avg of 7 Mbps, that's 70Mbps that's being transferred to the disk. Can a single 7200RPM drive handle the writes of that camera to the VMS?

 

 

(1)
Avatar
Josh Hendricks
Jan 31, 2017
Milestone Systems

There're more variables to the equation.

Does the VMS record to the live path and then archive elsewhere on other storage? If so, you will have up to 70Mbps writes and up to 70Mbps reads, simultaneously.

How much playback is expected (number of simultaneous clients and streams?)

Are you recording always or on motion/event?

If event based, estimated percentage of video that is actually recorded?

If event based, does the VMS allow you to buffer to RAM, or does it only prebuffer to disk?

I'm with Milestone, but our presales guy Jared Tarter is on here and would probably give more accurate numbers. My rule of thumb is a 7.2k rpm drive will not give more than 7MBps and a 15k rpm drive will not exceed 15MBps.

That's an over-simplification but I think it's in the ballpark. It also gets sticky when you start worrying about raid 5/6. Raid 0/1/10 scale relatively predictably.

The answers to the above questions will make a significant difference to the end result. The best thing to do is use the VMS vendor tools or services to build a spec ahead of time. Not only will it give you peace of mind, but speaking from experience it gives you tremendous strength if you are stuck dealing with what appears to be a performance problem after executing based on written recommendations.

SSD is a long story with varying schools of thought. If you use SSD for live storage, just make sure you are using enterprise grade SSDs. There's definitely a difference in both performance characteristics and longevity.

RS
Robert Shih
Jan 31, 2017
Independent

To quote the inane retort of our childish president: "WRONG!"
A WD Gold 10TB hits 249 MB/s sustained.

Even a Black can do that, but overall, 7200rpm drives are no slouches.

A WD Purple @ 5400rpm still goes 150 MB/s.

(1)
(2)
(1)
Avatar
Josh Hendricks
Jan 31, 2017
Milestone Systems

"The 10 TB drive offers a 249 MB/s sustained sequential transfer rate"

I would love to see that, but in practice I'm not sure that most VMS workloads are sequential enough to come close to that.

There may be a VMS with better media database performance than Milestone but I can only speak to the way our database works. All database block files are pre-allocated in contiguous space to avoid fragmentation. In Corporate, there are multiple "columns" of data stored for each camera. Besides video there is audio, motion metadata, and motion/recording timeline metadata.

With many cameras on one server, even if you have a pure write load, the HDD head is still going to have to jump around some, and as soon as you do that, you murder the manufacturer sequential performance specs.

That said, I have not done HDD performance testing myself in years and the drive may very well do more than 10-20 MBps on a surveillance workload. Whoever the VMS is, they will be the best source for a spec.

 

(1)
Avatar
Joseph Parker
Jan 31, 2017

Some hard learned lessons on Raid sets:  Raid 5 is not reliable enough, even with a hot spare.  I've learned that lesson twice now, so don't do it unless you incredibly randomized HD batches.  RAID6 is reliable, but takes a beefy controller.  RAID 10 isn't as fault tolerant as 6, but better than 5 and it gives you write bonus so you can use slower drives.  I have close to a 100 cameras recording to a RAID 10 array with 5400 RPM drives, no issues.  Raid1 isn't a bad idea for smaller systems, especially with faster drives.  One other piece of advice, use a separate LUN for your OS.  It's safer, faster, gives you far more predictability in your storage array utilization.  

(1)
MM
Michael Miller
Jan 31, 2017

What VMS are you using where you are having issues with RAID 5? 

Also, I would recommend using dedicated drives for the OS not just a separate LUN if you want maximum performance 

Avatar
Joseph Parker
Jan 31, 2017

The problem with RAID 5 isn't specific to the VMS.  The issue is the single disk fault tolerance combined with incredibly slow rebuild times.  I've had 2 arrays fail in the last couple years, and the last one was just over a year old.  Both were due to multiple disc failures (WD Purple NE and Gold).  AND we use hot spares, so as long as the drives don't fail with a couple days of each other there shouldn't be an issue, except that multiple simultaneous drive failure within a manufacturing batch isn't actually uncommon.  As far a dedicated drives, that is what I meant.   I may be using LUN incorrectly, but I've always referred to it as a separate RAID set on the same RAID controller.  On our typically 8 bay build we'll do 6 storage drives and 2 OS drives using RAID1.  

MM
Michael Miller
Jan 31, 2017

The VMS could play a part in this depending on how they write video to the storage drives.    I have RAID 5 setups running for years without issue.   Though we are using Enterprise grade SAS drives.  Every time I have tried the WD Purple drives we have had issues so we stay away unless they are small single drive systems. 

For our large SANs we use SAS drives with RAID6 with a max of 12 drives in a RAID.

(1)
UI
Undisclosed Integrator #1
Jan 31, 2017

Hi Michael,


So what drives are you using if not the WD Purple?

U
Undisclosed #3
Jan 31, 2017
IPVMU Certified

RAID 10 isn't as fault tolerant as 6...

How so?

Avatar
Joseph Parker
Jan 31, 2017

In a 6 disk array both can have 2 fail and still function, but with RAID 10 you have clusters, in this case 3 and 3.  If you lose 2 disc from one cluster, you are boned.  I'm not good with odds making, but this setup give you a much higher chance of loss than RAID 6. 

(1)
Avatar
Josh Hendricks
Jan 31, 2017
Milestone Systems

In a six disk RAID 10, if you lose a disk, there are 5 remaining, two in one set and 3 in the other. That leaves a 60% chance (3/5) the second failure results in data loss.

So RAID6 has a risk advantage over 10, but generally 10 will have a speed advantage. We often recommend raid 10 for live storage and raid5/6 for archiving.

Avatar
Joseph Parker
Jan 31, 2017

Absolutely.  I believe  on a 6 disk array you get a 4x write bonus.

U
Undisclosed #3
Jan 31, 2017
IPVMU Certified

For smaller arrays this is true, but as the array gets larger the exposure during a rebuild increases dramatically because of the time and the number of drives with RAID6.  For example 12 drives.

Where as RAID10 can tolerate up to n/2 drive failures, even during rebuild, which take only as long as it does to copy a drive.

Surveillence in general would typically be smaller arrays, IMHO, so I agree with RAID6.  

Avatar
Joseph Parker
Jan 31, 2017

"I have RAID 5 setups running for years without issue."

Me too, until I did.  We use Dell servers, and Dell actually strongly recommends against RAID5 due to it's high failure rate.  I carried on knowing that, and boy did I pay the price.  I will agree about the Purple Drives, they are now relegated to non RAID configs only.  

I've not yet had the chance to use a SAN on a camera deployment (one day!), but our IT division uses them pretty often.  We just did a 30 bay EMC SAN with 15k SAS drives, 3 SSDs for caching,  and 10Gbps lan cards for a TV station.  It sounded like a fighter jet taking off on startup, pretty amazing piece of tech.

(1)
EP
Eddie Perry
Jan 31, 2017

This is very hard question to answer as it depends on multiple factors:

 

Hardware- motherboard configuration and raid controllers have hard limits on how much data they can transfer at any given time. this is further limited by ram speed (although most motherboards today are estimated based on max ram speed any way.

        your motherboard may only allow 100mbps to pass though the north bridge (bus) onto a raid controller that will only allow 80mbps to pass though its bus onto a 7200rpm hard drive that can only write 150mbps max.( WD Black) now I made these numbers and know that there are much faster designs like Intels QPI (no slow north bridge) and far faster raid controllers. but my point is any time you have a choke a point in your hardware that is the fastest you can write to disk. networks cards, also can effect you speeds as well if they not built into the motherboard.

 

Software- yep you use software to write to disk, so if you have a software limit of 5mbps doesn't matter if your hardware can handle 300mbps, 5mbps is the speed limit. this occurs normally in software Raid setups which I am highly against for video. it can also be based on how many threads of data streams ( i.e grab video streams 1-10 and write them to disk now) you have going on at one time. if your software (drivers, firmware, etc) doesn't make the full use of your hardware you are not going to hit the hardware limit.

 

Network- cloud, NAS, SAN etc are all remote ( as is not part of physical machine that is doing the work) has limitations as well. need to shove 100mbps of data to a NAS over a DSL line good luck with that,it would be like trying to fit into your old clothes when you were 8 years old.  how about a 10mbps over a 1G fiber connection to a SAN, it would like a hotdog in a hallway.

 

I am against SSD for video still haven't seen anything that is cost effective yet

(1)
U
Undisclosed #2
Jan 31, 2017

Good comments! the more people contribute the more we learn from each other. My contribution is this, from the top down or bottom up look at each modular piece and find a way to understand it. Compare case studies and lessons learned, you are never alone in success and failure. Especial in technology which allows us to regroup, alter and develop new features based off of our tweaks, mistakes and odd requests that challenge our way of tinkering around the garage. Always build the Rolls Royce along side the Volkswagen so you are aware of the value in both.

(1)
Avatar
Jon Dillabaugh
Jan 31, 2017
Pro Focus LLC

Purple drives are good for smaller installations, but you will learn (as we did) that they aren't fast enough to support 100 cameras. In fact, I wouldn't try to record 100 cameras on one server anyways. I would divide and conquer. Split the load between multiple servers. Even throw in a hot spare server if your VMS allows for failover and it is critical not to miss events. Everything is relative.

New discussion

Ask questions and get answers to your physical security questions from IPVM team members and fellow subscribers.

Newest discussions