Subscriber Discussion

New Server Build For 40 X 3MP Samsung Cameras HDD Questions

UE
Undisclosed End User #1
Feb 09, 2017

I am working on a new server build for my Exacqvision server (thanks for everyone's comments on my thread earlier).  I have a question about my storage.

 

My server will probably be a Dell R720XD, 2 x E5-2660, 16GB ram, H710 raid controller, quad 1GB NIC.  For storage, would you guys recommend RAID 5, 6, or 10?  What hard drives would you suggest?  WD Purple?  WD Red?  WD Gold?  Something else?  I plan to use 8+ drives but the server can handle more.

 

I know, I didn't mention how much actual storage I need.  I am less worried about the amount of TB.  15-20TB will be plenty and I will be able to get that with just about anything I do.  I am more concerned about the type of HDD and the raid configuration.  If RAID 10 is suggested, I am ok with buying more drives to get the storage I need.

 

Thanks in advance for the help.

 

MM
Michael Miller
Feb 09, 2017

I would recommend SAS drives that are designed to work in RAID setups.  

(2)
UE
Undisclosed End User #1
Feb 09, 2017

What raid would you suggest?

UI
Undisclosed Integrator #2
Feb 09, 2017

I build my servers with WD Re 4TB drives in a RAID 6 array

(1)
MM
Michael Miller
Feb 09, 2017

RAID 6 will work fine.  I would recommend putting the OS on dedicated SSD 

(5)
UM
Undisclosed Manufacturer #4
Feb 10, 2017

What great in SAS for CCTV recording ?
Is it really that good ? Or like 120FPS vs 60FPS that good ?

Avatar
Jon Dillabaugh
Feb 10, 2017
Pro Focus LLC

Reliability 

UM
Undisclosed Manufacturer #4
Feb 10, 2017

Reliability between SAS and SATA are the same.
The only different in reliability is SAS SSD vs SATA HDD.....

(1)
UI
Undisclosed Integrator #5
Feb 13, 2017

Check the Mean Time Between Failure (MTBF) of SAS drives vs SATA drives.  WD Xe SAS drives have 2 million hours MTBF whereas WD Red have 1 million hours MTBF.

SAS supports more error codes.

SATA has fewer error codes and tends to shut down the drive for relatively minor issues.

SAS drives are generally designed to be clustered in a data center / server (though WD Red meet this as well) and generate less heat, dBA, and more.

All that said, I used a different VMS when "rolling my own" servers so my requirements were likely different than the OP.  For Milestone I would have 2x SAS SSDs in RAID 1 for the OS, 4x fairly small 10K RPM SAS HDDs in RAID 10 for the Live Drives (video recorded in last 24 hours), and the remainder being fairly inexpensive sizable 7200 RPM SATA drives in RAID 5 or 6 for archiving.  This was for a max of 80 cameras when 5 MP resolution was the best available, fairly rare, and no smart codecs.  AFAIK 3 years later there have been no server failures and only a couple hard drives have burnt out across 50 or so servers.

(4)
(2)
UM
Undisclosed Manufacturer #4
Feb 14, 2017

Yes, SAS support more error correction.
But as I said before CCTV using a low IOPS environment.
Not so much different for lower error rate.

About the MTBF. They (HDD manufacturer) told us the new product is greater than old product. What a shock.

They told us SAS is work for 7x24 but not like SATA working for 8X5 too, right ? This absolute is not the true. Everyone know that.

5MP x 80cameras with 5Mbit/s CBR ? 400Mbit/s writing speed.
Good design but a little overkill in raid10.

And some of our clients using SATA for over 5 years and no failure at all.
(3array, RAID5 , total 36pcs SATA)
Personally I really don't trust MTBF too much.

(2)
UI
Undisclosed Integrator #5
Feb 15, 2017

"About the MTBF. They (HDD manufacturer) told us the new product is greater than old product. What a shock."

No offense, but as a distributor I am willing to bet you are pushing whatever will make the sale as well.  Two sides to the trying to make money coin.  

The fact that enterprise SAS drives have a 5 year warranty whereas the SATA drives have a 3 year warranty stresses the manufacturer is willing to back it up with some cost absorption on their part. As an integrator I don't want to service the server in 2 months at my cost, which is far greater than the cost delta between a SAS and SATA drive.  This particularly true when these servers could be on a different continent.  Rarely does a distributor take that risk.

 

"5MP x 80cameras with 5Mbit/s CBR ? 400Mbit/s writing speed.  Good design but a little overkill in raid10."

Perhaps.  Keep in mind that at least once a day Milestone is both reading and writing to those RAID10 drives for an indeterminate time period.  A push from the live drives to the archive drives happens on a manually set interval.

 

"And some of our clients using SATA for over 5 years and no failure at all.  (3array, RAID5 , total 36pcs SATA)
Personally I really don't trust MTBF too much."

I would consider that anecdotal.  As I said we used SATA drives for the archive.  Rarely was there a failure but it did occur.  If you are so confident that a SATA drive will not fail why not just use RAID 0 or even JBOD?  This would avoid the overhead of RAID 5, the need for a RAID controller, the loss of a drives worth of storage, etc.  That's more cost that could be shaved ;)

UM
Undisclosed Manufacturer #4
Feb 15, 2017

I mean you no disrespect.
But the first thing is SAS have a 5 years warranty, SATA have a 3 years warranty is not true.
WD RED Pro is SATA only and it have 5 years warranty.
HGST ultrastar 7K600 also have SATA version and 5 years warranty too. Both version have the same MTBF.
https://www.hgst.com/sites/default/files/resources/Ultrastar-7K6000-DS.pdf

And a lot of HDDs have SATA and SAS version and both have the same MTBF, right ?
So. It's not logical to think that SAS can bet SATA in MTBF.

For a high IOPS purpose storage. I'm always choose SAS 10K.
But in CCTV. It's just like the topic in other discussion.
 'Sell Using Negative Emotions?'
End user worry about SATA is not good enough.
It's all about calculation.

And the Milestone thing.
You are right. It should around 20% write 80% read in milestone.

Oh com'on
I never said that SATA will not fail.
It just SATA & SAS technology should have the same error rate only.

Maybe some HDD manufacturer will design a SAS HDD with a lot of good feature/material to prevent data loss/increase the life time.
But it's not the true about SAS technology is more reliability than SATA.

UM
Undisclosed Manufacturer #3
Feb 09, 2017

Put your OS on WD black drives, in RAID 0.

Put your recording drives in WD Re4 (enterprise class) drives and configure them in RAID6 (so 2 more drives than the actual required recoding volume- these 2 extra drives are required for calculating parity).

Get a redundant power supply for your server. 

Have your server conected to some type of back-up power (at least a UPS, maybe even a UPS connected to a generator)

You will NOT have any server downtime in this scenario, and you won't break the bank.

UI
Undisclosed Integrator #2
Feb 09, 2017

I agree with all that UM3 said but I build mine with mirrored SSD's for the OS

(4)
UM
Undisclosed Manufacturer #4
Feb 10, 2017

Really ? no server down ?
What if layer 7 error make service down ?

(1)
Avatar
Jon Dillabaugh
Feb 10, 2017
Pro Focus LLC

I second the SAS drives for storage, as they are designed for RAID usage. Don't skimp here.

I also say go for dual SSD (RAID1) for the OS. The speed for bootups is nice without having to use real RAID. Having the mirror drive gives you online redundancy.

Sounds like a nice server and should be a beast.

UM
Undisclosed Manufacturer #4
Feb 10, 2017

Normally always using RAID 5 in CCTV. RAID 6 write performance is too low."
RAID 5 + hotspare is work enough


Any HDD work fine in surveillance . Unlike IT industry. We are using very very low of IO.
Recording storage always connected by a single server or using internal storage.
A very few operators will read data from storage simultaneously.

I would not suggest using RAID10 in normal case.
It's kind of waste of money and performance.

(2)
(4)
UM
Undisclosed Manufacturer #4
Feb 14, 2017

For anyone who interest about RAID5,RAID6 writing speed

EP
Eddie Perry
Feb 14, 2017

Some math......

40 x 3MP samsung cameras at H264 main profile no smart codec = ~186-192 MBs peak ( round to 200 MBs for all I care)

Dell Poweredge 710xd - Intel C600 chip set with QPI, No losses, bottlenecks there to worry about 6/GT's

                                        no problems here

 

Raid controller H710 PERC - Data transfer rate 6GB per sec per disk/port

                                 No problems here either

                                              

                                              All drives are 4TB     

Write speed

Disks  Sata- 7200rpm WD red NAS pro 202 MBs sustained ( not peak)

                 - 7200 rpm WD gold 182 MBs sustained ( not peak)

                 - 5400rpm Purple  150 MBs peak ( best guess about 90mbs sustained)

                 - 7200rpm Black (R 2.0 with 128 cache) 218 MBs sustained)

          SAS- 7200 rpm WD Gold  182 MBs sustained ( not peak)

                - 7200 rpm Segate Iron wolf 214 MBs sustained ( not peak)

                - 7200 rpm HGST Ultrastar 7K4000 172 MBs Sustained( not peak)

 

Raid 6 vs raid 10

12 4TB drives just cause is more than 8,

Usable space Raid 6 -40TB, Raid 10- 24TB

Max raid write speeds across array Raid 6, 10

WD red - 1616 MBs, 2204 MBs

WD gold - 1456 MBs ,1986 MBs

WD Purple- 720 MBs , 982 MBs

WD Black - 1744 MBs , 2378 MBs

WD Gold SAS-1456 MBs , 1986 MBs

Seagate Ironwolf- 1712 MBs , 2335 MBs

HGST Ultrastar- 1376 MBs , 1876 MBs

 

subtract 200 MBs from any number above to see how much wiggle room you have.

 

final thoughts

7200rpm Sata drives have caught up to 7200rpm SAS drives in this day and age Unless you want an extra hefty warranty that comes with the SAS drives and the head ache that comes with it when you have to replace that drive with an exact one in a couple of years you will be fine with Sata drives for something Small like this.  nice reliable Sata drives not cheap ones....

as for Raid 6 or 10. well that's up to you if you need more space and there is someone that looks/ takes care of this thing at least once a month- 3 months, then I would say raid6 shouldn't be a problem, if no one is going to look at this server until something happens between years then raid 10 is a better choice as your drive failure tolerance is greater with raid ten by about 4 drives.

Raid 6 more storage, 2 drive failure,

Raid 10 ~half of raid 6 storage, 6 drive failure.

 

 

 

 

(1)
(3)
(2)
EP
Eddie Perry
Feb 14, 2017

oh yeah I forgot that is at 30 fps for each camera

(1)
UI
Undisclosed Integrator #2
Feb 14, 2017

Eddie,

It seems that you are not aware of the difference between MBPS and Mbps.

MBPS stands for MegaBytes per second

Mbps stands for Megabits per second

 

1 MegaByte = 8 Megabits

 

Using the above reference your  statement above:

"40 x 3MP samsung cameras at H264 main profile no smart codec = ~186-192 MBs peak ( round to 200 MBs for all I care)"

should really spell out: 40 x 3MP samsung cameras at H264 main profile no smart codec = ~186-192 Mbps peak ( round to 200 Mbps for all I care)"

 

When speaking about bitrate we measure in Mbps, when speaking about hard drive read/write speed we measure in MBPS.

 

For example, using your numbers above:

"7200rpm WD red NAS pro 202 MBs sustained ( not peak)"

That drive at 202 MBPS would translate to 1616 Mbps. at your 200 Mbps max write requirement for the above 40 cameras. That's 12.38% of your sustained Read/Write speed.

 

I hope this helps.

 

(1)
(1)
UM
Undisclosed Manufacturer #4
Feb 15, 2017

You are using 90%read for your calculation. It's almost never happen on CCTV server.
Try 10% or 5%

UM
Undisclosed Manufacturer #4
Feb 15, 2017

BTW. This test is using SSD and your calculation of raid6's write speed is KO his reading test from IOmeter. Reality always hurts.
If you want to going on IOPS or high write speed, H710 is too slow. (H710 is enough for CCTV)
And you know that it will drop a lot after using Seq. write test, right ?

https://blog.xbyte.com/2014/08/14/testing-the-limits-of-the-dell-h710-raid-controller-with-ssd/

(1)
EP
Eddie Perry
Feb 15, 2017

yeah I am well aware of how Dells H710 Perc controller list's "Data transfer Rates" As "Up to 6GB/s" can be miss leading. But for what the Original poster requirements are and the fact hes not going to use SSD's  Its still well within the disk max write speeds that we deal with in the real world for SAS and SATA drives. 

Didn't feel the need to go into full detail for a reply post though.

 

Although It would make a great full article or "books/Guides" that IPVM puts out, but with as fast a technology changes it would be obsolete in 6 months. 

 

UE
Undisclosed End User #1
Feb 15, 2017

A dell raid card H710p with 1GB ram will max out around 50,000-60,000 IOPS with SSD enterprise drives in RAID.  I know those because I have servers running this configuration and have ran several tests on the storage array.

SAS/SATA spinning drives cannot push this raid card.

UM
Undisclosed Manufacturer #4
Feb 16, 2017

May I know this IOPS is random or seq write ? 4k or 512 ?
Testing tool is crystal ? HD tune or IOmeter?
60000IOPS = 234.375MBps. Single SSD can do better than this.
It's look like your bottleneck is H710p if you are using RAID5/6 in your server
Or maybe SSD ?
RAID 5 SSD should at least 6 digits IOPS in random write.

EP
Eddie Perry
Feb 14, 2017

Undisclosed Tardegrator #2

I wasn't counting networking traffic as write speed other wise I would have typed Mbs and referred to it as network traffic. but I didn't, why is that?

Now upon reading your response I realize that  the people who are uneducated or dont deal with server design on a daily basis might miss under stand what I wrote there.

For them I will clarify'

200 MBs is based on the hand experience I have with using Samsung/Hanwha cameras in the 2-4MP range with recording options such as software based recording. and ~200MBs means 200 MegaBytes per second not to be confused with network traffic 200 MegaBits per second. 

You see when you do calculations it helps to use the same units of measurement to arrive at correct numbers. otherwise you just have a huge mess on your hands.

I hope this helps

(1)
UI
Undisclosed Integrator #2
Feb 14, 2017

Eddie,

So based on your numbers you are stating that each of your cameras is streaming at 40 Megabits per second...

 

Something is wrong with that especially if the cameras are 2-4 megapixel cameras.

 

Those numbers are astronomical even when recording in MPEG format.

New discussion

Ask questions and get answers to your physical security questions from IPVM team members and fellow subscribers.

Newest discussions