72TB Storage Solution Needed?

Simply put... I have a project that requires 72TB of storage. Will be using Exacq or Milestone for VMS. Should I look for a big server? or should i be using a basic server with NAS?


How about a 256ch NVR which can take up to 16HD for total of 96TB

Depends... What is the brand, what is the turn around for service time, how scale-able is the system and cost. If it is off brand and it falls in the same pricing category as an Enterprise Server with a SAN then I am not interested. I will just spec in an Enterprise server with SAN and reap the additional benefits of Enterprise support, quality and scale-ability.

What you're after is not uncommon, and can potentially be done as a "big server", without the need to get into more complex architectures.

Some of the decision may be based on the number of cameras, which will determine the number of servers and the optimal config.

BCDVideo has received good recommendations from other IPVM members for storage servers, and help in selection.

Exacq and Milestone both also have server/appliance options, which may be a good choice more for support reasons than bottom-line price. The Milestone Husky is listed as topping out at 48TB, but they may be able to do a larger storage option.

Your best option is probably to nail down the VMS first, as final server config may be different for each.

After that, figure out if your camera count is going to dictate 1 server, or more than 1.

Then, you can do a more direct side-by-side comparison between options.

I think your choices have equal weighing tradeoffs. One big server will usually be less money and less space, but then it becomes a single point of failure for some major components, like power supplies.

We've used Seneca before and they've always been great. They had some major hiccups last year due to acquisition, but they've seem to have gotten over that hurdle.

The problem with Milestone Huskys is they only support RAID 5 or 10. And they don't support (officially) iSCSI for external storage.

We have had very great success with a vendor called BCDVideo (www.bcdvideo.com).

They can design a server/storage system and guarantee that it will meet the IO demands, etc.

I have no qualms recommending them.

Hi Jon,

I am assuming that the video you are talking about is to stay 'in-house' to your facility? If so, then in my opinion, I definitely look for some type of Enterprise level NAS solution(s). One thing to also consider when beginning to look at that level of storage is the type of I/O channel controller(s) used as depending on how many people, systems etc... are access the data/video the throughput rates are critical in overall performance.

Some basic areas I would consider if I was building such a system and would impact your decisions when planning your storage....

1 - How much of your storage will be considered 'active and available' all the time vs. 'inactive and for archival purposes only? Your 'active' storage areas could use the more expensive very hi speed and throughput technologies available while the lessor accessed video could reside on lower tech and lower cost JBOD (old IT term = Just a Bunch Of Discs)

2 - Remember as your planned storage increases so does your backup and archival schema planning must be taken into consideration. How much will be backed up? when? off-site vs. on-site? Where? etc.. when is the video considered 'obsolete' and it can be permanently deleted?

3 - Consider planning the storage hardware into physically separate, smaller manageable yet integrated groupings which can be easily taken off line and serviced if necessary.

4 - Use IT data farm best practices

Below are some quickie web links I found that may be of use when thinking about not only capacity planning but also performance and I/O throughput.

http://searchstorage.techtarget.com/report/Troubleshooting-and-identifying-data-storage-performance-bottlenecks

http://www.snia.org/sites/default/education/tutorials/2011/spring/file/Anjan_Dave_Storage_Mgmt_Spring_2011.pdf

http://intransabrand.com/pdfs/wp/Intransa-and-Genetec-Omnicast_wp.pdf

http://searchdatabackup.techtarget.com/feature/Protecting-petabytes-Best-practices-for-big-data-backup

http://searchitoperations.techtarget.com/feature/Data-center-operations-best-practices-checklist

good luck - Joel

Hi,

I have used Synology NAS on previous projects that require large storage and normally connect via iSCSI as the server then treats the NAS as a local drive.

https://www.synology.com/en-global/knowledgebase/DSM/tutorial/Virtualization/How_to_use_iSCSI_Targets_on_a_Windows_Server

Synology NAS are easy to setup and based on my experience the quality is good and fault rate is low. I have sold at least 400 Synology NAS in my previous role and only had 3 come back of which 2 were genuine faults.

Have a look at the RS3614xs+ to see if it is suitable for your project.

https://www.synology.com/en-uk/products/RS3614xs+#overview

I do not currently work for or sell Synology products.

I my experience iSCSI (NAS) solutions work but they are the entry level SAN solution. Having installed both Synology and QNAP NAS solutions as iSCSI targets I have found that they get bogged down in larger systems when you start reviewing a lot of video. We have also had issues where the iSCSI target doesn't always automatically reconnect when Windows boots. Since switching to Fiber Channel SAN solutions we have a seen a sufficient improvement to the point where you can't tell if the video is stored the local drives or FC SAN.

To the poster I would ask your customer if they plan to expand the system. If the answer is no then I would go with a server with built in drives. If they plan to expand the storage I would recommend server with a fiber channel storage solution.

Michael, to be honest I've never considered Synology and QNAP to be serious NAS products, even in the IT world. I did disk write speed tests with Synology and similar Iomega (since bought by Lenovo) StorCenter iSCSI NAS devices before and their performance was almost half of what we were getting from JetStor products we were using at the time.

I'm not saying Fiber Channel isn't a good transport, but it was another reaffirmation that you get what you pay for sometimes, and while I know a lot of SMB use Synology, QNAP, Buffalo and similar, I wouldn't consider they heavy duty players.

I my experience iSCSI (NAS) solutions work but they are the entry level SAN solution.

IMHO, iSCSI is not NAS. The access method is completely different.

There maybe be 'NAS' boxes which can be iSCSI targets and NAS, but iSCSI should not be lumped in with the typical file-level abstraction NAS drive.

More over there may exist iSCSI devices which underperform, like the ones you mention, but this is not due to their iSCSIness.

Jon,

I'm working on a project right now where we are replacing an existing Exacq server with a new one. At the same time we are expanding the number of cameras in the system. After looking al all of the recording requirements (bandwidth, retention time, number of cameras, resolution, frame rate, etc) we came up with a requirement for about 60TB. However, I have recommended and plan to order an Exacq Z-Series server with RAID 6 and an upgraded CPU. The Z series server I'm ordering will come with twelve 6TB drives, but the chassis will have eight unused drive bays for future expansion. I am not affiliated with Exacq in any way (just a happy customer).

@Tom what is the process for populating the rests of the drives? Do you expand the current RAID/disk group or do you build a second RAID?

It depends on your throughput requirement for the server. The higher the throughput, the more lower-capacity hard drives you will need. Use your VMS's server hardware calculator to determine the minimum hardware required for the server. Then contact a professional server builder to design the hardware. eDigitalDeals is a Milestone Systems partner, and this is how we do it, except we build the servers in-house.

The storage requirements for OnSSI and Milestone are now very different.

Milestone Corporate still benefits from the fast LiveDB drive array that gets Archived to the larger storage array several
times per day.... which is usually every 4 hours/6 times per day.
The Enterprise and lower code levels do not benefit as much but it is still a good idea to
archive the video because the archiveDB is easier to
repair.
Note that the Enterprise code level can not handle as high a throughput as the higher code level.

Your camera input bitrate, with how much you are keeping, will determine the actual size of the LiveDB space.

OnSSI uses Drive:/Dir structures and using multiple can provide performance benefits up to a point.
The overall bandwidth in addition to where Motion Detection gets run is a key sizing factor here.

Assuming a RAID6 solution, the 72TB will need 16 drives of 6TB...or 12 drives of 8TB.

OnSSI will need either a 12 or 16 bay server (assuming the OS is NOT part of the array which is true in our builds).

If using Milestone Corporate, lets assume 4 Fast RPM SATA drives in RAID10.
The overall size determined by the incoming bitrate.
Milestone Corp will need a 16 or 20 bay server.
Milestone Enterprise would match the OnSSi spec.

As others have said, your overall Bandwidth will dictate the practical storage requirements.
Higher BW will typically mean local, direct connect or higher cost network infrastructure.

iSCSI works a lot better in the Win Server 2008 and 2012 OS especially when you activate MPIO.

Veracity Coldstore makes an interesting storage option. You can use 6TB drives and fill the chassis for 90TB (84TB usable) or only fill part of the chassis with 8TB drives (10 drives) to get what you want (72TB usuable). The design usually has only two drives spinning at a time to reduce power, heat, and drive wear but can support reading from all the drives if simultaneous access to older data is necessary. The design allows for easy extraction of data by pulling a single drive. I believe they warranty all drives in their system for 5 years and will perform forensic recovery if you lose anything. We are just bringing up an ~400 camera site currently.