Storage Issues In HD Surveillance - 200 Cameras, 3 Months Reliable Retention, Archiving - Suggestions?

Archiving as in offline storage or backup of specific events?

I don’t work for them, but for long time storage I believe Veracity Cold Store has a great solution. 

Well there are always a lot of factors involved and most commercial projects don't end up buying beyond 15-30 of retention, unless they are required to by some agency.

You would need to know if you are going to be archiving the full stream (hopefully not), events only, can you prune down frame-rates and resolutions if so at what time periods, and where are you storing it: on-site, replicating off-site, long-term archiving locations? Now you need to also factor in bandwidth, energy costs, maintenance, and what type of media would be the most cost effective, and a lot of other things.

What VMS are you looking for or is this just CCTV storage in general?

I'm going to cover these a few questions/responses at a time:

Retention, archiving and retrieval have not evolved much IMO.

Are you referring to the VMS software or hardware solutions? 

From a hardware perspective, we are talking about PC or appliance based video stream recording applications here. This is a very small technology category, so it is generally going to rely on technology that exists everywhere else first. At the end of the day, most systems are going to utilize spinning drives in a chassis, because that's all they need.

From a software or solution perspective, I'm not sure what a Linux or Windows-based VMS could do to evolve the processing of data streams to hard disk. One solution example is the rise of VSaaS manufacturers, relying on the cloud for retention, or a hybrid local storage with the cloud as long term retention (IPVM Directory of VSaaS / Cloud Video Surveillance Providers)

I would like to see a discussion about the state of storage in 2018. What solutions exist for storage in our sector? Archiving in particular. We require massive amount of storage 200 cameras and 3 months of reliable retention is not trivial. Archiving is another complicated issue at least from my point of view.

Have you experience issues with an existing installed system that has been prone to failure when trying to retain 3 months of video reliably?

This is where storage discussions really become relative. 200 cameras for 3 months to many people would be massive amounts of storage. I am sure there are manufacturers and integrators on IPVM that are managing 5,000-10,000+ camera systems that require a whole additional level of scale. More on this below.

Waiting to hear from the collective. Would like suggestion and experiences about strategies, brands, practices, etc ...

For that specific example, if I was looking at retaining 200 1080P cameras for 3 months, with a mix of continuous and motion recording, I would ballpark estimate between 200-300TB of storage. Depending on the physical architecture (multiple buildings, multiple campuses) you might look at 3 or 4 basic COTS servers, or surveillance-focused OEM manufacturers like BCDVideo (IPVM Profile), or Seneca can work with you to design a custom solution for your situation.

Higher up the market from a complexity standpoint, are storage vendors who may or may not have surveillance marketed offerings, but focus on VMware, all-flash storage arrays, video stream hosting, 3D CAD operation, etc. These are names you've heard of like Dell EMC or HPE, or lesser know, manufacturers (in the surveillance world) like Netapp, PureStorage, Nutanix, and many, many more.

To summarize it, I mostly I agree with where you're coming from, if you want your storage solution to be confusing, it's a pretty quick walk to get there. However, this could just be a walk to avoid and keep it as simple as possible.

Storage is the cheapest it has ever been. Issue is labor to retain and labor to review once it is archived. 

Of no event has happened, what is the point of storing the video forever?

most cost effective , hardware and labor wise, is size your storage to your needs. When you do have an event, have a dedicated place to store the archives for desimination to those that need it 

The Veracity Coldstore solution is probably the best around for long term storage and retention in terms of reliability and cost effectiveness.  It is also amazingly Eco friendly as its power consumption compared to a sizeable RAID 5 etc solution is incredibly low, which users really like and look carefully at these days.  It's about 60w.  They also give 5 year warranty on their box and drives?

Be sure to consider how the data will be protected.   There are a few different ways this is done these days beyond the basic RAID method.   As for RAID, be sure to use RAID6 for the large array sizes you will be using due to the rebuild times that exist now with the larger hard drives.   RAID5 will leave you exposed to a 2nd drive fail during a rebuild whereas RAID6 will not.

The good thing about the traditional RAID method is that there are many choices.   I will let others comment how well the alternative storage solutions perform.

Quantum has a good solution for large-capacity, long-term storage. They use a data management platform and tiered storage approach. I know they work with LENSEC and Perspective VMS.

The HDD keeps increasing the capacity, and now 12TB per drive is available. The trend will continue. We at Rasilient focus on large number of camera count and PB storage projects. We have a 5U chassis that can store 1PB.

The question is what happens when the 12TB drive fails. How long it takes to rebuild the RAID? If it takes a week, what happens another drive fails?  Does the performance suffers during rebuild?

Our ZM, Proactive Cloning technologies are designed just for that. It first deals with "soft errors" or "transient errors" from the drives, so there is no waste of time to replace a drive that can be recovered. Secondly, the "soft errors" provide good insight of the state of the drives. We can do online proactive cloning before the drives are actually failed. So, we don't enter into RAID rebuild.

Retention, as said above, is often underlooked during planning/sales and overlooked during installation.

So many variables in what your retention will be. Internally I've come up with a rough ballpark - 2.1mp, camera, 4-8 FPS, 2kbps require approx. 750gb per 90 day period. Highly variable on the scene complexity, motion intensity, etc. Outdoor cameras will require more, etc. This estimate can vary WIDELY but it's a good starting point for during the planning stages.

We get our clients to 90 days with the above guidelines. 200 cameras would rough out to be 150TB of storage. Not accounting for moving footage to archival etc.

Are you running blade servers or NAS boxes or on a SAN? What VMS are you using? The various ones will have different natively integrated/supported methods of external storage. 

For archiving, are you using internal RAID, or pushing the data out to one of the above?

Good question.  From what I see most systems are not installed with the proper storage, backup and disaster recovery processes in place.  I have not used Pivot3 however I have read up on them over the past month or so.  

Pivot3

I believe Iron Mountain "OEMs" Pivot3 for their offering.  

Iron Cloud

Good luck and I would like to know what solution you decide to go with if you do not mind posting it once you know.

First, check the VMS's minimum hardware requirements based on frame rate, bit rate, and expected motion hours. The CPU and GPU you choose should have a higher benchmark score than what is recommended by the VMS calculator. Then, design a server with as many points of redundancy as possible: double or triple-redundant power supply (will be very loud, so ideal for a private equipment room) and dedicated RAID card with BBU with hot spare(s) are required.

The server's storage devices should support the use of each partition, e.g. RAID1 SSDs or SAS drives for the OS, SAS drives for short-term recording, and either SAS or enterprise-grade SATA for long-term archiving. Choose a chassis that is certified for the OS, RAM that is certified for the motherboard (or else Supermicro and other motherboard manufacturers won't support you), and drives that are compatible with the RAID card. Choose a RAID card that can support external JBOD storage, if future expansion is required.

Cloud-based recording/archiving and/or edge storage on devices are additional optional redundancies. 

My company has been building custom NVR and storage solutions in various form factors (cube, 2U, 3U, 4U, 8U) for all types of organizations including government agencies, museums, police/fire stations, retail, HOAs, and more since 2004. We install the OS and VMS and deliver a plug-and-play server at a lower price and better value than you can find anywhere: free shipping within the Continental USA, 3-year warranty, free lifetime technical support, prices start at under $2000, all major credit cards and PayPal accepted. Our most popular VMS configurations are Milestone XProtect and Digital Watchdog. Servers typically ship within 3-5 business days after passing a multi-point quality control process and burn-in.

Feel free to contact us if you have additional questions: 310-370-9500, x1 for Sales, or email info@CustomVideoSecurity.com. 

Depends on details, it can be cost efficient to use cloud surveillance service.

Do you look at this direction? 

"I would like to see a discussion about the *state* of storage in 2018."

I see what you did there...