How To Reduce Storage For 500 3MP Cameras?

Environment:

500 each 3MP Cameras

Full Time Record due to audio recording requirements

Customer requires 90 retention

Question:

I have tried 3 different storage calculators and they vary between 967TB to 1.9PB

Not sure why such a variance but is there some way setting can be modified to gain maximum storage?

Calculator input is:

500 cameras, 3MP, 10 FPS, Motion 100% (Record Always)


Ken, good question.

First, in terms of the variance between 967TB to 1.9PB, it is because bandwidth consumption is not an exact science, it depends on a lot of site-specific / camera specific factors (see Bandwidth Guide For Video Surveillance). In your case, the calculators are estimating somewhere between 2 to 4 Mb/s per camera, which is fairly realistic for 3MP, 10fps.

Have you already chosen cameras? That can greatly impact bandwidth consumption even for the same resolution and frame rate (e.g., IP Camera Bandwidth / Storage Shootout and e.g., 'smart' codecs like Axis Zipstream Tested). Let us know what you are doing for cameras first and then we can go from there.

Thanks for getting back so quickly, greatly appreciated.

I will read the bandwidth guide you sent.

In certain areas of the campus (indoor short range, small areas i.e. stairwells) I was thinking of droping the resolution to 2MP or even 1.3 since the picture quality would be acceptable.

The camera line is D-Link commercial line not consumer.

I will read what you sent and see what I can do from there.

The system is on its own network. Cat6e, 1GB switches. 10GB between server and storage, fiber between building, if that makes any difference?

Ken,

I would definitely test those D-Link cameras. Even if it just 1 of each model. Aim them in the places where they will be, whether it is hallways, lobbies, parking lots, etc. Record the bandwidth.

Important, do the same at night, since night time bandwidth generally spikes (see Testing Bandwidth vs Low Light).

That will give you an accurate projection of the estimated bandwidth usage.

With the cameras already picked, there is not a lot that can be done to reduce bandwidth. You can adjust the compression level to reduce bandwidth / storage but be careful that it does not reduce quality. If those models supported privacy masks, you might try using them to reduce bandwidth / storage in unimportant areas of the scene (see Reducing Bandwidth Through Privacy Masks?)

A possible out is to compare 90 days retention with 90 days accessible. You could archive on to a cheaper medium and only keep 15-30 days online. The question is "How often and how fast do I need to review 1-3 month old recordings?" Something like the Veracity Coldstore could be used by just cycling out the disks before overwriting or a streaming tape drive(both require regular manual intervention) . Also, what are the requirements for not losing data - do you need a RAID setup to protect the storage, or can they accept occassional loss of data?

Mark, good suggestions.

Ken, what VMS are you using?

You could consider Coldstore - see: Veracity Coldstore Overview. In addition, there are some tape systems like this that work with video surveillance.

I am asking about VMS choice because these storage options do not integrate with most VMSes.

You're correct in that it doesn't work with a normal RAID NVR, that is what is appealing about Coldstore - records linearly on several disks simultaneously - RAID 1 - so only s few disks are active at any time and there is a) increased disk reliability, b) lower power, and c) any time interval is removable on one disk per server (more if it overlaps or the duration exceeds the time period of the disk). For a SAN/NAS solution, external archiving is about the only choice.

I am not disputing Coldstore one way or another. My point is that if he has already locked in a VMS, he needs to first determine whether it even integrates with Coldstore (similarly to how he is locked in to D-Link cameras).

We also had a customer inquire about something similar in the past and wanted to know if there is some way to do it cheaply.

What may be overlooked is that, 500 cameras is not all going to be recorded by 1 NVR. Without going into the more exotic (expensive) models, a 64 channels NVR is the typical everyday "large" NVR. So we are talking roughly 10 NVRs without pushing them to their max limits for a more robust system.

Now these large NVRs typically already come with 8 to 16 drive bays. Throw in the typical 4TB drives and we get 320 to 640TB of storage. A decent amount of online storage already.

For additional room, we then looked for SANs as backup storage and found them to be exorbitant priced, like in the hundreds of thousands. Much more expensive than the NVRs. So one of the straight forward option was to just add more NVRs with the benefit of having all video online for immediate retrieval.

Researching some more and we found this site: http://www.45drives.com/

We do not know it's performance metrics nor ever tested one so we asked the customer to do their own due diligence.

We proposed both solutions to the customer and I don't know what the outcome of the project was.

I noted that COLDSTORE was mentioned in this discussion (disclosure: I represent Veracity, the manufacturer of COLDSTORE).

John is correct - COLDSTORE is an option for this kind of application. It is ideal for video retention times of 30days or more, and the longer the retention, the more suited it becomes. Before John (rightly) admonishes me for making this a sales pitch, I did have a technical point (well, two actually) :

COLDSTORE can exploit the latest 8TB SMR disk drives, which are incredibly low cost per TB and means that the end user can get very high capacity cost-effectively. "SMR" stands for shingled magnetic recording and means that the writing tracks are partially overlapped to get higher density on the disk. This is ideal for archive purposes where you are not going to change the data. It is totally unsuited for normal database or IT style use and cannot be used in a RAID5 or 6 system. However, the SFS or sequential filing system within COLDSTORE writes the whole disk sequentially (and optimally) and then makes no further changes to it until the retention time is reached and the disk is overwritten with new data. This means that these very high capacity SMR drives are perfect for COLDSTORE and thereby for long-retention video surveillance applications.

COLDSTORE is regulary used in applications requiring 90, 120 days file retention, and we have done a number of projects with 6 months and even 2 years' file retention. These are typically multi-petabyte projects.

I would also agree with John that the key to reducing the storage requirement for Ken Jenkins' project is to carefully choose the right IP camera. Choosing one with even slightly better compression efficiency makes a big difference when the retention times are long. However, also look at the storage solution - some solutions have a far lower total cost of ownership than others, so take into account capital cost, disk costs, power consumption over the expected lifetime, cooling equipment costs, cooling equipment running costs and last but not least, UPS equipment costs. The more power consumed by the storage system the more expensive all of these will be.

Sorry, did I mention that COLDSTORE uses 60W for 120TB of storage ? Half a watt per terabyte is good news for the TCO calculations mentioned above. (That was my second technical point).

I hope this is helpful.

Alastair, while we have you, what VMSes now support Coldstore? An updated list posted here would be helpful.

This is ideal for archive purposes where you are not going to change the data.... It is totally unsuited for normal database or IT style use and cannot be used in a RAID5 or 6 system. However, the SFS or sequential filing system within COLDSTORE writes the whole disk sequentially (and optimally) and then makes no further changes to it until the retention time is reached and the disk is overwritten with new data...

Alastair, question. Assuming the retention period is 90 days, on day 91 and after, aren't you overwriting a full days operational data onto an old archive every day?

(Apologies for the delay in answering this valid question).

Yes indeed, once the array is full, we are overwriting data on the disk constantly. The sequential writing system allows the efficient (i.e. fast) overwriting of data on the SMR drives as it doesn't matter that we are deleting data a few tracks (cylinders) ahead of the writing point, due to the overlapped tracks and the wider "write" track width compared to the narrower "read" width which is what is left after the writing tracks are overlapped. The capacity which is lost due to this effect is negligible and in fact is made up many times over by the fact that the SFS does not require a directory area, as every large data block has it's own tiny header. Typically a directory filing system will use about 5% of the space on a disk, whereas the SFS uses less than 0.2% for the file headers. Everything is written sequentially, yet can still be read via random access.

If the SMR parts above doesn't make any sense, then please look up SMR drives on the web. We are preparing a white paper about the use of SMR drives for video surveillance, but I can't give you a release date for that yet. Please also note that the SFS works with any SATA drive and does not depend on SMR - it is simply ideally suited to exploiting the very high capacity of SMR type drives.

The VMS is Video Insight, thanks.

Ken : Video Insight is one of the VMS products that we support. In fact it is one of the earliest ones we did and so we have considerable experience of it.

John, thank you for the opportunity to update your readers on our COLDSTORE compatibilty list. I am happy to do so here. The following list of supported VMS systems is not exhaustive, but covers the more common systems :

Genetec

Milestone Corporate

Avigilon

Exacq

Video Insight

Salient Systems

Instek Digital

VMS UK

Verint

AD VideoEdge

In addition to these, at ASIS we will be announcing completion of integration with :

DVTel

Network Optix

and possibly two or three other major manufacturers which are currently covered under NDA.

Ken, Just test the camera for one or two days, disregard the calculators. Do you have any constrain in the bandwidth? VMSs in general use some % for the FiFo policy execution. In other VMSs you can set up the size of that space. I'm agree with undisclosed 1, I don't see the benefits of COLDSTORE (or even why we have to write they brand in capitals :))
I think one of the good points stated above and maybe only lightly touched is establishing resolution requirements before choosing the cameras. If 40 pixels per foot is required, why would one go to a multi megapixel camera when something far less would meet the goal. Over specifying unnecessarily taxes bandwidth and storage.
I think one of the good points stated above and maybe only lightly touched is establishing resolution requirements before choosing the cameras. If 40 pixels per foot is required, why would one go to a multi megapixel camera when something far less would meet the goal. Over specifying unnecessarily taxes bandwidth and storage.

That's a good point. If they are locked in to those cameras already, they could reduce resolution from 3MP to 1.3MP. Even better, they can reduce more bandwidth / storage, by keeping it at 3MP and increasing the resolution level. See: Resolution vs Compression Tested

Also, most calculators don't take audio into account, which considering all the cameras are recording sound, could be another couple percent storage needed.

I believe that the storage should be relatively slow in comparison with the amount of information. Any "mid-end" solution should accomplish your requirements. I think you should analyze the iops, bandwidth, etc. To pick the right hardware solution.

The options based on quality, according to the Dlink calculator, are 972 TB, 1633 TB and 2488 TB.

Those are reasonable, though fairly simplistic, estimates.