Long Term IP Video Archiving Options

Hi

Back in the old CCTV days and even with DVR long term archiving was a rather simple if cumbersome affair: You took the tapes, tag them and put them in a vault. You needed something that happened two years earlier you went to the (dusty) vault, found the tape and played it back (if the tape substrate had not crumbled to dust). With a DVR you could simply back up the HDD and be done or even remove the HDD entirely, tag it and be done in many cases….

How do we (reliably) archive and most importantly retrieve our precious, information-laden IP cameras video feed? The answer seems obvious: larger storage space … The cost of keeping 200 cameras for 3 years in HD at say 10 fps is not trivial. My question. How do we archive and retrieve our IP cameras wonderful images and keep our VMS features? Time stamps , etc … Your thoughts, comments, current strategies will be much appreciated … Not that cost is not an issue, we would like to have an idea of current practices and will decide on what seem most appropriate for our needs and budget. We believe we are not alone in researching this.

FWIW , I like what Veracity is doing with the Coldstore. Price of HDD is low enough (1 TB for enterprise –grade HDD is circa $100) that it could be worth being able to just tag the HDDs, store it and be able to play it whenever … They seem to be the only one doing so but are extremely limited in the VMS they support a serious limitation.


Beyond Coldstore, digital tape is the other general category that might be worth looking into here. For example, Long Term Storage / Digital Tape (Soleratec).

Depending on your VMS (Genetec, ?) and upstream bandwidth capacity, cloud archiving may also be an option...

Cloud is not an option. VMS is OnSSI. I am surprised that more people have not faced this problem. I would like something that can be backed-up stored and be able to be viewed when needed. This may require some physical manipulation.

The problem is not trivial IMO. Let's suppose one has 250 cameras and have found a way to have online storage of 6 months... let's make things easy, let us suppose 1080p @ 10 fps. Let us suppose 250 cameras H.264 .. Let us assume 500 TB... How long will it take to back up those 500 TB on tape? What about if you have feed coming from various sites or those from mobile sites? I am yet to find a good read or solution on this . thus the reason I am submitting this to this group of heavyweights.

I think you don't find much info because from a retention time period of 2 weeks to 90 days you would likely see number of people archiving that duration of video drop off dramatically. I'd wager that less than 5% of all users keep video longer than 90 days. Those that do often have special requirements, and can justify a budget to go along with their specialty case. Given the cost of storage, then most practical option is probably just online storage to meet the requirement.

Because video files are indexed differently than simple user files like documents and spreadsheets, archiving and retrieving video elegantly starts to get very complicated.

Have you looked at the Panasonic Data Archiver? > 600TB in a single rack. Archival quality Blu-Rays 50 GB optical disks > 50 yr life. > 1600 Mbps write speed. Save everything forever. Maybe somebody can get a price from Panasonic?

300GB optical disks out in July, 1TB disks ~1yr. This model only works for 50GB discs.

Hi Trying to revive this thread. The problem is current. Several customers whom we're trying to convert to a VMS are relcutant once we can't show them a clear solution for archiving; somehting they could so easily do with thir "old" CCTV system. I was showed tapes that saved a company hundred of thousand dollars in a litigation case ...the tapes were 4 years old... the video was crappy but good enough to save them hundreds of thousands.. Talk about ROI. We are caught with fancy video but can't archive them ..

I sincerely would like to see that discussion kept alive :)

I don't understand how the VMS manufacturers have such a myopic view when it comes to Archival. Back in the old days of CCTV, archival of videos was a given. You backed up the old Videos on tape/disks or what have you for further retrieval. Simple, manual but existing. Now you may have tons of High Definition, precious and valuable evidence but archiving these is not a simple affair, worse it doesn't seem to be in most VMS manufacturer’s priority. True Milestone and OnSSI (those are the ones we know best) have some kind of "Video Aging" options in their most expensive offerings but this is not a very straightforward solution. You can’t just archive the discs and use them later really.

We are at the point of considering going for things like Hikvision or other NVRs at the remote sites. It seems easy to archive their videos, just plop an external drive to back up the HDD in the NVR (we have our misgivings about how long it would take to archive 8 TB of videos on tape ...). One simply use a similar NVR to view these videos... It is cumbersome , manual and tedious and integration with an exisiting VMS is not clear but what are the archival options when it comes to most if not all VMS?

This is a curious oversight and we are waiting for a player to grab that niche and run with it . To repeat: Veracity Coldstore is a nice solution but support for most VMS is nonexistent and their list of supported VMS has not grown since they announced the product a few years back . Whom else play in that space? Are there any solution for integrated long-term HD video surveillance arhiving?

A, this is not the answer you want to hear but this is what I believe is happening.

First, I don't think this is the manufacturer's 'fault'. Manufacturers are most certainly responding to customer demand / interest. And almost all customers have determined that it is less hassle / less expensive to extend hard drive storage than it is to go to tape.

That said, there are a few digital tape companies that have some VMS integration - Soleratec and Spectralogic are the two I have heard over the years, both niches but could be the right solution for you.

Agree with John. There are two forces at play:

1. Everybody wants more pixels and the best possible framerate. The cameras deliver, and the HDD capacity delivers. This is the most powerful force for manufacturers because it gets the most demand from customers.

2. Requests for long term archive do exist, but they're rare--especially beyond (say) 90 days. And where they do exist, this request is in conflict with force number 1, above. So it's just not going to get as much attention.

Technically, there are a few facets to the solution(s):

1. Where possible the VMS should support archival as a feature. Ideally you store the latest video at the best quality, then use a VMS feature that transcodes resolutions or framerates when transferring to long term storage. This makes the best use of the long term storage and is usually an acceptable compromise in terms of quality.

2. The long term storage must have a lot of bits. These are either very large HDD arrays or, tape, optical, or cloud. Cloud is typically not practical due to the impracticalities of large transfers, HDD arrays can get prohibitively expensive. That leaves long term tape and optical solutions, which are doable, but tend to be pretty nichey.

3. The long term storage may need to be managed as well. Even tapes need to be labeled, swapped out, retrieved, eventually retired and reused or replaced.

For a while there it looked like HDDs had eclipsed tape to the point that tape would never again serve as a cost effective way to backup HDDs. But advances in tape technology have come along to prove otherwise. Even companies like Google use tape for backups and companies like AWS provide long term off-line storage as a service.

I’d love to hear more about where you end up in terms of solutions. As Undicslosed B says, you’re going to have to start with some very clear requirements. Both in terms of functionality as well as cost/performance. I think you’ll end up at tape and need strong support for archival storage from the VMS. But it would be good to know where you end up.

Can you put some parameters around what your actual requirements are for an archival system?

Like how much do you need to archive in TB/day

And what you are will to pay in $/TB

There are solutions out there but I'm unsure whether we are talking small deployments or enterprise ones.