Backing Up 550TB Of Milestone Storage

Does anyone have recommendations for hardware vendors backing up 550 TB of Milestone XP video footage and have the software to automatically prune the database on a rolling 2-year age? The software should base the pruning of the date the video file was created, not the date of file getting copied to backup (for obvious reasons). Thanks.


you can take a look at Spectra Logic. They evidently have a certified solution for Tape and NAS storage with Milestone. I saw them while at the Milestone show a couple of weeks ago. Never used them so you would have to get more info from them but at least they went thru a cert process (got tested) by Milestone.

Yes, I've used Spectralogic for normal disk backup in other lives. I have a pre-sales engineer meeting with me next week to see what they can do. I'm hoping there are others out there as well that support Milestone.

Stephen,

Here are Milestone's list of high level potential options. They emphasized speaking with a Milestone reseller. You also may want to reach out to a local Milestone rep or SE and see if they can give more personalized recommendations:

  • Archive daily and use regular tape backups, then manually reuse the tapes after 2 years. (cheap but probably not an option)
  • Archive daily and use an automated tape backup system and have it delete backups after 2 years
  • Archive daily and use a hybrid tape backup system. As the backups actually appear to be online the VMS can delete the archives after 2 years
  • Use Coldstore boxes from Veracity and set the system to archive to the Coldstore boxes, and let the VMS delete recordings after 2 years.
  • Purchase a SAN/DAS/NAS with 550 TB and archive to it, let the VMS delete recordings after 2 years.
  • Use one of our partner made storage integrations – again set VMS to archive to it and delete after 2 years.

A few other comments from me:

Spectralogix is one option, I believe Soleratec Long Term Storage / Digital Tape is another. We have seen a number of people use Coldstore for long term low cost storage as well.

I was thinking about your option 2 and 3 - Archive daily and use an automated tape backup system and have it delete backups after 2 years, or Archive daily and use a hybrid tape backup system. As the backups actually appear to be online the VMS can delete the archives after 2 years. Not sure what you mean by hybrid tape backup system. My current setup has a master and two slave servers archiving to a QNAP NAS. I was thinking of procuring a solution that will allow me to backup the archived data from the QNAP and then have VMS delete the aged video (over 2 years) off the QNAP and tape solution. I'm crossing my finger that whatever solution provider Milestone approves will do the data pruning automatically.

I represent Veracity and we are the manufacturer of COLDSTORE video surveillance storage systems.

COLDSTORE is a sequential storage system and was designed specifically for video surveillance applications, being quite different from traditional RAID systems. COLDSTORE was designed to make hard disks much more reliable (even the very low-cost ones) by removing or mitigating the three key factors which cause hard disks to fail -temperature, vibration and wear.

Due to this and other factors including extremely low power consumption and the ability to use very high capacity archive-style SMR hard disk drives, COLDSTORE is an ideal solution for long-term file retention, and indeed we do have customers who keep video data for two years.

The advantages of the COLDSTORE option over the other approaches would include:

Simplicity: The Milestone system would archive to the COLDSTORE as frequently or infrequently as required (daily or hourly or somewhere in between - depending on the numbers of cameras and size of the files) and it can be set to automatically delete the recordings from COLDSTORE after the defined archive period (in this case two years). The NVR(s) can archive to multiple COLDSTOREs and so the set up is very easy to arrange.

Low Total Cost Of Ownership: The very low power consumption, ability to use low-cost hard disks (and supply them yourself if you wish), very limited cooling required, low maintenance, very long system and hard disk lifetime (expected > 7 years) and low cost of UPS required results in a very low total cost of ownership over the expected useful lifetime of the solution.

Reliability: COLDSTORE uses a mirrored-overlapping pair writing pattern, so that all data are fully mirrored during the critical writing process, providing recording security.

Each COLDSTORE can currently hold 120TB of raw disk capacity (112TB effective) and so five COLDSTORE units would meet your requirement. Please contact us via our website if you would like further information.

Alastair, you claim "Low Total Cost Of Ownership" and "five COLDSTORE units would meet your requirement."

So what is the pricing of that? This way we can better judge / contrast the low cost, especially compared to digital tape which also claims to be 'low cost'.

Stephen,

Which version of Milestone are you using?

I ask because the Archiving process built in to the code can do what you want. The Expert/Corporate code level has more multi level archiving options that you can take advantage of vs the basic 'two level' one in the lower code levels.

In general...the Live Database target is where all the new video goes and it has a 'retention time' setting that will flag it as 'ready to archive'. The archive process is 'normally' supposed to set up run every 4 hours (see your last quote from MS)...but this can be as frequently as every hour depending on what your camera load looks like.

Note that the default Archive schedule is once per day and if you have the LiveDB/ArchiveDB storage set up...you MUST change this schedule to match your camera rate. Failure to do this simple step is the cause of many issues.

The Archive Target has the longer time limit associated with it and the Corp code level allows for FPS pruning as each new Archive level is processed.

These concepts are covered in the MS classes and are very important from a performance perspective.