How to Handle Hard Drive Failures

Author: Ethan Ace, Published on Feb 13, 2012

No matter how well maintained a surveillance system is, hard drives eventually fail. Indeed, A recent member's poll showed that most find hard drive failures to be 'a common and significant problem'. What happens when a hard drive fails can vary significantly depending on the system type used, the priority of the system and the concerns of the end user. In this note, we examine the most common real world issues and provide guidance on how to best deal with hard drive failures.

Impact of Hard Dive Failure

How much of an impact a failed hardrive has on a system depends on the architecture of its storage. There are three typical scenarios:

  • Single drive for OS and archived video: In systems which use the same drive for the server or DVR/NVR OS, as well as video storage, failures are most critical. Loss of the drive results in the entire system going down, with users unable to view live or archived video. The unit is essentially dead until it can be replaced in its entirety, or a drive with the OS reloaded.
  • OS and archives on separate drives: In many DVRs, the operating system is kept separate from video storage, either in flash or on a seperate hard drive. This prevents video from being lost should the OS drive fail. Conversely, it prevents complete failure of the unit if the video drive fails. The system may still be used for live viewing until the drive is replaced and archiving may begin again.
  • RAID: When using RAID for storage, whether it be internal or external, the system, including archiving may continue to run when a drive fails. Depending on the RAID level, one or more drives fail without issue. When a new drive is inserted, the RAID array begins rebuilding the lost data. During this process, performance (mainly throughput handling) may be reduced. Some high-end enterprise systems allow for full performance during rebuilds, though lower-end units do not.

One key consideration is how and when users will know if archiving has stopped. For small systems especially, the recorder is often located in an unstaffed location, such as wiring closet, without any check of whether it is working or not for extended periods of time. Thus, often, the first time a user realizes the recorder is not working is after an incident has occurred. Some DVRs and VMS platforms are able to send notifications if archiving stops for any reason, but we suspect these are often not set up.

Client Response to HDD Failure

Depending on the client and how they view the criticality of their surveillance system, response to a HDD failure will be different. For some, going without their system while it is repaired is acceptable, as they rarely have incidents which need review. For others, downtime is absolutely critical, and immediate response and repair of the hard drive is required. 

Integrators may keep spares on hand for end users with stringent downtime requirements, as part of their service contract. Other users may simply receive a loaner unit, to provide basic recording functionality while the production unit is sent out for repair.

Written Procedures

No matter which steps are taken, due to the potentially negative reaction some users may have when their system is down, it is essential that response procedures are agreed upon, and put in writing as part of the warranty or service agreement. While users will still often direct their frustration at the integrator, this reduces the potential for contention or blame.

Recovery

Get Video Surveillance News In Your Inbox
Get Video Surveillance News In Your Inbox

While it is possible for data recovery services to read some, or even most, data from a failed hard drive, this is often a costly prospect. On average, recovery of a single hard drive can cost well over $1,000. Complex physical problems (caused by severe crashes) may cost upwards of $2,000 as more manual work is required. This makes data recovery an expensive process, reserved only for severe cases.

Potential Liability

No matter who the client is, the potential for liability issues is present when a surveillance system is in place, but not recording. Incidents, no matter how infrequent, may occur at any time. Video of a critical incident may be key evidence in litigation (such as accidents, vandalism, slip and fall, etc.), making hard drive failure a real risk to users. 

For users whose video surveillance is governed by regulations, hard drive failure is potentially more of a risk. For those in government or gaming verticals, who are required to store video for certain durations, an unnoticed hard drive failure could place them in violation of these regulations. For this reason, these users nearly always use RAID storage, sometimes RAID 6 and beyond, to guard against multiple hard drive failures. 

Recommendations

For users of small surveillance systems, the best choice is to use DVRs with separate drives for OS and video. For redundancy, inexpensive NAS units, which offer RAID 1 (disk mirroring) are available for under $300 USD. Not all DVRs are capable of using NAS storage, however, so users should ensure compatibility before purchasing.

For users of enterprise-level surveillance systems, RAID is very common. The main concern is the RAID array's performance during a rebuild. If a drive fails, no data is lost. However, low performance arrays can practically take down a sysem if read/write capability dips too low during a rebuild, which may take 6-10 hours or more. Users should beware of this, and verify with vendors that performance will remain high during rebuilds.

For all users, if video is critical, selecting a recording platform which will notify them of hard drive failure is always recommended. Immediate notification is the best way to protect against lost video.

1 report cite this report:

Top Video Surveillance Service Call Problems (Statistics) on Feb 28, 2018
In our most recent statistics series, over 150 integrators told IPVM the most common problem their customers have with their video systems....
Comments : PRO Members only. Login. or Join.

Related Reports

IP Camera Cable Labeling Guide on Sep 14, 2018
Labeling cables can save a lot of money and headaches. While it is easy to overlook, taking time to label runs during installation significantly...
Dell Launches IoT for Surveillance on Sep 05, 2018
Historically, Dell has been a PC and server provider (e.g., "Dude, you're getting a Dell") and widely used for surveillance storage. However, in...
Drain Wire For Access Control Reader Tutorial on Sep 04, 2018
An easy-to-miss cabling specification plays a key role in access control, yet it is commonly ignored. The drain wire offers protection for readers...
Why Vivint / Best Buy Failed on Aug 31, 2018
DIY has bested Vivint. In 2017, Best Buy and Vivint partnered with Vivint employees on the floor of 400+ Best Buy stores, helping customers with...
Exit Devices For Access Control Tutorial on Aug 28, 2018
Exit Devices, also called 'Panic Bars' or 'Crash Bars' are required by safety codes the world over, and become integral parts of electronic access...
Inputs/Outputs For Video Surveillance Guide on Aug 24, 2018
While many cameras have Input/Output (I/O) ports, few are actually used and most designers do not even consider them. However, a good understanding...
Backup Power for Large Security Systems Tutorial on Aug 24, 2018
Choosing the right backup power system depends on system size. While small and medium systems greatly benefit from using UPS battery backup...
Synology Surveillance Station VMS Tested on Aug 22, 2018
With so many low-cost NVRs and enterprise VMSes, is there any place in the market for NAS-based VMSes? Recently, IPVM bought a Synology NAS for...
2Gig Gun Lock / Motion Detector Tested on Aug 17, 2018
Safer guns for families and an opportunity for security dealers to sell more services? That is the aim of Nortek's 2GIG 'Gun Motion Detector'...
Backboxes for Video Surveillance Tutorial on Aug 15, 2018
Backboxes are a necessity in surveillance, whether for managing cable whips, recessing cameras, adding wireless radios. But it can be confusing to...

Most Recent Industry Reports

25% China Tariffs Finalized For 2019, 10% Start Now, Includes Select Video Surveillance on Sep 18, 2018
A surprise move: In July, when the most recent tariff round was first announced, the tariffs were only scheduled for 10%. However, now, the US...
Central Stations Face Off Against NFPA On Fire Monitoring on Sep 18, 2018
Central stations are facing off against the NFPA over what they call anti-competitive language in NFPA 72, the standard that covers fire alarms....
Hikvision USA Starts Layoffs on Sep 18, 2018
Hikvision USA has started layoffs, just weeks after the US government ban was passed into law. Inside this note, we examine: The important...
Chinese Government Praises Hikvision Following Xi Jinping on Sep 17, 2018
The Chinese government council responsible for managing China's state-owned companies praised Hikvision’s obedience to China’s authoritarian leader...
Amazon Ring Spotlight Cam Tested on Sep 17, 2018
Amazon's Ring has released their latest camera entry, the Spotlight Cam, which we bought and tested in our Consumer IP Camera Analytics...
European Mega Security Firm Verisure Pushing Security Fog on Sep 17, 2018
The European mega security firm Verisure (Securitas Direct), with a reported 2 million customers, is pushing security fog, as shown in this BBC...
IP Camera Cable Labeling Guide on Sep 14, 2018
Labeling cables can save a lot of money and headaches. While it is easy to overlook, taking time to label runs during installation significantly...
Favorite Intercom Manufacturers 2018 on Sep 14, 2018
Intercoms are certainly increasing in popularity, driven by the integration of video and IP networking. But who is the favorite? On the one side,...
Vivotek 4MP Camera Tested (FD8379-HV) on Sep 13, 2018
Next in our series of updated 4MP testing, we bought and tested Vivotek's FD8379-HV, and entry level 4MP model claiming "top-notch quality video in...
Ascent / MONI Faces Lender Lawsuit and Debt Crisis on Sep 13, 2018
ASCMA, aka Ascent, aka Brinks Home Security, aka MONI, aka Monitronics is being sued by a group of their lenders who allege: As of June 30,...

The world's leading video surveillance information source, IPVM provides the best reporting, testing and training for 10,000+ members globally. Dedicated to independent and objective information, we uniquely refuse any and all advertisements, sponsorship and consulting from manufacturers.

About | FAQ | Contact