Actual ROI Of Deployed Surveillance Systems

Regularly, people talk about the projected ROI of a new / planned system (see our guide) but rarely do you see real studies of the actual ROI from a deployed production system.

In a recent discussion, a consultant asked us to cover this. Here's what he said:

"I would love to see some impartial studies based on scientific data that show the effectiveness (or non-effectiveness) of video surveillance systems in various types of environments. There are lots of commonly-held beliefs and folklore about the deterrent value of video surveillance, but little actual factual data to back it up."

Expanding in a follow up with some specific points:

"Things that I think would be desirable in any type of study include:

  • Accurate data gathering system in place, such as security incident reports or police crime reports.
  • Metrics are quantifiable rather than subjective. ("25 losses per month before we installed the system, 5 losses per month after we installed the system..." vs "gee, things seem a lot better..."
  • Relatively high-frequency events are used as data points (10 to 15 car prowls per month occurring in parking lots vs 1 rape that occurred in parking lot since it was built 15 years ago.)
  • Solution being analyzed can be isolated (if cameras, better lighting, and increased security patrols were all added at the same time, who can say which measure is responsible for reduction in crime?)
  • Data and conclusions are subject to peer review.
  • Person(s) conducting study have no stake in its outcome.

With regards to the last point, I have found that end-users who have spent a boatload of their company's money on a security solution are sometimes not the best source of objective information. If the system is a failure, they are often reluctant to admit it as they feel that they are at least partially responsible for the outcome. A few brave souls will step up to the plate and tell you the truth, but in many cases, the end-user tries to paint a unrealistically flattering picture to hide his system's shortcomings."

I think IPVM needs to go out, find end users and do this from scratch. Short of that, I am not optimistic about finding real analysis. Thoughts?


John,

I think that the biggest problem with calculating ROI is trying to determine how many incidents a system prevented just by being there. That is something I have wrestled with ever since I've been in the business. This is especially true in the casino vertical.

Take the case of employee theft. You can generalize that if there were no cameras or if they were ineffective and some employees got away with stealing, that behavior would likely increase as employees became emboldened by their or others' actions. In that vein, if dishonest employees are caught and fired, or better yet prosecuted, that would have a deterrent effect on others who might be considering such actions.

But how do you quantify that?

Deterrence is clearly tough to measure in any security system, by definition, because you must estimate how many things did not occur.

However, there are two ways it can be done fairly reasonably:

  • Going from no system to a system, one can presume that overall decreases in incidents are impacted by the system though, of course, this is uncommon today as most already have systems and other simultaneous factors (like economic changes or increases in guards) could factor
  • Upgrading a system, which is most typical today.

Upgrading

Let's see you measure a baseline of incidents occurring, incidents solved, operational expenses, etc. Then you upgrade cameras from SD to 5MP. You could then measure how those parameters changed afterward, assign a dollar value to them and compare to the upgrade cost. What do you think about that?

Of course, you have to factor in things like, HOW the client sees the system helping them. The most common assumption is that surveillance will be used to prosecute or hopefully prevent crimes like thefts, assaults, vandalism, etc. In my experience with restaurant clients in particular, systems are often just as useful for "ass-covering" - for example, customer slips on some stairs, claims the stairs were wet or had had something spilled on them... video is retrieved indicating the customer tripped over his own shoelaces... and the client avoids a million-dollar insurance claim or lawsuit.

I think it would hard to quantify that as a "million-dollar savings" as the burden of proof is on the complainant and it may have been tossed out of court even without the video evidence. I suppose one could calculate what was potentially saved in lawyer's fees for the defense.

In cases like these though, deterrence isn't the intent of the system, nor is "recovery" per se.

Matt, you can use the count and cost of historical incidents (e.g. slip and fall) compared to what occurred after the system was deployed.

Also, for restaurants who use the system as a remote management tool, you can factor in the value of time savings for managers not to have to physically visit the site, etc.

Matt, you can use the count and cost of historical incidents (e.g. slip and fall) compared to what occurred after the system was deployed.

There is that. Still, it gets trickier as it's not a fixed cost; eg. someone shoplifts a rack full of beef jerky (it happens, my co-worker watched it happen on a site!) is easy to work into ROI calculations, vs. a POSSIBLE judgment award or POSSIBLE settlement, either of which MAY be for anything from $1 to millions of dollars.

Of course, if you assume the average settlement/judgment is going to be six figures or more, it's pretty easy to justify just about any expendature :)

Also, for restaurants who use the system as a remote management tool, you can factor in the value of time savings for managers not to have to physically visit the site, etc.

I had a store manager contact me once because he couldn't access the cameras in his store remotely, and asked me to look into it, then noted that I should email him, not phone, because he was on vacation in Hawaii. I fixed the problem, then emailed him back and told him, "TURN OFF THE COMPUTER, GO OUT TO THE BEACH, YOU'RE IN HAWAII, DUDE!"

I suppose the store owner would consider this a benefit, as he's getting the manager's time for free :o) +1 for ROI!

Possible, but at least in our case, I think the improvements offered by HD/MP cameras will be much more subtle and harder to quantify, except for certain limited applications. For instance, our current hybrid system provides excellent "live" picture quality via our analog matrix but reduced resolution and increased noise through the encoders/NVRs. We've designed our camera layouts with analog in mind and deployed our cameras to provide sufficient clarity (love that term) to provide suitable evidence.

The one place we know will benefit from HD is table games, where our existing analog cameras allow us to identify the suits of cards "live" through our matrix but the 4SIF/D1 limitation of encoders prevents after-the-fact card suit identification. Since a replacement system will certainly eliminate the analog matrix to simplify monitor wall design, we would also lose the ability to identify card suits during "live" viewing.

The same may be true of other applications but my feeling is that we would be deploying the majority of HD/IP cameras for the same reason: the conversion to all-digital monitoring will reduce our ability to see what we were able to see on our current system; at least when viewing cameras "live".

I don't forsee that many cases where HD will be enough of an improvement over our analog system to allow us to see things we can't currently see unless we use the overused "one MP camera can replace x number of analog cameras" scenario. That factor interferes with our ability to determine the ROI of the upgrade.

Carl, I think you misunderstood me. My point wasn't that HD will deliver ROI than SD but that you should be able to measure it by comparing the upgrade cost to the number of incidents that you were able to solve with the HD cameras that you would not have with SD, etc.

In other words, I am talking about the ROI measurement process. Maybe for you it's negative, but there should be a reasonable process to estimate the ROI of an upgrade - positive or negative.

John, Not sure I would agree. My point is that it is far more difficult to judge any improvements when the original analog system was optimized for the cameras' capabilities in the first place. How much better is sufficient than sufficient?

Personally, I think there are far too many variables that come into play to accurately (or scientifically) judge any 'results'. Including, but not limited to, the skew of the tester, the many assumptions that have to be made without a 'base line' point of reference [beyond the measure before, measure after analysis] for all of the other variables, the 'other uses' ROI probabilities, etc.

It's a statistical challenge that can, at most, be 'guestimated'.

A marketers dream.

Dear Mr. Pessimistic, there's value in, at least, trying to better understand and diagnose the success of a system, even if it's not perfect.

A point I completely agree with. :)

I'm merely trying to answer the posted question (Actual ROI).

If nobody has come up with an actual equation that can be used to determine actual ROI by now, I think that alone says enough....

ROI projections are more art then science. Neither good nor bad, as long as you accept that thesis and don't take it literally (imho). :)

P.S. I posted using IE just for you... ; ]

If the existing system is sufficient and the upgrade provides no additional value (i.e., cases solved, time saved, etc.), than you can conduct the ROI. Investment = X, Return = 0, etc.

In this scenario, the ROI is not undefined, it's simply zero.

That would be true if the only criteria used is cases solved but there are a number of other criteria that can be added to the equation: system usability, added functions and features, warranties, and in our case, footprint, power and cooling cost savings over the long term.

Ok, than add them in. If an upgrade reduces power and cooling costs, than add them in as operational savings. If the new functions and features reduce time spent by operators, estimate how much time is saved over X number of years and add it in.

The return is simply all the savings the security system delivers - whether it's reduced theft/loss/incidents or lower operational costs or time saved, etc. Sum them up and compare to the cost of the ugprade.

That's exactly what we did, along with the consideration that most of our hardware is reaching end-of-life and would have to be replaced anyway.

Great, then you should be able to estimate ROI reasonably, one way or the other.

Ok, but those are evaluation points which were not mentioned in the original post.

"either of which MAY be for anything from $1 to millions of dollars."

Yes, but that's like saying a person may die at 11 or 110 years old. You don't need to simply give up. There are historical patterns to work with that allow a business person to make reasonable estimates of average loss / gain, etc.

"I suppose the store owner would consider this a benefit, as he's getting the manager's time for free"

Certainly a tangible benefit for the owner / buyer here, one that you could estimate in additional time worked or incidents solved, etc. But look at it the other way, having that system may have enabled him to go on vacation without feeling scared or concerned that he would be totally out of the loop. Either way on the manager portion, this provides value to the buyer/owner in increased work / productivity, which can be estimated in an ROI calculation.

We have one datapoint that provides an interesting before/after dataset.

One November, we purchased a nice little shop we had enjoyed frequenting for many years. It seemed a well functioning ongoing business that had suddenly come on the market. It seems it had changed hands and then spent the next three years going bankrupt. We kept all elements in place including employees, space, etc.

The security installer was unable to support a video monitoring installation until late January or early February.

I just reviewed records for a quick comparison of revenue before and after video installation.

We had no video in December (historically by far our best month of the year), but we had video installed well before March (one of the worst months of the year).

The first year, we made the same revenue in December and in March. There was no VMS in December and there was a VMS in March.

In all subsequent years, we consistently made 2 1/2 times the income in December as we did in March.

While those results are subject to interpretation, in our estimation, while other sources provided various evidence of employee dishonesty, the video largely curbed it.

In retail, though, the accepted technique in measuring loss / ROI is shrinkage. That looks specifically at what was lost between products received and products sold (i.e., things that were stolen, disappeared, etc.).

Looking at total income tends to be too broad because there are so many other factors that drive that. Maybe you are just an excellent sales person or business manager :)