I Have A Banking Client Who Wants To Centrally Store Video From 20 Branches Using Milestone

The client needs to upgrade from current analog camera to IP and here are the points.

a. centralized storage

b. the setup could be edge storage, edge recording server, centralized server and centralized storage which is the recommended setup by Milestone? But alternatively can Hikvision achieve the same scenario perfectly? What the user wants is just to actively backup the video file from branches to HQ centralized storage daily and meanwhile they agree to upgrade the brandwidth for each branches at least to 15-20Mbps. And at each branch the HD recording is required

c. regarding the strorage to Milestone for a 700-800 cameras scale which one is better Netapp or EMC Isilon? Can help to advise the prons and cons? the estimated storage size is about 800TB in total.

d. to be frank from an system integrators perspective I'll prefer turn key solution like Hikvision, Panasonic etc.. but i'm not sure if any of the brand can fulfill their requirement?


Michael, first to clarify, do they want to (1) record it first locally at each branch (with a recorder) and then back it up to their HQ or (2) send the video straight from their cameras directly to the HQ?

This choice has a big impact on reliability, cost, etc. From your description, I am not positive which of these 2 approaches your client wants to take.

Hello John, they don't specify on that however the idea threw by Milestone is to have

a. edge storage/SD card

b. local recording server/ 7 days retention

c. HQ recording server/210 days retension

d. HQ centralized storage for backup purpose

actually what customer want is just option d from above list. so it could be either way however the option (2) you mentioned may have certain risks like data loss and because of the nature of their business(banking) I think they will either go with

a. centralized storage with local cache

b. centralized storage with local storage(longer retention)

thanks in advance for the help!

The approach Milestone is recommending is going to be fantastically expensive, given all the parts and the expense of each part.

Have they considered HD analog (CVI, TVI, AHD, etc.) for the cameras? That would give you 1080p plus lower cost and simpler install. You could then connect the HD analog encoders or DVR to a VMS system like Milestone or Genetec if you preferred.

You mention 210 days storage? Is that a requirement? Motion based recording? Just events? Continuous? With 700-800 cameras, you might be looking at PBs of storage.

What about ATM and teller transaction integration?

Since they plan to phase out all the analog devices I think they will only consider IP based solution this time.

1 year retention for HQ stroage is required while 210 days here is just sort of cache memory on central milestone server collecting data from each remote nodes before sync to HQ central storage(external storage like EMC/Netapp). Basically they need all the records to be kept for at least 365 days which is 1 year. The whole project is going to be carried out by phases and during the very first two phases only 490 cameras will be deployed which we need to plan out for now. Probably let's just take it as --

490 cameras in HD while 290 of them are remotely located over WAN and the entire recording archive retention is 1 year. If they are to use Milestone solution here's what I can think of

a. Camera - AXIS vs Samsung (which one has the better compatibility and interoperability?)

b. Storage - Netapp E series vs EMC Isilon (which one is considered better?)

c. Milestone recording servers which could be any brand..

b. bandwidth upgrade is not a concern however since it's costly camera like AXIS with Zipstream technology or Hikvision using Smart H264 or H265 would be preferred..

Besides, I'm also thinking to introduce Hikvision IP solution to them however I heard from someone that Hikvision VMS solution doesn't support centralized solution quite well either in architecture of

a. remote cache --- central storage from Hikvision

b. remote NVR --- /central storage from Hikvision

Aprreciate your advice on above..

They prefer to have the pinhole camera. But I'm not too sure if there's any integration can be done with transaction.. sorry I'm quite new to this..

"But I'm not too sure if there's any integration can be done with transaction.. sorry I'm quite new to this.."

This is important. If they want to search for transactions and pull up related video (i.e., we had a disupte from a customer 6 weeks ago, find the video to see who is right), you will need to be able to support this.

Get your client to verify this asap so you can factor support for it (not every does) and the cost, if needed.

Hello John, regarding the transaction integration I've personally checked with Hikvision and Milestone and the feedbacks show that Milestone supports searching of the footage if its recording file is sync back from remote branch and looked up from the central storage while the Hikvision loses its footage info which means user has to do the review from the begining in the same scenario. If I understand correctly user wants it to be done.

Appreciate your advice. Thanks a lot!

If they want to search for transactions and pull up related video...

Is this type of functionality covered by standard POS integration, or is there more to it than that?

Michael, what country is the customer located in?

15-20Mbps gives you the ability to transfer roughly 150-200GB per 24 hours, assuming the bandwidth is exclusively available for backup/data transfer and is not being shared by the branch for other functions.

A camera with a 3Mbps average bitrate (low, but acheivable. possibly.) will accumulate approximately 30GB of data per 24 hours. If you're doing continuous recording and want to backup video offsite your 15-20Mbps of bandwidth would support about 6 cameras, and if the network is ever down for more than a few hours you'll fall behind and will need to skip some of the video for backup.

You're specifying 20 branches and 700-800 cameras, which comes out to 35-40 cameras per branch. Even with motion-only recording on a majority of those cameras I don't think the math works out to have video backed up to a central location.

For systems like this, from an overall design approach the most practical solution is just to record to 2 servers simultaneously. This means you need the bandwidth to support the aggregate of all your camera streams, which you'll find comes out to the same rough estimate as what you'd need to maintain a 24 hour backup cycle.

If the 35-40 cameras per site estimate is correct, then you'd need somewhere around 100-150Mbps of dedicated connectivity for this to work.

Another approach is to record locally and use a platform that supports some kind of data aging, transcoding, etc. to reduce the size of the stored video. Then you could backup the "pruned" video. In this case, you're most likely looking at something where the onsite video is at full-resolution for 30 days, then pruned, then backed up remotely. This would give you some amount of long-term retention ability at the central site, but would not give you the redundancy of the full-resolution video.

Hi Brian,

Exactly. They have about 20 cameras for each branch and the rest 300 cameras mentioned above is for HQ buildings. So if let's consider 20 cameras per branch with continuous recording or motion based recording, it could easily take up 20Mbps from the dedicated WAN circuite subscribed from telco providing the camera supports the stream compression technology rather than H264. Meanwhile a local server with 7 days retension storage will be catered for first level recording backup and this recording server will perform the sync back to central recording server and then cetnral storage to keep for a retension of 365 days. If that's the ideal scenario I'll then propose with such product supports

a. edge storage with SD card which sync back speed is accpetable after the failover scenario happens

b. A solution that supports local server recording with prunning technology before transmitting back to central recording server

c. VMS of which the content of footage info won't be lost during the sync back mainly used when dispute happens

d. Camera supports stream compression technology

e. If the bandwidth of 20Mbps is locked down only certain camera like ATM pin-hole and camera on top of the counter supports 1080P continuous recording and the rest only supports motion based 720p just to meet the bandwidth limitation

f. If VMS like Milestone is considered, is there any preference between EMC,Dell and Netapp storage?

thanks a lot for your advice!

I like the way Brian thinks. So far I hear 800 cameras in HD using 1tb per camera for a years storage, not likely if they want HD. As John said this will be PB's of storage. I have heard data upgrade is no problem before and yet, 15 to 20 Mbps per site would require 300 to 400 Mbps at the central location. Have they considered using a tape method where video would be exported daily to tape or other means and then travel to the HQ by transport with other documentation. Old school but possibly more effective.

"Have they considered using a tape method where video would be exported daily to tape or other means and then travel to the HQ by transport with other documentation. Old school but possibly more effective."

Exactly what is wrong with the industry. Sorry.

"Exactly what is wrong with the industry, sorry". Great explanation.

I am not a fan of tape and think it's very niche at best. That said, for someone who wants to keep video recordings for a year, it might be worth looking at. The plus side is probably lower storage (and energy) costs (I don't know how much, if any can quantify, please do), the downside is that it's harder to retrieve video when you need. However, if the long term video (say after 1 or 2 months) is rarely needed, it might be worth it.

Another "Undisclosed" taking offense with a negative response to their suggestion. Very original.

It sounds like this guy has a good relationship with this bank. He is doing the right thing by researching the best way forward in getting them everything they are requesting. He will spend countless hours doing this process, which in turn, will be awesome for him going forward for other clients. He is trying to set them up for years of success with their new system. This will not be a "cheap" solution, but his job is to not only do this important step, but to also show them how much more expensive the system will be if they use some outdated "cheaper" solution.

A few questions for you "Undisclosed":

*What are the internal soft costs of making, storing, validating, verifying, and maintaining your tape solution? Will it all hold up in court if one of these questionable chain of custody "tapes" are needed?

*Who is going to train all of the branch employees to perform this task? And their replacements when they all leave, quit, move around?

*When integrators decide to sell something less than ideal, aren't they the ones who are stuck forever supporting an outdated installation design, and at who's cost?

Sometimes it is better to walk away from a job when ideals and budget do not even remotely fit. When integrators start offering desperate solutions, it always ends badly. It is always better to walk the customer through the process, and let them see the negative side of using older outdated technology, rather than proposing that as a solution. In your post, your tape solution would be great if it came from the bank, then you could walk them through the problems. We all expect that from the other side of the table in meetings. When that comes from our side of the table, it scares me.

"What are the internal soft costs of making, storing, validating, verifying, and maintaining your tape solution? Will it all hold up in court if one of these questionable chain of custody "tapes" are needed?"

Since you are the one who is alluding to tapes not holding up in court, do you have any evidence to support this? Please share.

"*Who is going to train all of the branch employees to perform this task? And their replacements when they all leave, quit, move around?"

You do not need 'all the branch employees' to do this. Given that the plan is to centrally store video, you would have 1 or 2 people at HQ who would access this when needed. The branch people would put in the call to HQ. Maybe this is acceptable, maybe this is not but your implication is too severe.

"*When integrators decide to sell something less than ideal, aren't they the ones who are stuck forever supporting an outdated installation design, and at who's cost?"

You are presuming that it is 'outdated' and 'less than ideal', but you have provided no evidence outside of the allegations above that you have not supported.

Listen, again, I am not a fan of digital tape, but if you are going to have to store PBs that rarely will be accessed, looking at digital tape is reasonable.

...but if you are going to have to store PBs that rarely will be accessed, looking at digital tape is reasonable.

If looking at digital tape is reasonable, then what about mirroring the drives at the branch locally?

When a drive fills up, take one of the mirrors and put it on the Brinks truck, put a new one in.

Plug-it in to a cold array at corporate or let it sit in a vault, depending on how often you need it. After the retention period, send the drives back down to the branches at regular intervals.

Sure you are going to have to buy a lot of drives in the first year, but in the cloud scenario you need just as many, but in this case that just start out at the branches...

Use shingled drives, WD now has a 10 TB unit.

Advantages:

Reliable. As reliable as the drives used in the mirror.

Simple. Few moving parts. No copying / transferring.

Predictable. Costs scale linearly.

Unimpeachable. Data never leaves the media from which it was written. Brinks physical transfer.

Now that is some undisclosed old school VHS magic...

Jeremy,

I'll step out and clarify that I posted as Und1. I tend to do that so as not to propose any solution which could be considered marketing my own solutions. In this case I don't have a dog in the hunt so here goes.

I saw a lot of concerns in the management of storage and transmission in the original post.

However, I did see the OP as a simple question and get no answer. All he asked was NetAPP or EMC. Who has a preference and why.

Brian mentioned the amount of bandwidth to get the data to the HQ and unless there is fiber out there, I usually don't see 30+Mbps uploads around and the cost is pretty high when you can get it unless you are on a campus. Download, no problem. Upload....big problem. What about the aggregate bandwidth and data management at the HQ?

I saw the 800 TB expectation and wondered just how high of recording resolution they were going to expect in that range for 365 days with about 400 cameras remote and about the same on the campus.

I saw a question about ATM integrations with an answer "They prefere pinhole cameras" and that caused a little concern as banking will often want ATM data integration for searching and that can change the project parameters.

I saw a request for a single solution so in the US, that would be Avigilon, MileAxisCanon, DVTel, March/Infanova, IndigoVision, [Pelco] and Verint.

There was a question asked about what country and I beleive the real intent was to understand the banking regulations. No answer there.

Yes, the idea is old school. I didn't see IBM Tivoli presented as an option. Fast to Slow and then Tape for long run. I didn't see ColdStore as an option.

Defensive, no. I was hoping you would outline a reasoning for the simple one word response. Still hoping to see one.

Greg Cortina - FLIR

Michael having deployed already the exact same solution, I would suggest a bank should be looking out for quality, scaleability and propoer SI delivery not only pricing. We deployed a combination of Cisco, Milestone Corporate with X protect retail (if they want transaction based searches), EMC in the data center infrastructure and Lenovo on the edge for local 31 day storage. The requirements for this specific market included minimum 1.3mp for general views and 2mp for specific cameras in terms of recording. I cant go into specifics but we have 27 sites all linked back using vpn. The architecture is relatively straight forward. You federate milestone, using express on the edge with interconnect licenses, and milestone corporate in the HQ. If you have any breakage in links when the lease line connections come back up, the data bases synchronize automatically and the video is available on both sides. We are recording 4 months in the NOC and 31 days on the edge storage. Another point worth mentioning is Axis has a new technology called zipstream which reduces your storage requirements tremendously not to mention data traffic. Bottom line, for a bank environment you want a very robust infrastructure from well known proven manufacturers that can scale out (after 12 years in this business I sadly have to say you can count those on your right hand). I respectfully don't agree with some of the comments made above regarding pricing and trying to get things on the cheap. What is cheap today will cost you and your client alot in the long run. Build a relationship, provide them a well engineered solution, support them technically well, and your business will continue to grow.

Really appreciate your sharings. Actually we did evaluate similar solutions to yours which consists of

Cisco POE switches for IP Camera

Cisco Stackable switches for Uplinks of the storages(EMC Isilon)

Milestone xProtect Corporate Server & Camera licenses as below for 1st phase (sample BOM)

XProtect Corporate Base License XPCOBT 1
XProtect Corporate Device License XPCOBT 268
XProtect Corporate Milestone Interconnect Device License XPCOMIDL 268
One Year Care Plus for XProtect Corporate Base License YXPCOBT 1
One Year Care Plus for XProtect Corporate Device License YXPCODL 268
One Year Care Plus for XProtect Corporate Milestone Interconnect Device License YXPCOMIDL 268

Samsung Camera

Product Description Product ID Qty Comments
Samsung 2MP Full HD Network IR Dome Camera SND-L6083R Wisenet Lite 60
Samsung 2MP Full HD Vandal-Resistant Network IR Dome Camera SNV-L6083R Wisenet Lite 170
Samsung 2MP Full HD Network Dome Camera Wisenet III SND-6084 Wisenet III
Samsung 2MP Full HD Network IR Dome Camera Wisenet III SND-6084R Wisenet III option
Samsung 2MP Full HD Vandal-Resistant Network IR Dome Camera Wisenet III SNV-6084R Wisenet III 19
Samsung 2MP Full HD Network Remote Head Camera SNB-6010 Wisenet III 19

Lenovo(IBM) server used for remote/central recording/mgmt server

EMC Isilon of 700TB storage for the 1st phase

However it turns out after conducting the failover to edge storage which is the SD card on Samsung Camera it seems taking hours just to sync back about 5mins of video recording for 20 cameras at one branch. It's quite unacceptable and probably we are looking into other brand to replace it.

If we are eventually going with Milestone solution probably we have to evaluate another brand for storage and it inevitably could be Netapp E series. Both have been certified by Milestone but I'm not too sure if one outruns another regarding the stability and performance?

we are still at the initial starge in that the solution could change day by day. Hikvision is also another branded company for CCTV solution and they have certain site references in our country as well but what about their client feedbacks or is their product suitable for banking environment?

Thanks in advance!

Michael

"However it turns out after conducting the failover to edge storage which is the SD card on Samsung Camera it seems taking hours just to sync back about 5mins of video recording for 20 cameras at one branch. It's quite unacceptable and probably we are looking into other brand to replace it.

If we are eventually going with Milestone solution probably we have to evaluate another brand for storage and it inevitably could be Netapp E series."

You are assuming that EMC was at fault? You ruled out that it was insufficient bandwidth or the fault of the camera or the VMS? Can you clarify?

Hi John,

So far the EMC hasn't been connected. The test happens between the Camera and milestone recording server over the LAN.

If the EMC has not been connected, why are you saying, "If we are eventually going with Milestone solution probably we have to evaluate another brand for storage"

If you have only tested the camera and Milestone over the LAN, it means you don't have a SAN problem, you almost certainly don't have a bandwidth problem (i.e., LAN), but you likely either have a camera or VMS problem. Yes/no?

hi John, yes you got it right the problem could be either from the SD card from Sandisk, parts from Samsung Camera or milestone recording server but so far we haven't come to conclusion yet.

Hikvision is also another branded company for CCTV solution and they have certain site references in our country as well but what about their client feedbacks or is their product suitable for banking environment?

Well since you mention it and we're talking about cloud banking here, I'm surprised that Hikvision wasn't eliminated just due to the several widely publicized security breaches recently. Is this a concern?

Hi thanks for the sharings. I'm not too sure if it affects the network that's totally isolated from the internet? They will be using Metro-E (private network) for CCTV traffic.

Will there be no mobile access for anyone?

Hi you are right. there won't be mobile access in that case.