Thanks Nidal for sharing that. A few questions:
1. The Camera MD column is all set to NO, is motion detection on the server side on this one server?
2. Notice the FPS, seems to be restricted with the number of frames, have you found 9 to be the max on this one server?
3. Also, the cameras are not writing? notice the last write time column which is well before the last time an image was received from the camera...just trying to understand this report and if it was taken at the same time as the perfmon image.
Wondering if this setup is logical/sustainable given the bottlenecks you mentioned and the report data which indicates it was last recorded in May.
From my experience of large installations (2,000 max) one of the challenges is the number of server required. By reducing the server count the cost would be reduced- hardware, OS licenses, management , power, cooling, maintenance etc...
Some of the bottlenecks with a reduced number of servers would be the network connectivity and the storage throughput.
I can share with you below screen shots of a large school that has great infrastructure that allowed to connect all 1,721 of their Act TCM 7411s (average at 1.9Mbps) to one Dell 720xd, equipped with dual Xeon E5-2603 1.80GHz processors, 24GB of Memory running Server 2008 Enterprise connected with 10G. The attached screen shots shows network utilization at 3.65Gbps and 66% CPU utilization. The cameras are recording 50% of the video or 2Gbps via a second fiber connected Dell Compellent. The other attachment is a export from the Server Statics that detail each cameras specific resolution, FPS and bitrates (pay close attention to the Server column.)
The VMS used was Video Insght.
dropbox link - single server 1721 cameras
I have been involved in deploying Exacq and Lenel video in those numbers. I have been involved with pushing large numbers with ipConfigure, but not at the 5,000 camera number.
I could list the lessons learned here, but it would take a while. If you want to message me separately, I can give you some of them.
Robert, Which software solutions did you use? Thanks in advance
Josh, Thanks for the information. I'm more interested in the software administration solution, rather than the infrastructure precautions. Do you know any centralized installation with these number of cameras using milestone? Thanks
My bad John. Next time I will email you.
Here is some key things to look for:
- Network Setup
- How is their network setup ? Types of switches/ WAN/ Wireless used ?
- This is huge and often missed - make sure you understand what you are going to be using for network transport from edge to core
- How is security's relationship with IT ?
- This will help you understand what you need to take on vs- leveraging their IT team
- DHCP / DNS host naming is w/ pre reserveration really helps with this many cameras- IPs get out of hand quickly without a simple way to reference what is what
- Look for camera manf. that have good independent managment tools- really helps in setting up and changing settings independent of VMS if needed
- Local vs Central Storage
- Are you going to store at each location or in the local telco closet or stream to central location ?
- Remote Access
- If you are going to be managing or implmetning the system VPN access is huge with this many cameras because things will go wrong and nice to be able to troubleshoot off site when needed
- Indepdent inventory managment
- Have a good system / CRM for managing each part of system- cameras IP / MAC server info etc- this will be vital as you build out and maintain
Whether you use Milestone or another large VMS they all have their PROs & CONs- talk to as many large users as you can - it will really help you shed light on the challenges- no matter what VMS you choose a lot of the challenges will be the same- network issues, servers problems, capciity etc- these are not unique to a single VMS.
Josh, if you have something to share on the topic, please do so on the discussion. Otherwise anyone who wants to privately assist, email me to facilitate rather than post offers on the thread.
You can reach out to me most my clients are very large and several use milesone.. what specifically are you looking for ?
John, They will be at 1 single location, to 1 single (maybe 2) control rooms (More like a city rather than a "store scenario"). Robert, Not particularly Milestone, but the client is really interested on them. What experience do you have in this kind of projects?
Is this 5,000 cameras in a single (or few) locations ("campus") or is this 5,000 cameras scattered across hundreds of locations ("stores")? Features and fit vary depending on how the cameras are physically grouped.
Are you looking for generic information on deploying multi-thousand camera systems, looking for info on deploying Milestone with 5K cameras, or would you be interested in alternative VMS' experience.
I have not done a Milestone deployment, but I have done similar sizes with two other VMS manufacturers.