Genetec and Milestone 500 Camera Recorders Announced

But why

Just added a poll on this: how appealing is a 500+ camera recorder to you? Take a minute and answer above.

"In most networks, gigabit Ethernet is the fastest interface available, and is limited to ~800 Mb/s actual throughput or less"

Yes, my experience is that is burst, with sustained throughput being actually quite less, maybe 60% of the wired connection's rated speed.

Can Miletone and Genetec clarify if that is sustained throughput or burst?

Below is the correct info for Avigilon's HD NVR Premium servers. We have a hand full of these in the field with one pushing 230 cameras and ~500Mbps. For large enterprise customers these large capacity servers are very useful IMHO.

I'd be curious to know how well server side VMD works if enabled for every camera on a 500+ camera NVR?

Anyone got any experience with that?

single point of failure eeek!

prefer my VMS split between multiple Virtual machines and Multiple physical servers

Exacq does not not have a fixed limit, you can license until it craps out. With SATA drives performing at 3 Gbs and up network is the weakest link. Assuming you bond your interfaces plus configure the switch for it and by establishing a secondary viewing & management Lan.

I could be wrong here, but if a client has 500+ cameras, they likely have 10GbE LAN too, right? Why would 10GbE be too costly? Wouldn't a server of this scale be a premium over a cluster of smaller servers anyways? What's another >$1k for a 10GbE NIC?

I really dislike the use of number of cameras per NVR. VMS/NVRs should be rated in throughput with a camera max listed as secondary information. I like how Avigilon and the manufacturer I am evaluating specifications indicate max throughput then max number of cameras. Many still list # of cameras only and you have to dig deep to find throughput information.

Evaluating the throughput rating of NVR's has been a big question of mine for some time and I currently have a design that I have been working on that has over 700 physical cameras with over 1000 camera streams (some are multi-imager cameras). Total throughput is around 12Gbps and storage is approximately 3 to 4 PB for 30 days (24x7x30).

The manufacturer of the VMS we are evaluating claims a throughput of 400 Mbps using their NVR (customized Dell server) and 300 Mbps for 3rd party server. These numbers are recording throughput and that the same rate can be achieved for video retrieval.

The client requested that the project follows their current data center guidelines and that we add to the existing VSphere environment using Cisco UCS servers and a Dell storage area network.

For this project, we evaluated the option of using Cisco UCS B Series Blade Servers (Dual CPU Xeon E5-2660 B420 M4 Blades) with one NVR instance per CPU. The Cisco UCS chassis supports VIC (virtual interface computing) network interface cards that allow for high speed fabric connection directly from the virtual machine to the core network using multiple 10Gb interfaces. The servers would be connected to a Cisco Fabric Interconnect switch that is connected to a Nexus 7004 core via multiple 10Gbe interfaces.

For the storage solution we are looking at Dell Compellent FS8600 storage controllers connected to the Cisco 7004 via multiple 10Gbe interfaces. This solution drops off the fabric at the Cisco 7004 utilizing 10Gb iSCSI to the Dell SAN. For the storage backend we are using Dell S5000 switches and SCv2080 storage arrays.

I would think the server configuration above would provide the 400 Mbps throughput rating equal to the manufactures provided server or even higher. Without having a way of testing or confirming the actual bottlenecks in the NVR platform it's hard to make any assumptions.

The other issue is the cost of the virtualized server and storage area network in this solution is a more expensive option than using the manufacturers NVRs and the management without a qualified IT department would be a nightmare. The manufacturer offers 90TB single NVRs which would provide all storage in the NVR without need of any external or attached storage. From a keep it simple standpoint the standalone NVR would be the best option.

12000 Mbps total throughput / 300 Mbps (400 minus some breathing room) = 40 physical NVRs for this project with 3600 TB of storage. Using 300 Mbps (they recommended 200 Mbps for virtualized) we would need ~60 virtual machines.

Other issues that come up would be server side video analysis and transcoding. What is the impact of storage only on the CPU and at what point of utilizing the CPU beyond this causes any impact to the storage throughput. The manufacturer being evaluated does have a tech paper showing different camera/server configurations and results that help make some assumptions but it is based on their NVR solution.

It would be nice if IPVM could start a section on NVR throughput testing and bottleneck evaluation. I would love to hear from other users here on their experience with NVR throughput, especially in a virtualized environment.

EDIT: One point I forgot to make and was a reason for my comments above. We are using high resolution high bandwidth cameras for all designs now. As such we have a need for high throughput and not high camera count NVRs. The solution I indicated above would only be 15 to 25 cameras per NVR.

NOTICE: This comment has been moved to its own discussion: 700 Camera Server Sizing Question

Milestone confirms their calculation assumes camera side VMD. Server side VMD of 500 cameras would likely add a very high processing load. Related: VMS Server Load Fundamentals Tested

I have personally found Milestones recommendations as to what a specific set of hardware can and cannot do to be useless

"on that hardware you should have no issues with 60+ camera's"

when we find services starting to fall over and other issues when we get close to 40 cam's on the hardware we listed to them, this is without anyone actually accessing the specific server.

I've always found their estimates way off.

Theoretically anything is possible and comes back to the "bottleneck." It is not what they do tell you in the press releases but what they don't tell you.

First concern is the read/write speed of the hard drives which is where you can first back up. So the answer to increasing the throughput is higher speed drives which are disproportionately more expensive and then financially illogical.

Next you have to look at the RAID array configurations. RAID 10 is going to double the Mbps throughput of a RAID 5 scenario. Of course RAID 5 you would lose approximately 25%+ with storage and RAID 10 50% in storage. Unrelated to this is what no one ever seems to consider is what happens when the RAID becomes degraded - testing shows that in a RAID5 you can lose more than 80% of your throughput during a rebuild while a RAID10 is nominal.

Next is the process of virtualization. Using a blade server and virtualizing the distribution and storage arrays.

Once you have figured out the actual throughput then of course we know it is a matter of math. The age old question then of how to make the math work of lower frame rates and resolution which subsequently lower overall bitrates. The question i often ask is if you are going to lower the resolution and bitrates on a high megapixel camera then why did you spend the money in the first place on a camera which has the capabilities to provide higher quality images. Bitrate = Quality.

So yes probably even 1,000 cameras is possible as well if you work the math and the hardware!

You mean I can put up one box and record all my eggs in a single basket! Wow! Think of the savings!

I smell BCDVideo in here somewhere, not that that is a bad thing - but 500 camera server is just asking for trouble if it goes down.

I remember more than a decade ago when we were marketing 32 channel DVRs and we were told that's ridiculous no one is going to buy it - who in their right mind would put more than 16 cameras on a single server and take the risk.

Update: I just saw an old relevant discussion to this. 2 years ago, IndigoVision released an NVR spec'd for 700 cameras / 2,000Mb/s throughput.