Here are some common approaches and rationales for managing large numbers of cameras:
More powerful servers managing 100+ cameras each. Rather than using NVR appliances, purchase COTS servers with multiple CPUs, higher amounts of RAM, etc. For example, the Avigilon NVRs come with a single CPU and 6GB RAM (see specifications). More powerful COTS servers can reduce total number of boxes needed and total cost.
- Storage centralized in a SAN or NAS instead of storing inside of each NVR appliance (i.e., one central 'pool' of say 300TB vs 10TBs inside each NVR appliance). This is easier to manage and reduces inefficiencies as storage is shared amongst all servers. For example, you do not need to worry about any single NVR needing more storage or wasting storage on a particular NVR.
- VMSes running on virtual machines often allow for more cameras to be recorded/managed on a single server (i.e., multiple VMS instances, one per VM) as well as facilitating transfer of that VM to a new machine (for service, fault tolerance or expansion).
With regards to Avigilon's NVR, they are specified to 32MB/s (i.e., 256Mb/s) throughput and/or 128 cameras. At 33 cameras per NVR, that implies an average bandwith consumption of 8 Mb/s, which is quite a lot even for full frame rate, multi megapixel continuously recorded video (unless it's JPEG2000 and then....)
Avigilon's employees and partners have gone berzerk on twitter, mostly with ad hominin attacks (calling us 'such a dick', etc.) but have not offered any specific facts or technical details that would explain the decision.
1. Avigilon will not record to NAS
2. 256 Mb/sec standard advise by lots of VMS companies
3. 8 Mb/sec about right for H.264 specially at night time
I personally never put more then 24-30 cams per server ( 2MP at 30 frames )
Virtual Machines choice is up to customer not up to me
I offer they decide
IPVMU Certified | 12/25/13 12:33am
there is a potential for this having been a install specified by an engineer... in that case spec may have called for avigilon nvrs vs. the cots servers you mention... sometimes you can substitute with an equal or greater product... without much for details I am just speculating...
The most telling part was I wasn't even recommending running the OS on a VM, I was personally thinking about the storage aspect. That didnt seem to dawn on anyone...
Really the number of cameras per server is irrelevant without bandwidth. Also, many times customers plan on expansion so completely loading servers to the max may not be preferred. Without complete details this is absurd speculation and completely fruitless albeit entertaining
Without knowing the application and customers concerns, objectives and requirements it impossible to say whether this is a good design for this application.
We would not by default design or engineer a system like this but we have been asked by some of our customers to provide solutions like this and believe it or not there is some very valid logic/reasoning for requesting solutions like this
Why choose only Avigilon for servers/storage, cams and software? Customer wants complete turn key solution from one vendor "one throat to choke / no finger pointing / one call for service". You may find it weird but our largest customers are the ones that request this the most for standardization, efficiency and support.
Why not consolidate servers storage? Customer wants to minimize / limit the exposure / affected devices in any single piece of equipment failure (including servers and storage).
Why so few cameras per server? Customer has cameras set to high frame rates and bandwidth reducing the number of cameras per server or System is designed for worst case bitrates Or customer plans on doing failover or camera additions in the future and has budget now for servers... or bla bla bla.
These are some possible reasons behind a design like this - but...
Again, without knowing the application and customers concerns, objectives and requirements it impossible to say whether this is a good design for this application.
John what you know is enough to speculate - that's it.
I will say this - we have servers from Avigilon running over 100 megapixel IP cameras on a single server and we have some running over 180 analog cameras on a single server. However we have some running 30 or fewer cameras as well. We have applications where we go with servers straight from dell and create a few VMs on them and utilize centralized SAN for storage. We also have applications with customer provided storage and servers. Crap - we actually have Avigilon running on $400 pcs from Walmart with 40 analog cams and a few IP no problem. The fact here is all of these applications we utilized the appropriate servers and storage based on the specific requirements, concerns and budgets for each customer. Every one of these customers are happy as well.
I see no reason why Avigilon or the integrator should try to explain the application on this site simply because you decided to post an article. This does not give Avigilons competitors any type of advantage either. Trying to use this speculative, incomplete ponderance of an article to gain a competitive advantage would be ignorant.
I know as well as you by now that you do these articles for fun probably when you are bored or have nothing else to write about - to see what responses you will get. This holds no real value to any of your users (other than entertainment) because it is completely speculative. It's a "bait" article - good bait though trying to get a company or your users to defend a product because of your negative speculation...
As this is a solely speculative discussion and you are speculating this is a bad design I'm simply stating there could be good (relative) reason behind this design. I do not know the installer or the application. I'm just providing some insight into possibilities other than the only negative possibilities laid out by John. As "proper" design is relative AND dependent on so many factors that are not provided in this discussion we have absolutely no clue who is closer in the ppossibilities laid out.
Ultimately, if the customer is happy with the end solutuon and within budget everything else is a moot point.
Lots of Avigilon people are saying John is merely speculating when saying that 32 NVRs running 1000 cameras is not the most optimum architecture for this deployment.
Please provide ANY reasoning that makes sense for utilizing such an architecture in THIS particular install.
I would like to wish everyone a Merry Christmas! Hope you all are having a great day and have the opportunity to spend valuable quality time with all those that are important in your life.
John, Sean: I think it would be helpful to members if you could identify the material facts in this case.
On first reading all I see is
- A rack with what appear to be 13 double height nvrs
- A negative statement regarding how infrequently one sees 32 nvrs
Regressing thru the twitter thread I see a statement about 1000 cameras by someone different than the one making the 32 nvr statement. I can't really say I saw much 'showcasing'.
Are there other tweets/conversations that are relevant?
Perhaps John could (or has already) reach(ed) out to JP for clarification. Maybe even offer him a temp login for a roundtable discussion??
On the bright side even Ryno_Video admits "It's not often you get to see 32 Avigilon NVRs on a site.", so such low density configurations for whatever reasons may be not very common.
P.S. If the 32 nvrs are in a single rack we're talking about one of these puppies:
Now that's a rack!
FWIW only three of the NVRs seem to be on...
I have to agree with some of the undisclosed about "Real World". Sometimes providing the right solution isn't what is going to get you the job, because unfortunately someone else will give the customer what they want, even if it's wrong.
As both a Dell direct partner and an Avigilon dealer, I can tell you that there is not all that much difference in cost to me between the two servers. While I read most if these comments I did not read them all (this seems to be a hot topic), did anyone consider that maybe the dealer decided to utilize Avigilon servers for the sheer fact that Avigilon would have not have a leg to stand on if there were performance issues surrounding the servers? I can't tell you how many times i have installed servers that meet or exceed the "Recommended" specs only to have a manufacturer question them right out of the box if we were discussing performance issues. The next thing they question is the network itself, which is usually dropped once I ask them which if the 3 Cisco CCIE's in my office they want to discuss that with.
I have my first medium scale Avigilon install within the next 30 days (365 streams), I will let you know how it goes. I hope all I am hearing here about Avigilon is just a bunch if noise, but if it is not, the engagement will be short lived.
Some VMS systems can only handle video databases of a certain size. The larger the database gets, the slower playback and retrieval time gets. And then some just have a maximum limit, like any SQL or Oracle database. You can get more powerful machines, maybe, but some databases only scale to certain sizes, so you have to spread out overall size across multiple machines. (And more powerful servers and SANS sometimes cost almost as much as multiple smaller servers.) 33 cameras maybe not seem like a lot of cameras that would make a sizable database, but if the client requires a very long retention time (60 days... 90 days.... 180 days), that's a large databse for just a few cameras.
I've never sold an Avigilon NVR, always use COTS and buy the software separetelly, a bit more cost effective for me and the customer with pretty much the same warranty.
Without knowing the application it's hard to judge if the hardware investment was justified. Also I'm assuming with a 1,000 camera deployment at least a couple of those 32 servers are set for standby failover.
Of course Avigilon OEM's the DELL Rack Servers for their HD RACK NVR's. 24x7 4 Business Hour Critical support worldwide, and for many companies this will mean a lot.
If we have a local client, where we are writing and supporting an remote/onsite service support agreement, and they had a absolute minimum camera count, and a absolute MAX budget, I would consider an alternative server, one that we have extensive IT experience with and would match, (actually nearly double) the 32MBytes/Sec recommended max on the Avigilon Server.
We've tested the Intel SR2xxxx servers, with LSI controllers and RAID10 configurations and have exceeded 64Mbytes continuous write speeds.
I believe I could place 100 Cameras on each of 10 servers, and have Each Camera to have a secondary/tertiary to split any one server that failed and divided it's 100 cameras with 11 going to each of the remaining 9 servers....
We have this in miniature in place......
But we sure like taking a server out of the box, and getting 5 minute setups, so doing my own server would be carefully decided, and done as a last resort to meet the needs of the client. Of course we assume the tech risks, but having been doing that in the IT world for 25+ years, it would work as expected.
I've re-read the posts, and we've all made assumptions about cameras based on our own experiences. Does everyone know that Avigilon has a 16 and 29MP JP2K camera at 2IPS high and compression that can create 50mbits and more in bandwidth. Assuming the installation is going with the PRO series cameras, I suppose 30 cameras could stress an NVR, and based on the retention requirements. The math might be perfect.