Subscriber Discussion

How To Design A Network For A 1000 4MP IP Camera System?

MH
Marcin Heniborg
Oct 19, 2017

Hey there,

I'm in the surveillance industry for about 10 years now (IP probably 5). It's hard to admit but if I came across a challenge from the title I would have a hard time to go about it. Does anybody have a good guide/book/youtube (really anything) that explains the aspect of building a 10gbps backbone network? I'd like to learn all the specific terms related to networking that may be beneficial to an IP surveillance system like: QoS, layers, stacking, aggregation, VLANs and all that could be a part of compedium of knowledge to approach the title's challenge with full understanding and ease. I know much about the cameras and servers themselves already. Network design is a big question mark. 

RS
Robert Shih
Oct 19, 2017
Independent

How many frames per second? What compression? Smart codecs involved? If you're going ham with 30fps and doing a Smart H.265 from your average manufacturer you could keep the bitrate down to 2Gbps which can dramatically change your logistics.

At 15fps, you're in 1Gbps range. Also, how do you plan to spread out your cameras?

Backplanes for high-end switches typically exceed 1Gbps, so there's a good possibility you can avoid building an entire 10Gb network just for this.

(1)
(1)
(1)
U
Undisclosed #1
Oct 19, 2017
IPVMU Certified

If you're going ham with 30fps and doing a Smart H.265 from your average manufacturer you could keep the bitrate down to 2Gbps... At 15fps, you're in 1Gbps range. 

FWIW, the impact on bandwidth due to framerate when using progressive CODECS is typically not anywhere near linear.

See Video Surveillance Bandwidth Guide 2017

Avatar
James Talmage
Oct 19, 2017
IPVMU Certified

My understanding of Smart Codecs is that they only deliver those drastic savings for low interest scenes, with only moderate savings for very busy scenes.

I've got to imagine a 1,000 camera install is either an airport or some other transportation hub. Scenes likely to push codecs to the upper bounds of their bit rate.

MM
Michael Miller
Oct 19, 2017

Yes, this is true.  I have a stadium project with about 100 cameras using H.264 with Smartcodec.  When nothing is going on I see ~150Mbps but when the house is packed it's between 800-900Mbps.  

(2)
RS
Robert Shih
Oct 20, 2017
Independent

Luckily, I used only high traffic numbers in these calculations (which technically is the same numbers as h.265 vanilla) so he's got room for bandwidth overhead.

Avatar
James Talmage
Oct 19, 2017
IPVMU Certified

While it's an interesting exercise, any job that size is going to include requirements that you have engineers on staff with all sorts of training and certifications.

Unfortunately, I don't have a good starting point for you. Pursuing a CCNA certification might be your first step.

MM
Michael Miller
Oct 19, 2017

We have several projects in the 600, 800 and 1000+ camera range.  If this is a dedicated network and in one building or campus this really isn't that big of a challenge.   If this is a converged network or a multi campus environment this can get challenging. 

For 10GB you want OM3 or better fiber plus SFP+ GBICs and switches.   Use a 10GB core switch and access switches that have 10GB up links. 

(2)
(1)
DP
Donald Peters
Oct 20, 2017
IPVMU Certified

OM2, OM3 and OM4 all support 10GBASE so it really depends on the length of the run.  Only OM3 and OM4 support 40GBASE.  Single mode is an option also.

(2)
(1)
MM
Michael Miller
Oct 20, 2017

Yes you are correct! 

UI
Undisclosed Integrator #2
Oct 20, 2017

63 HIKVISION 16 channel recorders.

02 48 port D-Link Network Switches

64 Premade Jumper cables. 

(3)
(5)
Avatar
Brandon Knutson
Oct 24, 2017
IPVMU Certified

13 Powerstrips

(2)
UI
Undisclosed Integrator #2
Oct 24, 2017

I forgot the plastic folding table “rack”

UI
Undisclosed Integrator #3
Oct 20, 2017
Hey guys, thanks for the responses!
the idea is to dissect the problem of this size and to be able to understand all the quirks of the solution. At the same time on smaller project there really is no challenge. Get one switch connect all the cams, do some closed eyes configuration and the job is done so unfortunately that knowledge won't help. In a bigger size projects you need to think about network segmentation, sub netting, right backbone, right switches, right fiber and everything else I'm not even aware of, therefore my question here.
Let's consider a scenario on a picture below. It's a manufacturing plant that consists of 2 buildings of quite a size. Cameras are 4MPx, recording requirement 15fps for 90 days, the scenes are usually busy. There are around 1000 cameras on this image, there are 25 cabinets for a network equipment. How do I even start to design a backbone fiber, where do I need 10gbit and where am I ok with 1gbit.
And above all what brand and model of switches, transceivers etc. should I use.
Thanks.

Example - link to the picture

U
Undisclosed #4
Oct 20, 2017

90 days of recording is a lot, you might need a third building to allocate all the recording servers! I think 30 days of recording is a more realistic situation, and even the 4MPx resolution could be unnecessary upon my experience (I would choose a 720p or 1080 resolution).

 

 

(1)
(1)
DP
Donald Peters
Oct 20, 2017
IPVMU Certified

Using some basic assumptions and calculations the maximum network bandwidth required is 25.3 Gbps (hub and spoke combined).  I agree with UD4 that storage is a larger problem.  Again, assuming 50% motion with high detail and using rough math here -  4MP at 15 fps / 90 days = ~ 7.4 petabytes.  

For performance and security you should also consider creating logical groups of cameras and using VLANs to segment.  Federating your VMS installation is also worth considering. 

This seems like a fairly complex project and may not be one to learn advanced networking on.  I would bring in a network design firm if you don't have that skill set on your team.  

(1)
UI
Undisclosed Integrator #5
Oct 24, 2017

Talley up the bandwidth at each point, add them together and work it back to the headend. When you start approaching that 1gb number, That's where you need the 10 gig uplinks. Go with /21 minimum subnet. And don't forget to consider viewing clients. I would avoid VLaning the individual camera network so as to not require a router or its cpu. Make the headend physically attached to both customer and camera lan. Also 15fps seems high for a typical warehouse, recommend lockers for personal belongings with camera coverage at entry if there worried about small objects.

Also your probably going to exceed throughput at the server itself using single raid setups, so multi nic and teaming as well as probably multiple servers should be considered. Teaming is cheaper than 10gb interface on the servers which is Overkill because it's hard to write data at that speed anyway.

Or just do it the easy way and put a server in each building/IDF

(1)
RS
Robert Shih
Oct 24, 2017
Independent

Just about perfect.

(1)
DR
Dennis Ruban
Oct 24, 2017

such an interesting task, I'd like to have a project like that. And it's not only about money :)

So, the big difference between security and network guys you can see in the comments here. Nobody cares about scalability, reliability and future upgrades. People just trying to save a dime, and solve a current task. As I noticed, most of the security companies work like "do and run", they take a current project, make some profit and don't think about further support and development of the client's infrastructure.

You have 4MP now and in 5-10 years they'll want 20MP or 50MP cameras with a huge resolution and required bandwidth. So, you have to design your network thinking about future speeds, not 1 gbps or 2 gbps which you can utilize today. Just use cat5e for your endpoints, have your access level switches with 10GB SFP ports (Ubiquiti 48-port can do that, I think) but you can use 1GB SFP's to save some money for now. Use some big, reliable (maybe cisco, juniper, etc.) switch in 2n or 2(n+1) redundancy scheme for your distribution/core level. Or just go with Ubiquiti, to be consistent. They seem to be not bad. Read about STP, you'll need it for access-distribution layer connections. There's the whole bunch of youtube how-to videos about ubiquiti configuration.

I don't think you need QoS or VLANs, the only advanced network technology would be STP (actually, RSTP).

For a big network like that, I'd definitely use some NMS. Ubiquiti has some linux-based soft in a beta status but I've never tried that. PRTG is really good.

UI
Undisclosed Integrator #5
Oct 24, 2017

Fiber can be considered core infrastructure, conduit sizes, manholes, physical footprint / sqft of MDF/IDF etc, is core critical forwarding thinking. A slide in slide out network switch is not.

DR
Dennis Ruban
Oct 24, 2017

you're talking about a physical level and it's important as well. But leave it to RCDD guy who designs the cabling. Network design is a different thing and not too many are professionals in both. Yes, in the small companies guys wear many hats but it usually makes them average in all those areas

Avatar
Joseph Hirasawa
Oct 24, 2017
IPVMU Certified

Not to cut you off but I highly recommend you get an individual on staff who is a network engineer, sub this design portion out, or partner with an infrastructure networking company. This is a fairly detailed topic to cover without doing the heavy lifting for you.

 From the onset, you can potentially be dealing with a basic network infrastructure design but that can change based on questions like: 

  1. Are you looking to build a logically/physically separate L2 network for security cameras?
  2. Or, are you utilizing the clients current infrastructure?
  3. Will these 1000 cameras physically and or logically be located on the same LAN/WAN? (Possible routing protocols needed?) 
  4. Is there an element of firewalls & demarcation? (I hope so!)
  5. Are there concerns for future scaling?
  6. How redundant do you want the network architecture?
  7. Whats the budget?
  8. etc..
  9. etc....
  10. ect......

 based on how those (& more) are answered....it could get more complex in a compounded manner. 

Anything is possible, just matters on approach. Good luck!

 Thanks,

 -JH

(2)
New discussion

Ask questions and get answers to your physical security questions from IPVM team members and fellow subscribers.

Newest discussions