Experiencing Periodic Loss Of Video In A New IP Camera Installation

A dozen Pelco S6230 HD cameras are operating on a 10 Gb network, connected through Cisco 4507 R-E access switches. Picture quality is great and PTZ functions operate well, however we have been experiencing complete video loss at random times. Duration of loss is typically 15-30 seconds, with rare protracted outages.

The length of run to these cameras varies from 100 to 120 meters. For this reason ethernet extenders were used delivering PoE to the cameras, with the results as described above. Now the extenders are removed, and the cameras are locally powered via 24VAC, with exactly the same results.

Appreciate any comments and suggestions.


Have you ruled out that you do not have sufficient total PoE power? Those are Spectra PTZs and they surely take a fair amount of power.

Have not ruled that out. Good comment.

But the cameras are locally powered, no?

Sorry for the non sequitor. Yes there is local power now and the performance is similar to what it was with extenders providing PoE. I believe the previous commenter recommended using ethernet-only extenders as the solution, which we have been contemplating doing as our next step.

Do you know whose/what model extenders you are using?

The Nitek EL1500 U

Are the same cameras going offline everytime, or is it a different group?

observing them, it seems that some are more prone to that and others are stable.

You will still need the ethernet extenders in place, even with 24v AC powering the camera, as you stated that the distance is up to 120 meters, past the limitation of 100m of ethernet.

Hear you, but the performance with extenders was the same as without. May be that the resulting PoE power is insufficient as John suggested.

Agree with 2, put the extenders back on AND power locally, see if that fixes it.

Have a recommendation for an ethernet only extender? Looking at the Perle line up.

What do the lights on the switch show when offline?

Will be trouble shooting on Tuesday. Will let you know the outcome.

I would check the switch port config. If I recall a long time ago in my past, I had issues with the Autonegotiated speeds (1G, 100M half duplex, 100M full duplex) with some Pelco cameras and Cisco switches. They definitely werent 4507 switches, probably 3550 switches or something of that vintage. I believe we had to hard set the ports to 100 full, because randomly the Pelco cameras would try to renegotiate mid-stream, and the port would reset communications. It doesnt seem exactly like the same problem you were having. For example, we would actually drop pings and it would take longer than 15-30 seconds for the camera stream to reconnect. What VMS are you running? Are you losing all connectivity or just video streaming (port 80 vs port 554 issue)?

Could be that this applies to at least some of the cameras, the behavior you described being very similar. I will pass it on to our network engineer.

The VMS is Verint Nextiva.

Thanks!

I tried this on my exacq/axis version of the issue and it made it worse.

You could try using your existing PoE extenders and the local power together by lifting the blue and brown pairs on your CAT cable at the camera patch cable, assuming 586B wiring. This will remove the pairs used for PoE for testing purposes. I would recommend making a test whip by normally terminating the 8P8C on one end of a short cable, but a keystone on the other end with only the orange and green punched in. This will still allow the transceivers to receive PoE power, but just not the camera.

Had been considering that. Will put that to test if troubles remain post the port config Sean advised.

Thanks!

Lol. Jon I was going to suggest this to you yesterday, regarding your NVT extenders yesterday. I didn't because I couldn't be sure that the high power POE (60W) would only be on the spare pairs.

Here, I can't see for sure that it will work, but considering it's just POE+, I think it's likely to.

This was the info that NVT gave me yesterday as a permanent solution. I'm not sure I want to have missing pairs in the field as a permanent solution. That could be "fixed" by the next guy and mess things up.

Ha! Would not have thought you could do 60W on just the spares.

You can also use one of these common passive injector adapters to kill the spare pairs, just don't connect a DC adapter.

Thanks. Eases the process of killing the spares.

What is the headend, we have a similar issue

The VMS is Verint Nextiva.

I just saw the whole post. Didn't show up at first glance. At 120m you'll have no issue if your powering the cameras locally. The extra 93 feet will do little if anything at all on the bandwidth. A camera only needs maybe 20Mb tops and that 100m rating is for the full potential of the cable probably 1Gb if your using cat6. Cat5e has the functional equivalent of Cat6 at half the length. Cat 6 has the functional equivalent of 6e at half the distance.

Were having a very similar issue with axis and EXACQ and have not been able to nail it down, 10-15 second drops and random cameras. We have ruled out network switches by connecting with single port injectors directly to NIC interfaces on servers, still occurs. We dismissed power being an issue, because a power loss would cause a reboot and that is longer than 10-15 sec. So some process we believe in the camera is causing these. Maybe a time sync, NTP issue, maybe a password request with encryption errors. We are going to try the duplex option and see if that has an effect. Also as a quick test, you could try wire shark and see where the drop occurs, relating to either camera or server. Maybe even a constant ping to both devices may shed some light.

Cat 6 has the functional equivalent of 6e at half the distance.

6e or 6a?

Either one depending on brand, 6a is a 500Mhz spec in terms of speed and throughput, 6a has some other differentiators but as for speed 500Mhz is most brands 6E or 6+ version. I haven't looked into it yet, but i would assume 6a (10gb) has the functional equivalent to cat7 (40gb) at at half the distance. They just play with the twists and gauge for the most part once you take out the shielding and seperator aspects.

Good comments.

Cables have been certified.

Removing the extenders from the equation if deemed unnecessary would be very helpful.

Will speak to TEK about applying Wireshark should the problems continue following remedial efforts

NTP is absent at the moment as the connection to the network source is incomplete.

Thank you

Lot of good suggestions here already.

Can something external be triggering video loss? Have you observed anything in common about the times you're getting video loss?

Have you certified the cables and ensured you're getting the speed you need?

Have you checked for packet loss and latency?

Very random. Not related to activity or any other observable phenom.

One other thing to check if the cameras are not on their own lan or vlan that there is not something probing them (again wireshark is your friend here _:). We had an issue where a port scanner probing the network endpoints was pissing off the cameras periodically. In this case it was some anti-virus stuff IT had turned loose could be some other source as well. Was hard to track down as the scanner would wake up do a range of IP addresses and stop. Then wake up later and do a different range of IP address. BTW all the suggestions so far seem excellent.

Very interesting and unexpected twist!

BTW: the team troubleshooting session has been delayed due to some members unavailibility.

I will provide updates as they become available.

Thanks

Spanning tree.

15-30 second connectivity loss can indicate a Spanning Tree error somewhere in the chain.

I would check your switches/routers for any spanning tree logs/errors/notifications/etc