Subscriber Discussion

Why Are My Cameras Dropping Off The Network?

UI
Undisclosed Integrator #1
Sep 09, 2015

Hi Guys,

I am having an issue with two cameras (Hikvision DS-2CD4324F-IZH) in an installation. The cameras are configured as such:

[GigE Switch]
|
[10/100 Outdoor Switch]--Camera #3
| |------Camera #2
|-----------Camera #1

There are three outdoor cameras connected to an outdoor 10/100 switch, which is connected to a GigE switch at the back-end. Each camera is powered by 24VAC. The one camera that stays connected to the network (Camera #3) is connected to one 24VAC transformer (40VA). The two cameras that keep dropping off the network are both connected to a separate 24VAC transformer (40VA) - this transformer feeds both cameras power via one 2C#18 line.

The network cables connecting the cameras that keep dropping off the network are about 10 feet long and have been verified using a cable verifier. Camera #3 has yet to have an issue.

I have tried to reset the cameras to factory default, change the network settings around and follow other suggestions. The cameras themselves appear to not be dropping packets, but I will have to make a special trip to site soon to check this out.

Can you guys think of anything else that I could try? Should I replace the cameras and send the two non-working cameras back or is there something else I should attempt?

Any input is much appreciated - thanks!

U
Undisclosed #2
Sep 09, 2015
IPVMU Certified

Hi 1.

When you say they drop off the network, does that mean they are off the network until you power cycle them, or do they come in and out?

How long does it take for this to happen? Minutes, hours, days?

What are you using to split the power from the one 2C#18 line into two? How long are the power runs?

To reduce/eliminate the effect, if any, of the two cameras sharing power, have you taken one off line to see if the other one still drops off?

Similarly, have you considered swapping the good camera for one of the bad ones to see if it's the camera or the infrastructure?

UI
Undisclosed Integrator #1
Sep 10, 2015

Hi 2,

I should have been more clear on my explination - they are off the network until they are power cycled. They are, however, still powered when I remove the cover.

Duration is anywhere from 3 hours to 2 days.

The 2C#18 is in parallel using WP marrets.

I have just taken one unit offline, we'll see how things go. As far as swapping cameras, this is my next move. My curiousity is moreso if this is just a fluke that there are two cameras with similar symptoms or if there is another issue that I am encountering.

U
Undisclosed #2
Sep 10, 2015
IPVMU Certified

...if this is just a fluke that there are two cameras with similar symptoms...

If both units are new and from the same lot, and have never worked from day one, then it would certainly be conceivable that both units could share a defect. Maybe a problem with that particular production run.

On the other hand the fact that they share a single supply and the symptoms are consistent with power issues has to be considered as well.

I would lean toward one bad power supply than two bad cameras.

Reading the spec shows the camera draws 12W nominal and 13W with the heater, so a max of 26W from both cameras. Assuming a high power factor, the 40VAC should be close to 40W, so ~14W to spare.

If the power line is very long though, that would load the system to a degree; and there could be considerable voltage drop which could lead to sporadic failure.

Out of curiosity, does the 10/100 switch not have any POE?

Avatar
Billy Guthrie
Sep 12, 2015

You have to troubleshoot the system as-is; shooting from the hip is only going to make your life more miserable in the long run if you should decide to assume that the cameras are to blame.

Check the most obvious things first:

  • Check the power voltages on the cameras, are they in spec?
  • Is the main power source reliable?
  • Is the power and network cable reliable and in spec? (Did you home run all power and network cables?).
  • When the cameras appear to be off network, are you able to ping the cameras from the NVR or behind the Gig Switch?
  • If the pings fail, are you able to see the MAC addresses on the switchports where the cameras are connected?
  • Verify end to end IP connectivity; unlug the network cable from the camera, connect a host and ping to a remote device on the network (NVR, SAN, etc.)
  • Is DHCP being used for the cameras, or are they addressed statically?
  • Is the network a closed network (Is the network shared by any other resources/users)?
  • Are all three cameras connected to the same switch share the same firmware? Are there any differences in hardware revision numbers?
  • Last resort, if your switch has the capability of SPAN/MIRRORing a port, the use of Wireshark would be the next course of action if everything checks out. Last thing you want to do is send the cameras back to only get new ones that have the same issue; I have ran into this issue in the past.

Hope this helps.

Good Luck!

(2)
(2)
U
Undisclosed #2
Sep 13, 2015
IPVMU Certified

Hi, Billie!

I would agree that the steps you outline in your bullet points will, once completed, will likely reveal or least narrow significantly where the fault is. It's hard to argue with them on that basis. Though, I'm not entirely sure what you have in mind with your opener:

You have to troubleshoot the system as-is; shooting from the hip is only going to make your life more miserable in the long run if you should decide to assume that the cameras are to blame.

Are you recommending against the common technique of swapping 'known good' units for suspect ones in general?

You can certainly shoot yourself in the foot this way, I would agree. For instance a bad power supply causes a camera to permanently fail after 20 hours usage. You swap the cameras, the new one works at first but pretty soon you have two dead ones.

These interrelated types of failures do happen, esp. in your more complex systems. You can chase your tail for hours this way.

On the other hand in this case, where we likely have three or four major suspects and nothing permanently being damaged, as a practical matter, would you say it's the wrong course of action?

Avatar
Billy Guthrie
Sep 13, 2015

Hi Undisclosed 2!

Are you recommending against the common technique of swapping 'known good' units for suspect ones in general?

No; it would have been on my list of things that "I" would do, but definitely not in my top 10.

I have a systematic approach to most of the troubleshooting that I conduct in the field depending on the type of issue that is occurring; I am also not advocating that it is my way or the highway. What I have listed is what I would do when I am in the field. Once I know what is going on, then I take action (Hence, not shooting from the hip).

On the other hand in this case, where we likely have three or four major suspects and nothing permanently being damaged, as a practical matter, would you say it's the wrong course of action?

It is not my place to say whether it is wrong or right; everyone has his or her technique of troubleshooting. I can only recommend that you know the facts before you attempt at replacing anything; it has done me very well with no regrets.

Thank You and Aloha!

(1)
U
Undisclosed #2
Sep 13, 2015
IPVMU Certified

Thanks. That's good advice.

I'm not promoting anything either as an absolute. I was curious because I often have found myself in a slightly different situation, you know where because you are under the gun and just trying to 'make it work', you end up changing several things at once.

Like maybe you have no picture and so you replace a patch, reboot the router, and default the camera all in one go. Then when it works you get that good feeling followed by a queasy one because your not sure what fixed it. Maybe the customer is smiling ear to ear as he sees it working, and it only took 15 minutes... so there is that pressure not to try to recreate it to determine the true cause.

Aloha to you! (did I do that right?)

Avatar
Billy Guthrie
Sep 13, 2015

I'm not promoting anything either as an absolute.

I fully understand, no worries brother! I Just needed to make it clear because sometimes, things can be taken out of context.

It is better to say what the cause of the issue was versus, "Shucks, I dunno, process of elimination is what resolved the issue" which can be more negative depending on whom your customer is.

Back in the day when I was a NOC engineer, reports of packet loss on our network was an every day occurance. Customers would call in and want the NOC to investigate the issue. When the issue was resolved, the customers always wanted to know what the cause was. Typically, by the time we got to the issue, the issue resolved itself (Could have been the result of bandwidth saturation, routing reconvergence off of our network; could have been almost anything) The customer never wanted to hear the issue resolve itself. We then resulted in explaining that the packet loss was due to an evil bit in the IP Packet then referred them to this RFC (Notice the date on the RFC)

That ended pretty quick when a customer was offended; you never know what your customer knows. I think I am a bit off topic, so I will end it here.

Yes, you got it right; Aloha right back at ya!

U
Undisclosed #3
Sep 14, 2015

"Like maybe you have no picture and so you replace a patch, reboot the router, and default the camera all in one go. Then when it works you get that good feeling followed by a queasy one because your not sure what fixed it."

The queasy feeling is caused by that part of your brain that knows you shouldn't ever troubleshoot anything 'all in one go'..... :) This is simply bad practice.

LM
Luke Maslen
Sep 14, 2015
IPVMU Certified

At the 10/100 switch, try swapping the ports for Camera #3 and one of the other cameras, say Camera #1. If Camera #1 is now reliable but Camera #3 keeps dropping off the network, then the 10/100 switch would appear faulty.

Are there so many devices on the network that you might be running out of IP addresses?

Ensure there can be no IP address conflicts. I once had a printer which kept dropping off the network. It had a static IP address. When the printer wasn't going to be used for a while, I switched it off to extend its life and save power. Eventually I found that, in the DHCP server, I had failed to set the range of DHCP addresses above the IP address for the printer. So if the printer was switched off and I switched on a computer which used DHCP, it might grab the IP address of the printer as it was not in use at the time. When I subsequently turned on the printer, it would fail to work as the computer was already using its IP address. So make sure that nothing else can be using the same IP addresses as your cameras.

U
Undisclosed #2
Sep 14, 2015
IPVMU Certified

So if the printer was switched off and I switched on a computer which used DHCP, it might grab the IP address of the printer as it was not in use at the time...

The way I understand DHCP, it would assign the same IP address that the printer used regardless of whether it was turned on or not, as long as it was unallocated in the pool.

UI
Undisclosed Integrator #4
Sep 14, 2015

I didn't read through all the comments, but from my past experience, the first thing I would do is switch out the POE switch. I've had a new switch do this once or twice, was well within the power output under total load, but found the switch wasn't putting out the power necessary (defective switch). You may have a permissions issue or address conflict, but I'd start at the switch power and move from there.

UI
Undisclosed Integrator #1
Sep 14, 2015

Hi Guys,

Sorry for no response, I tried multiple suggestions (including swapping the outdoor switch) but was unable to resolve the issue for more than a few hours at a time.

Today I swapped both cameras. Let's see how long they stay on... I will be changing to a different manufacturer or a PTZ in this location if I can't get anything to work. The client, understandably, is not thrilled with how the cameras have been behaving.

Thanks for all the suggestions - you guys are awesome!

New discussion

Ask questions and get answers to your physical security questions from IPVM team members and fellow subscribers.

Newest discussions