Subscriber Discussion

Arecont Cameras Skipping / Freezing, Connected To Indigovision

Avatar
Salil Kallada
Jun 05, 2014

We have existing 24 nos of Arecont cameras (Model#AV8365) and 16 nos of Arecont cameras (Model#AV1315)

Now connected with Indigovision platform (NVR) (980472 , IndigoVision NVR /40 TB HDD Built in / RAID 6 ) and the Arecont camera frame is skipping or freezing. We are not getting the continuous video in live and recording.

This is the serious issue and we are losing the images.

Please help me to sort out this issue

JH
John Honovich
Jun 07, 2014
IPVM

Mohammed, sorry to hear about that, that is a tough one.

Can you try connecting one or two of those Arecont cameras to another VMS (like Exacq or Milestone)? My rationale for this is that if you connect it to those and you do NOT have skipping / freezing, than you care confidently go to IndigoVision and have them look into potential issues on their end. Alternatively, if you still have skipping / freezing on another VMS, then it's pretty clear an Arecont issue.

Good luck though, Arecont's infamous for their poor customer service and IndigoVision wants you to use their own cameras ;)

Avatar
Salil Kallada
Jun 08, 2014

Hi John

Thanks for the inputs. The Arecont cameras were connected to Milestone before and due to this freezing issue Indigovision came into picture saying that they will connect all their Arecont cameras to Indigovision NVR and confirmed to customer there wouldn't be such issue at all.

Now the problem still exists and customer is totally annoyed.Obviously the total payment is stuck!

LM
Luke Maslen
Jun 08, 2014
IPVMU Certified

Hi Mohammed, based on your answers to John and myself, the problem existed before you installed the Indigovision NVR and the problem is intermittent. Therefore it seems very likely the Indigovision NVR is not the cause of the problem. Given the problem is intermittent, it will be a little tricky to solve. Here are a few ideas which might help to isolate and identify the problem.

1. Are you using the same disk storage with the Indigovision NVR that you previously used with the MileStone NVR? If so, maybe it is not fast enough for all the cameras. Try reducing bandwidth a lot and see if the problem goes away. This could be achieved by unplugging several cameras or changing the compression options to use much lower bandwidth. This would not be a solution but it would test the possibility that inadequate disk storage is the problem. You would need to run the test long enough to be sure the missing frames problem has gone away.

2. Perhaps there is a problem with the network? Is the network switch only used by the cameras or is it shared with computers, printers and many other devices? It's possible the Ethernet switch cannot cope with all the network traffic. If so, you could put the cameras on their own, dedicated network.

3. Perhaps there is a problem with the cables? When the problem is present, use a computer and the same cabling to directly connect to a troublesome camera and monitor the video on the computer.

* If the problem goes away, the fault would appear to be with the NVR and/or it's storage. See point #1 above to test for a storage problem.

* If you see the same problem on the computer, it could well be a cabling problem. Use a short length of cable to directly connect to the troublesome camera. If the problem remains, the fault is with the camera. If the problem disappears, the fault was with the cabling.

I hope this helps. Good luck!

JH
John Honovich
Jun 08, 2014
IPVM

If the problem exists on both Milestone and IndigoVision, it is highly likely that it's an Arecont or network issue. You could rule the network element out by connecting a different camera to the IndigoVision NVR (Axis, IndigoVision, any non Arecont one). If that records / plays back smoothly, it further indicates Arecont.

On the Arecont side, have you verified that you are running the most up to date firmware? Beyond being a manufacturer general best practice / demand, Arecont has a history of serious problems that are later resolved by new firmware.

I would try those two steps first and then based on findings, re-assess after. Let us know.

Avatar
Mark McRae
Jun 07, 2014
Inaxsys Security Systems

Are all of these cameras connected to a single server? How many frames per second?

Each of those AV8365 cameras has 4 x 2MP sensors so it is like having 96 x 2MP cameras plus 16 x 1MP cameras. If this is on a single server, this seems like a significant amount of bandwidth/throughput...

JH
John Honovich
Jun 07, 2014
IPVM

Mark, it could be total load but the server model number he lists is spec'd for 500Mb/s and 200 total channels.

It would be worth checking total load on the IndigoVision NVR as well as whether these issues are happening non stop or only during certain times (like at night, etc.).

Avatar
Marc Pichaud
Jun 07, 2014

I do agree: check out VMS bandwitdh max input and outputs... ( not only inputs) because video wall and any client outputs using basic high resolution streams could saturate the server CPU as well. Multicast settings as well (does explain why we can't put too many camera per server with Genetec)

That's why also VMS using second or third low streams are recommended instead

then max Read / Write in term of bandwitdth trhoughput on the NAS, not only fps writting

Keep in mind a 8 cameras synchronized reading in fast speed can take 8x 30 Mbits= 240 mbits ! only for one operator working on a search.

VMS Bandwith Input: (but also throughtput to the NAS) so use same for NAS output

8 Megapixels : 24x8*=Mpixels outdoor with moves & lights changes= 192 for the REC only ...

10 Mpixels cameras : 16x10* Mpixels outdoor with moves & lights changes= +/- 160Mb

(*may be more at night) TOTAL: +/- 352 Mb/s input and 352 Mb/s output to NAS

If your server is doing motion PTZ tours, motion detect, or analytic add some CPU, used by pre/post alarms and analytic engines

+ any optionnal display streams : for example, VGA for multiple displays or QVGA for SmartPhones

So CPU should stand 700 Mb instant treatment minimum.. not easy on a single server

And during replay .. could reach 940 Mb ....(in that example)

Port miroring feature on Level2+ switchs can help to see whats coming in and out from the VMS during normal usage. Night time is always the best to do the stress test, with synchronized replay which probably kill most systems

RB
Rob Beck
Jun 07, 2014

I had this issue with a set of three avigilon cams running in JPEG2000 and one LG tied onto avigilon VMS running with a small four port 10/100 switch. I ended up swapping over to H-264 on all of the channels and bandwidth dropped significantly from say double digit (10-14)+ Mb per channel to 2-4 Mb per channel if I remember correctly. As soon as we did that the frame freeze loss issue disappeared. Our conclusion... bandwidth across the little 10-100 switch was too high for the location. We think the switch might be to blame as we are only running four channels on the recording unit.

LM
Luke Maslen
Jun 07, 2014
IPVMU Certified

Hi Mohammed, is this a new problem or have dropped frames always occurred? If it is a new problem, what has changed since it was working reliably, e.g. added more cameras, replaced an old NVR with a new model?

Avatar
Salil Kallada
Jun 08, 2014

Hi Luke

This is an intermittent issue.Sometimes it drops drastically.

I am trying all options.

Thanks

Mohammed Salil

Avatar
Ricardo Souza
Jun 08, 2014
Motorola Solutions • IPVMU Certified

First thing that would come to mind is check the Tput

From the datasheet (http://www.indigovision.com/documents/public/datasheets/Enterprise-NVR-AS-4000_Datasheet_A4.pdf):

100 channel version

Performance (Streams) 100 Recording plus simultaneous playback of 25 streams
Performance (Throughput) 128 Mbps in; 200 Mbps out; 328 Mbps total

200 channel version

Performance (Streams) 200 Recording plus simultaneous playback of 25 streams
Performance (Throughput) 500 Mbps in; 200 Mbps out; 700 Mbps total

Second thing that would come to mind is check the Network.

Are you using Multicast? If you network multicast is not configured well, you would probably get a disconnect from the cameras every 30sec.

Run Wireshark to see if there are any packets drops. In wireshark go to Telephony > RTP > Show All Streams to analyze each camera stream.

BK
Barry Kasselman
Jun 08, 2014

Hi Mohammed,

I would recommed you to contact your local partner support team - If this is a NVR issue we would be happy to assist you to resolve this matter and escelate - you can send a mail with all the relevant info to the global partner support mail - partner.support@indigovision.com -

Avatar
Marc Pichaud
Jun 08, 2014

Ricardo gave the solution : your 100 channels Indigo is probably saturated (even a 200 would be during searches and replay if you have a video wall)

You know that with an excel sheet. Don't spend time on network, but you can checkout NVR real bandwidth with the port miroring on the switch, easier than Wireshark. It s an IT procedure, no blabla. I'm very surprised people don't use it.

128 Mbps in; 200 Mbps out; 328 Mbps total... very low Input and completly impossible to fast speed replay 8 Arencont cameras (+/- 240 Mbits..) ! no power margin for that

Current CPU is overloaded and probably reach 70 to 100% all the time.

=> If you want to keep your NVR, you need to tradeoff: reduce fps, and quality, use second stream resolution for display, use CBR locked in, based on what cameras delivers in worst conditions in VBR (at night ) to keep a suffisient quality. If the Indigo pushes its own settings to the cameras (check that out) , like VBR and quality presets your are done. Then you should change for a more powerfull nvr.

Avatar
Marc Pichaud
Jun 08, 2014

If you can connect correctly from a new independant PC , to the cameras usually freezing with VLC and checkout that the Fps and bandwidth match what you expect, it does prove: the cameras are not saturated, the network is working well, BUT your server and/or your clients are overloaded. (could be Indigo and milestone, as with 40 very high megapixel cameras... it's like with 80 HD/FHD....

UM
Undisclosed Manufacturer #1
Jun 08, 2014
Why isn't someone from Arecont responding here? I haven't dealt with them directly in several years but it would seem that they are still their own biggest enemy. If you call and complain, experience tells me they will eventually give you your money back...and then tell you to never buy their cameras again. Just saying.
ng
nidal gharzeddine
Jun 09, 2014

Hi Mohammed,

Test the live feed of the cameras through the web browser, and keep it running for a few hours. Compare the quality of the video stream (browser VS NVR) and determine if your NVR is the cause or is it the network/camera.

Same quality on the NVR and the Browser:

If you’re getting the same quality on both, it wouldn’t be an NVR issue. Try reducing the camera count on edge switch (one a time) until you get the quality you’re looking for. Consider upgrading the edge switch.

In case the quality remains even after reducing the load on the edge switch, call Erecont.

Better quality on the Browser:

The NVR can’t take the load, or the camera’s driver on the NVR needs some work.

  • You might want to connect one of the cameras directly to the NVR (no switches) and check if you still get the same results, that could tell you if you have a driver issue.
  • If these cameras are out door, check if the quality is dropping around mid-day, might be the heat.
  • What switch models are you using? How many cameras per switch? Is the POE consistent?
  • What is the cables length? What is the quality of the cable?
UE
Undisclosed End User #2
Jun 09, 2014

Save your time and efforts - just get an end to end working solution from Avigilon.

UM
Undisclosed Manufacturer #3
Jun 09, 2014

It seems gave a pretty good breakdown of how to analyze the problem, but I have another question..... Are there any non-Arecont cameras also attached, and if so, what brand are they?

Avatar
Salil Kallada
Jun 09, 2014

Hello All

Good News is that Arecont's Technical team immediately on this issue from their side and inspecting all other possibilities . Thanks for the same.

Avatar
Marc Pichaud
Jun 09, 2014

Yes, but you should keep in mind that most issues are not coming from the camera ...(only if they are saturated with toomany streams and motion detection..) Most guys from vendors doens't know how a VMS is working and how to test a CPU load. I don't know what Arecont team can suggest on this ...just update your firmware ?

Good luck

U
Undisclosed
Jun 14, 2014

I am curious how this worked out. I have heard of this with other Arecont Cameras (not on Indigo or Exaq) And yeah, where is the vendor in this conversation? I am sort of assuming the awesome staff at ipvm.com pinged them for a response. Hope the project was able to deal.

New discussion

Ask questions and get answers to your physical security questions from IPVM team members and fellow subscribers.

Newest discussions