How To Make IP Cameras Work Better With VMS Software, From A Developer

I have written 15 or so VMS drivers for different camera manufacturers and so I see cameras from a different perspective to installers, manufactures, managers etc. Some cameras work well with VMSs, most are less than perfect, but the Arecont was the worst...

One thing I notice is that the cameras I like and do not like and pretty much the same as the cameras the installers like. I really like Axis, I do not like Arecont. Therefore, I am thinking that if Arecont changes things I don't like about their cameras, their problems might be solved. I am wondering how many  problems with the cameras are that they just do not work well with third party VMSs?

So from a VMS developers perspective, this is what I think Arecont should do:

Hire a good Windows C++ developer and write an application designed to test each camera model to the limits that I think it needs to handle in order to survive out in the field without doing annoying things that cause the NVRs to keep loosing the connection etc.

The application should do the following:

1) Connect to the camera using the following protocols using open source rtsp clients available on the web e.g. live555, and as many others that you can find as these are generally the ones NVR use, although not always. The application should then be able to connect to the cameras using the following protocols.

1.  tcp over rtsp (mandatory)
2.  udp over rtsp (optional)
3.  rtsp tunnerled through http (mandatory)
4.  MJPEG over HTTP (mandatory)

IMO, Areconts proprietary protocol shouldn't be needed these days.

2) For the H264 decoders use:

1. Intel IPP H.264 decoders
2. Intel media SDK hardware accelerated decoders
3. Optionally NVIDIAs GPU base decoders

In my experience the h264 emitted from at least some Arecont cameras cause problems with all three. VMS companies generally do not write their own decoders, they are most likely to use the Intel decoders.

3) Test all decoders on the full range of Intel CPUs, from Celerons to Cannonlake. The Intel decoders load different code depending on the CPU type. I have seen h264 emitted from Arecont cameras crash the Intel decoders on Pentiums (really really annoying). On generation 3 i7s it causes the decoders to throw memory exceptions that are at least caught. I do not think they crash in this case, but you still loose frames.

4) Test using different combinations of h264 configuration parameters. e.g. the crash I mentioned in (3) was somewhat eliminated if I force the cameras to use constant bitrate h.264 instead of variable when the VMS connects.

5) At an absolute minimum, simultaneously create a tcp connection for each imager and each stream. So if it is a x4 imager and supports dual streaming there will be 8 connections. Do this while having at least 2 web pages open, and make sure the cameras can handle this without dropping connection and crashing. This is at a minimum. The Arecont 8185DN that I have can't handle this, it reboots, looses connections and crashes all over the place. Might be stable for 2 minutes, and then drops connections again. By comparison, some Axis cameras can handle 20 concurrent tcp connections for example. 

6) Test this on a network, or simulated network that is noisy, has large packet loss etc, and at the same time send corrupt data to the cameras, drop connections unexpectedly etc. 

7) when the test application sends tcp packets to the camera, break them up with small delays. e.g. if the camera is expecting 100 bytes, send 10 and wait a bit, send 70 and wait a bit then send the remaining 20. When receiving a reply from the camera do the same. Call the socket receive call and retrieve 10 bytes and wait a bit, then get the rest...

8) Insert random timing delays between receiving frames, and parts of frames from the camera. This puts memory pressure on the camera, many don't handle this well. Make sure in such cases that the camera can throttle down, e.g. drop p frames until the next I frame etc, drop the quality etc. I have seen even some axis cameras slow down and freeze under this test.

9) Create and drop 1000s of connection to the cameras over a period of time to test for memory leaks. There are undoubtly cameras out there (not talking about arecont in this case) that will slow down until they eventually freeze or reboot. 

9) Run the application for 1 week and make sure there are no reboots at all, no memory leaks and an acceptable number of unaccountable lost connections. 

If the camera can not handle this test, keep improving the software, and if necessary the hardware (i.e. processing power and RAM) until it does work. 

NOTICE: This comment was moved from an existing discussion: Arecont Multi Imagers Dropping Communication To Servers