Analytics vs Rain: Axis, Bosch & Sony

By: Ben Wood, Published on Oct 21, 2013

Rain. This was a key objection to our Camera Analytics Test: Axis vs. Bosch vs. Sony. Members rightfully noted it is one thing for an analytics to do well indoors and outdoors at night but what about in the rain?

In this report, we share our findings of testing the same camera analytic in moderate to heavy rain. Here's what the day looked like:

We set up crosslines / tripwires from 20' to 200' away from our cameras.

Additionally, we tested a crossline perpendicular to the camera from 20' to 200' away.

Here are our key findings from this test:

  • No false positives were generated due to either rain (moderate-to-heavy) or droplets falling from camera housings.
  • Cross line performance was slightly reduced at long ranges (at and over ~150') in rain, when compared to the results from our Camera Video Analytics shootout
  • With a human subject, the Bosch NBN-733V's IVA and Axis Crossline Detection started to have missed activations at 150' and completely missed activations at 200'. With no rain conditions, the true alert rate was 100% until 150' and 200' for Axis Crossline and Bosch IVA, respectively. Detection success rates of Sony SNC-VB600's VMF continued to be low even at close ranges (70% at 20', 40% at 45' and 0% at 100') consistent with the no-rain results. 
  • When the object was a vehicle, no major differences in the detection performance of each camera were observed, compared to the test in no-rain condition.  

Recommendations

In addition to our overall recommendations of the previous report, these tests show that rain and other weather conditions will also be an important factor to consider when designing a reliable cross-line detection system. The maximum detection distance will be reduced in rain and it should be taken into account, together with other factors, such as mounting location, FOV and lighting conditions. Particular attention is needed when the target object is crossing an imaginary tripwire at long distances over 100-150' away.

Get Notified of Video Surveillance Breaking News
Get Notified of Video Surveillance Breaking News

However, false positive / alerts, while understandably feared, did not demonstrate increased problems due to the rain.

Test

We tested the cross line analytics in two common outdoor scenarios:

Tripwire

First, we began with setting a virtual tripwire along our parking lot/driveway, as shown in the overhead view below. 

Our subject crossed the tripwire 10 repetitive times at various distances from 20' to 250' away from the camera.

At 20', both Axis P3354 and Bosch NBN-733V detected with full accuracy, while Sony SNC-VB600's success rate was 7/10.

At distances between 20' and 75', success rate of VB600 decreased gradually and it missed all activations at 100'. P3354 and NBN-733V continued to perform reliably at this range.

At 150', both the P3354 and NBN-733V started to miss some activations. This is lower performance from both when compared to their accuracy at this range in our previous no rain test.

At 200', none of the cameras triggered at all. In clear conditions, the NBN-733V was found to be 100% accurate 200'.

Detection success rates of each camera from the virtual tripwire tests have been tabulated below. Check marks mean 100% success rate, while an X denotes full failure to detect. Since we have not observed any false alerts in this test, success rate is based on true alerts in the summary table.

Vehicle on Driveway

We also tested the accuracy of the crossline analytics of three cameras, targeting vehicle detection at three distances; 25', 100' and 250'. This scene was similar to the tripwire test, but lines were set up on the driveway, across the FOV.

Neither the Axis P3354 or Bosch NBN-733V missed any of the vehicle activations at any of the three distances. The Sony VB600, on the other hand, had missed activations even at the closest test range of 25', and gradually more at longer distances. These results were similar to what we saw in the no rain test. Therefore, we have not observed any effect of the rain in the detection range of bigger objects, such as vehicles, until 250'.


At ~100' Sony's accuracy drops to ~50%.

Finally, at 250', the SNC-VB600 is unable to detect our vehicle at all.


The table below summarizes overall performance of vehicles in the rain:

 

1 report cite this report:

Axis vs Sony Super Size Shootout (Q1659 vs SNC-VB770) on Sep 11, 2017
Super low light, super sized sensor cameras are a growing trend. In the past year, 2 of the most notable entrants for these IP cameras have been...
Comments (7) : Members only. Login. or Join.

Related Reports

Directory of 400+ Fever Camera News Reports Globally on Jun 22, 2020
This global directory tracks 400+ articles about thermal cameras used to detect fevers in response to the coronavirus pandemic. Articles are...
Uniview Heat-Tracker Temperature Screening Series Examined on Apr 22, 2020
Uniview is marketing #UNVagainstCOVID19 with their Heat-Tracker series, including not only thermal camera screening systems, but options for face...
Hikvision Fever Screening Thermal Solutions Examined on Apr 13, 2020
Hikvision is marketing "safer, faster, smarter" with their Fever Screening Thermal Solutions, (renamed last week to Temperature Screening Series),...
Detecting Coronavirus Fevers With Thermal Cameras on Mar 15, 2020
MAY 2020 Update: This post was our early examination of these systems being used in the pandemic. For more up to date coverage, see The Booming...
Latest London Police Facial Recognition Suffers Serious Issues on Feb 24, 2020
On February 20, IPVM visited another live face rec deployment by London police, but this time the system was thwarted by technical problems and...
Avigilon H5A Analytic Cameras Tested on Oct 07, 2019
Avigilon has released its H5A analytic cameras, claiming to "detect more objects with greater accuracy even in crowded scenes." We tested the...
VMS Export Shootout - Avigilon, Dahua, Exacq, Genetec, Hikvision, Milestone on Sep 13, 2018
When crimes, accidents or problems occur, exporting video from one's video surveillance system is critical to proving incidents. But who does it...
UK VSaaS Startup Ocucon on Jul 03, 2018
Decreasing exposure to fraudulent slip-and-fall insurance claims and lawsuits is one of the oldest selling points of video surveillance for retail....
Cold Weather Dome Test on Jan 25, 2018
Cameras frequently cite super low-temperature specs, with -30° or even -40° common. But to see how popular dome models actually handle real world...
Axis Network Radar Tested (D2050-VE) on Dec 19, 2017
Axis is expanding into a new market - radar. The company has released their D2050-VE Network Radar Detector, claiming to minimize false alarms, in...

Most Recent Industry Reports

Verkada: "IPVM Should Never Be Your Source of News" on Jul 02, 2020
Verkada was unhappy with IPVM's recent coverage declaring that reading IPVM is 'not a good look' and that 'IPVM should never be your source of...
Vintra Presents FulcrumAI Face Recognition on Jul 02, 2020
Vintra presented its FulcrumAI face recognition and mask detection offering at the May 2020 IPVM Startups show. Inside this report: A...
Uniview Wrist Temperature Reader Tested on Jul 02, 2020
Uniview is promoting measuring wrist temperatures whereas most others are just offering forehead or inner canthus measurements. But how well does...
Dahua USA Admits Thermal Solutions "Qualify As Medical Devices" on Jul 02, 2020
Dahua USA has issued a press release admitting a controversial point in the industry but an obvious one to the US FDA, that the thermal temperature...
Access Control Online Show - July 2020 - With 40+ Manufacturers - Register Now on Jul 01, 2020
IPVM is excited to announce our July 2020 Access Control Show. With 40+ companies presenting across 4 days, this is a unique opportunity to hear...
Hanwha Face Mask Detection Tested on Jul 01, 2020
Face mask detection or, more specifically lack-of-face-mask detection, is an expanding offering in the midst of coronavirus. Hanwha in partnership...
UK Government Says Fever Cameras "Unsuitable" on Jul 01, 2020
The UK government's medical device regulator, MHRA, told IPVM that fever-seeking thermal cameras are "unsuitable for this purpose" and recommends...
Camera Course Summer 2020 on Jun 30, 2020
This is the only independent surveillance camera course, based on in-depth product and technology testing. Lots of manufacturer training...
Worst Over But Integrators Still Dealing With Coronavirus Problems (June Statistics) on Jun 30, 2020
While numbers of integrators very impacted by Coronavirus continue to drop, most are still moderately dealing with the pandemic's problems, June...
FLIR Screen-EST Screening Software Tested on Jun 30, 2020
In our FLIR A Series Test, the cameras' biggest drawback was their lack of face detection, requiring manual adjustment when screening each...