Analytics vs Rain: Axis, Bosch & Sony

By: Ben Wood, Published on Oct 21, 2013

Rain. This was a key objection to our Camera Analytics Test: Axis vs. Bosch vs. Sony. Members rightfully noted it is one thing for an analytics to do well indoors and outdoors at night but what about in the rain?

In this report, we share our findings of testing the same camera analytic in moderate to heavy rain. Here's what the day looked like:

We set up crosslines / tripwires from 20' to 200' away from our cameras.

Additionally, we tested a crossline perpendicular to the camera from 20' to 200' away.

Here are our key findings from this test:

  • No false positives were generated due to either rain (moderate-to-heavy) or droplets falling from camera housings.
  • Cross line performance was slightly reduced at long ranges (at and over ~150') in rain, when compared to the results from our Camera Video Analytics shootout
  • With a human subject, the Bosch NBN-733V's IVA and Axis Crossline Detection started to have missed activations at 150' and completely missed activations at 200'. With no rain conditions, the true alert rate was 100% until 150' and 200' for Axis Crossline and Bosch IVA, respectively. Detection success rates of Sony SNC-VB600's VMF continued to be low even at close ranges (70% at 20', 40% at 45' and 0% at 100') consistent with the no-rain results. 
  • When the object was a vehicle, no major differences in the detection performance of each camera were observed, compared to the test in no-rain condition.  

Recommendations

In addition to our overall recommendations of the previous report, these tests show that rain and other weather conditions will also be an important factor to consider when designing a reliable cross-line detection system. The maximum detection distance will be reduced in rain and it should be taken into account, together with other factors, such as mounting location, FOV and lighting conditions. Particular attention is needed when the target object is crossing an imaginary tripwire at long distances over 100-150' away.

Get Notified of Video Surveillance Breaking News
Get Notified of Video Surveillance Breaking News

However, false positive / alerts, while understandably feared, did not demonstrate increased problems due to the rain.

Test

We tested the cross line analytics in two common outdoor scenarios:

Tripwire

First, we began with setting a virtual tripwire along our parking lot/driveway, as shown in the overhead view below. 

Our subject crossed the tripwire 10 repetitive times at various distances from 20' to 250' away from the camera.

At 20', both Axis P3354 and Bosch NBN-733V detected with full accuracy, while Sony SNC-VB600's success rate was 7/10.

At distances between 20' and 75', success rate of VB600 decreased gradually and it missed all activations at 100'. P3354 and NBN-733V continued to perform reliably at this range.

At 150', both the P3354 and NBN-733V started to miss some activations. This is lower performance from both when compared to their accuracy at this range in our previous no rain test.

At 200', none of the cameras triggered at all. In clear conditions, the NBN-733V was found to be 100% accurate 200'.

Detection success rates of each camera from the virtual tripwire tests have been tabulated below. Check marks mean 100% success rate, while an X denotes full failure to detect. Since we have not observed any false alerts in this test, success rate is based on true alerts in the summary table.

Vehicle on Driveway

We also tested the accuracy of the crossline analytics of three cameras, targeting vehicle detection at three distances; 25', 100' and 250'. This scene was similar to the tripwire test, but lines were set up on the driveway, across the FOV.

Neither the Axis P3354 or Bosch NBN-733V missed any of the vehicle activations at any of the three distances. The Sony VB600, on the other hand, had missed activations even at the closest test range of 25', and gradually more at longer distances. These results were similar to what we saw in the no rain test. Therefore, we have not observed any effect of the rain in the detection range of bigger objects, such as vehicles, until 250'.


At ~100' Sony's accuracy drops to ~50%.

Finally, at 250', the SNC-VB600 is unable to detect our vehicle at all.


The table below summarizes overall performance of vehicles in the rain:

 

1 report cite this report:

Axis vs Sony Super Size Shootout (Q1659 vs SNC-VB770) on Sep 11, 2017
Super low light, super sized sensor cameras are a growing trend. In the past year, 2 of the most notable entrants for these IP cameras have been...
Comments (7) : PRO Members only. Login. or Join.

Related Reports on Video Analytics

RealNetworks SAFR Facial Recognition Profile on Sep 25, 2019
RealNetworks entered the surveillance market by giving away their analytics to schools for free, and is now targeting large commercial users with...
Hikvision Acusense Analytics Tested on Sep 23, 2019
Hikvision touts "The Magic Behind It All" in their new Acusense line are 'deep learning algorithms' inside these cameras and recorders. But how...
Directory of 70 Video Surveillance Startups on Sep 18, 2019
This directory provides a list of video surveillance startups to help you see and research what companies are new or not yet broadly known. 2019...
Vivotek "Neural Network-Powered Detection Engine" Analytics Tested on Sep 17, 2019
Vivotek has released "a neural network-powered detection engine", named Smart Motion Detection, claiming that "swaying vegetation, vehicles passing...
AI Video Surveillance (Finally) Goes Mainstream In 2020 on Sep 03, 2019
While video surveillance analytics has been promoted, hyped and lamented for nearly 20 years, next year, 2020, will be the year that it finally...
Scylla AI Video Analytics Company Profile on Aug 29, 2019
Scylla, an AI analytics startup, says they are targeting 1 Billion dollar valuation in 5 years and it "is not rocket science" to detect weapons and...
Anyvision Facial Recognition Tested on Aug 21, 2019
Anyvision is aiming for $1 billion in revenue by 2022, backed by $74 million in funding. But does their performance live up to the hype they have...
Verkada People And Face Analytics Tested on Aug 16, 2019
This week, Verkada released "People Analytics", including face analytics that they describe is a "game-changing feature" that "pushes the...
Dahua Analytics+ Tested on Aug 07, 2019
Dahua's analytics have performed poorly in past shootouts. But now, they claim their new Analytics+ "algorithms significantly improve accuracy and...
Honeywell Speaks On NDAA Ban, New Non-Banned Cameras and Cybersecurity on Aug 06, 2019
For years, Honeywell has depended on Dahua, a company with a poor cybersecurity track record and now banned by the US NDAA, for the development and...

Most Recent Industry Reports

Resideo Stock Plunges 40%, CFO Ousted on Oct 23, 2019
The horrible year for the ADI / Honeywell Home spinout, Resideo, just got worse, with their stock plunging another 40% today. Not even a year...
Access Control Door Controllers Guide on Oct 22, 2019
Door controllers are at the center of physical access control systems connecting software, readers, and locks. Despite being buried inside...
Alarm.com Acquires OpenEye on Oct 21, 2019
Alarm.com is targeting commercial expansion and now they have a commercial cloud VMS with the acquisition of OpenEye. In this note, based on...
Government-Owned Hikvision Wants To Keep Politics Out Of Security on Oct 21, 2019
'Politics' made Hikvision the goliath it is today. It was PRC China 'politics' that created Hikvision, funded it, and blocked its foreign...
Integrated IR Camera Usage Statistics 2019 on Oct 21, 2019
Virtually every IP camera now comes with integrated IR but how many actually make use of IR or choose 'super' low light cameras without IR? In...
Alarm Veteran "Demands A Criminal Investigation" Of UL on Oct 18, 2019
The Interceptor's Project pressure against UL continues to rise. Following Keith Jentoft's allegation that "UL Has Blood On Their Hands", Jentoft...
Camect "Worlds Smartest Camera Hub" Tested on Oct 18, 2019
Camect is a Silicon Valley startup that claims the "Smartest AI Object Detection On The Market", detecting not only people and vehicles, but...
Hikvision Global News Reports Directory on Oct 17, 2019
Hikvision has received the most global news reporting of any video surveillance company, ever, ranging from the WSJ, the Financial Times, Reuters,...
Camera Calculator V3.1 Release Improves User Experience on Oct 17, 2019
IPVM has released a new version of our Camera Calculator, V3.1, with significant user experience improvements, a new development plan, and an...