Analytics vs Rain: Axis, Bosch & Sony

By: Ben Wood, Published on Oct 21, 2013

Rain. This was a key objection to our Camera Analytics Test: Axis vs. Bosch vs. Sony. Members rightfully noted it is one thing for an analytics to do well indoors and outdoors at night but what about in the rain?

In this report, we share our findings of testing the same camera analytic in moderate to heavy rain. Here's what the day looked like:

We set up crosslines / tripwires from 20' to 200' away from our cameras.

Additionally, we tested a crossline perpendicular to the camera from 20' to 200' away.

Here are our key findings from this test:

  • No false positives were generated due to either rain (moderate-to-heavy) or droplets falling from camera housings.
  • Cross line performance was slightly reduced at long ranges (at and over ~150') in rain, when compared to the results from our Camera Video Analytics shootout
  • With a human subject, the Bosch NBN-733V's IVA and Axis Crossline Detection started to have missed activations at 150' and completely missed activations at 200'. With no rain conditions, the true alert rate was 100% until 150' and 200' for Axis Crossline and Bosch IVA, respectively. Detection success rates of Sony SNC-VB600's VMF continued to be low even at close ranges (70% at 20', 40% at 45' and 0% at 100') consistent with the no-rain results. 
  • When the object was a vehicle, no major differences in the detection performance of each camera were observed, compared to the test in no-rain condition.  

Recommendations

In addition to our overall recommendations of the previous report, these tests show that rain and other weather conditions will also be an important factor to consider when designing a reliable cross-line detection system. The maximum detection distance will be reduced in rain and it should be taken into account, together with other factors, such as mounting location, FOV and lighting conditions. Particular attention is needed when the target object is crossing an imaginary tripwire at long distances over 100-150' away.

Get Notified of Video Surveillance Breaking News
Get Notified of Video Surveillance Breaking News

However, false positive / alerts, while understandably feared, did not demonstrate increased problems due to the rain.

Test

We tested the cross line analytics in two common outdoor scenarios:

Tripwire

First, we began with setting a virtual tripwire along our parking lot/driveway, as shown in the overhead view below. 

Our subject crossed the tripwire 10 repetitive times at various distances from 20' to 250' away from the camera.

At 20', both Axis P3354 and Bosch NBN-733V detected with full accuracy, while Sony SNC-VB600's success rate was 7/10.

At distances between 20' and 75', success rate of VB600 decreased gradually and it missed all activations at 100'. P3354 and NBN-733V continued to perform reliably at this range.

At 150', both the P3354 and NBN-733V started to miss some activations. This is lower performance from both when compared to their accuracy at this range in our previous no rain test.

At 200', none of the cameras triggered at all. In clear conditions, the NBN-733V was found to be 100% accurate 200'.

Detection success rates of each camera from the virtual tripwire tests have been tabulated below. Check marks mean 100% success rate, while an X denotes full failure to detect. Since we have not observed any false alerts in this test, success rate is based on true alerts in the summary table.

Vehicle on Driveway

We also tested the accuracy of the crossline analytics of three cameras, targeting vehicle detection at three distances; 25', 100' and 250'. This scene was similar to the tripwire test, but lines were set up on the driveway, across the FOV.

Neither the Axis P3354 or Bosch NBN-733V missed any of the vehicle activations at any of the three distances. The Sony VB600, on the other hand, had missed activations even at the closest test range of 25', and gradually more at longer distances. These results were similar to what we saw in the no rain test. Therefore, we have not observed any effect of the rain in the detection range of bigger objects, such as vehicles, until 250'.


At ~100' Sony's accuracy drops to ~50%.

Finally, at 250', the SNC-VB600 is unable to detect our vehicle at all.


The table below summarizes overall performance of vehicles in the rain:

 

1 report cite this report:

Axis vs Sony Super Size Shootout (Q1659 vs SNC-VB770) on Sep 11, 2017
Super low light, super sized sensor cameras are a growing trend. In the past year, 2 of the most notable entrants for these IP cameras have been...
Comments (7) : PRO Members only. Login. or Join.

Related Reports

Bosch AI Camera Trainer Released And Tested on Apr 09, 2019
Bosch is releasing a highly unusual new AI feature - 'Camera Trainer'. Now, coming as a standard feature in Bosch IVA/EVA analytics, one can train...
Hikvision DeepinMind 2019 Test on Jun 06, 2019
In 2018, Hikvision's DeepinMind AI NVR performed terribly, recognizing vehicles, animals, and other objects as humans, misclassifying demographics,...
Wyze AI Analytics Tested - Beats Axis and Hikvision, Now Improved, Now Being Terminated on Nov 25, 2019
When we originally tested Wyze's free person detection deep learning analytics in July 2019, they performed well, with few false alarms, but a long...
Dahua Analytics+ Tested on Aug 07, 2019
Dahua's analytics have performed poorly in past shootouts. But now, they claim their new Analytics+ "algorithms significantly improve accuracy and...
Yi Home Camera 3 AI Analytics Tested on Sep 10, 2019
Yi Technology is claiming "new AI features" in its $50 Home Camera 3 "eliminates 'false positives' caused by flying insects, small pets, or light...
Vivotek "Neural Network-Powered Detection Engine" Analytics Tested on Sep 17, 2019
Vivotek has released "a neural network-powered detection engine", named Smart Motion Detection, claiming that "swaying vegetation, vehicles passing...
Axis Perimeter Defender Improves, Yet Worse Than Dahua and Wyze on Sep 19, 2019
While Axis Perimeter Defender analytics improved from our 2018 testing, the market has improved much faster, with much less expensive offerings...
Consumer IP Camera Analytics / AI Shootout - Arlo, Google / Nest, Amazon / Ring, Hikvision / Ezviz, Wyze Cam, Yi Home on Sep 26, 2019
AI analytics are hitting the mainstream in the consumer camera market, with entrants Wyze and Yi Home releasing free people detection on their...
Camect "Worlds Smartest Camera Hub" Tested on Oct 18, 2019
Camect is a Silicon Valley startup that claims the "Smartest AI Object Detection On The Market", detecting not only people and vehicles, but...
Halo Smart Vape Detector Tested on Jan 16, 2020
The Halo Smart Sensor claims to detect vaping, including popular brand Juul and even THC vapes. But how well does it work in real world...

Most Recent Industry Reports

Brivo Business Profile 2020 on Jan 27, 2020
Brivo has been doing cloud access for more than 20 years. Is the 2020s the decade that cloud access becomes the norm? CEO Steve Van Till recently...
Favorite VMS / NVR Manufacturers 2020 on Jan 27, 2020
In 2018, a new winner emerged and a former top choice declined. Now, there is a new #1, a new top 5 finisher and 2 major VMSes in decline. Our...
"Hikvision Football Arena" Lithuania Causes Controversy on Jan 24, 2020
Controversy has arisen in Lithuania over Hikvision becoming a soccer team's top sponsor and gaining naming rights to their arena, with one local MP...
Axis and Genetec Drop IFSEC 2020 on Jan 23, 2020
Two of the best-known video surveillance manufacturers are dropping IFSEC International 2020, joining Milestone who dropped IFSEC in 2019. The...
Multipoint Door Lock Tutorial on Jan 23, 2020
Despite widespread use, locked doors are notoriously weak at stopping entry, and thousands can be misspent on locks that leave doors quite...
Avigilon Shifts Cloud Strategy - Merges Blue and ACC on Jan 23, 2020
Avigilon is shifting its cloud strategy, phasing out its Blue web-managed surveillance platform as a stand-alone brand and merging it with its ACC...
Verkada Paying $100 For Referrals Just To Demo on Jan 22, 2020
Some companies pay for referrals when the referral becomes a customer. Verkada is taking it to the next level - paying $100 referrals fees simply...
Camera Analytics Shootout 2020 - Avigilon, Axis, Bosch, Dahua, Hanwha, Hikvision, Uniview, Vivotek on Jan 22, 2020
Analytics are hot again, thanks to a slew of AI-powered cameras, but whose analytics really work? And how do these new smart cameras compare to top...
Intersec 2020 Final Show Report on Jan 21, 2020
IPVM spent all 3 days at the Intersec 2020 show interviewing various companies and finding key trends. We cover: Middle East Enterprise...
Vehicle & Long Range Access Reader Tutorial on Jan 21, 2020
One of the classic challenges for access control are parking lots and garages, where the user's credential is far from the reader. With modern...