Analytics vs Rain: Axis, Bosch & Sony

Author: Ben Wood, Published on Oct 21, 2013

Rain. This was a key objection to our Camera Analytics Test: Axis vs. Bosch vs. Sony. Members rightfully noted it is one thing for an analytics to do well indoors and outdoors at night but what about in the rain?

In this report, we share our findings of testing the same camera analytic in moderate to heavy rain. Here's what the day looked like:

We set up crosslines / tripwires from 20' to 200' away from our cameras.

Additionally, we tested a crossline perpendicular to the camera from 20' to 200' away.

Inside, we share our test results of who performed best and worst, plus how much the rain impacted performance.

****. **** *** * *** ********* ** ********* ********* ****: **** **. ***** **. ****. ******* ********** ***** ** ** *** ***** *** ** analytics ** ** **** ******* *** ******** ** ***** *** what ***** ** *** ****?

** **** ******, ** ***** *** ******** ** ******* *** same ****** ******** ** ******** ** ***** ****. ****'* **** the *** ****** ****:

** *** ************/ ********* **** **' ** ***' **** **** *** *******.

************, ** ****** * ********* ************* ** *** ****** **** **' ** ***' ****.

******, ** ***** *** **** ******* ** *** ********* **** and *****, **** *** **** *** **** ******** ***********.

[***************]

Key ********

**** *** *** *** ******** **** **** ****:

  • ** ***** ********* **** ********* *** ** ****** **** (********-**-*****)** ******** ******* **** ****** ********.
  • ***** **** *********** *** ******** ******* ** **** ****** (** and **** ~***') ** ****, **** ******** ** *** ******* from *** ****** ***** ********* ********
  • **** * ***** *******, *** ***** ***-****'* *** *** **** Crossline ********* ******* ** **** ****** *********** ** ***' *** ********** missed *********** ** ***'. **** ** **** **********, *** **** alert **** *** ***% ***** ***' *** ***' *** **** ********* *** ***** IVA, ************. ********* ******* ***** ** **** ***-*****'* *** ********* to ** *** **** ** ***** ****** (**% ** **', 40% ** **' *** *% ** ***') ********** **** *** no-rain *******. 
  • **** *** ****** *** * *******, ** ***** *********** ** the ********* *********** ** **** ****** **** ********, ******** ** the **** ** **-**** *********.  

***************

** ******** ** *** ******* *************** ** *** ******** ******, these ***** **** **** **** *** ***** ******* ********** **** also ** ** ********* ****** ** ******** **** ********* * reliable *****-**** ********* ******. *** ******* ********* ******** **** ** reduced ** ******* ** ****** ** ***** **** *******, ******** **** ***** factors, **** ** ******** ********, *** *** ******** **********. ********** attention ** ****** **** *** ****** ****** ** ******** ** imaginary ******** ** **** ********* **** ***-***' ****.

*******, ***** ******** / ******, ***** ************** ******, *** *** demonstrate ********* ******** *** ** *** ****.

****

** ****** *** ***** **** ********* ** *** ****** ******* scenarios:

********

*****, ** ***** **** ******* * ******* ******** ***** *** parking ***/********, ** ***** ** *** ******** **** *****. 

*** ******* ******* *** ******** ** ********** ***** ** ******* distances **** **' ** ***' **** **** *** ******.

** **', **** **** ***** *** ***** ***-**** ******** **** full ********, ***** **** ***-*****'* ******* **** *** */**.

** ********* ******* **' *** **', ******* **** ** ***** decreased ************ ** ****** *** *********** ** ***'. ***** *** ***-**** continued ** ******* ******** ** **** *****.

** ***', **** *** ***** *** ***-**** ******* ** **** **** activations. **** ** ***** *********** **** **** **** ******** ** their ******** ** **** ***** ** *** ******** ** **** test.

** ***', **** ** *** ******* ********* ** ***. ** clear **********, *** ***-**** *** ***** ** ** ***% ******** 200'.

********* ******* ***** ** **** ****** **** *** ******* ******** tests **** **** ********* *****. ***** ***** **** ***% ******* rate, ******** ******* **** ******* ** ******. ***** ** **** *** observed *** ***** ****** ** **** ****, ******* **** ** based ** **** ****** ** *** ******* *****.

Vehicle ** ********

** **** ****** *** ******** ** ************ ********* ** ***** *******, ********* ******* ********* ** ***** *********; 25', ***' *** ***'. **** ***** *** ******* ** *** tripwire ****, *** ***** **** *** ** ** *** ********, across *** ***.

******* *** **** ***** ** ***** ***-**** ****** *** ** the ******* *********** ** *** ** *** ***** *********. *** Sony *****, ** *** ***** ****, *** ****** *********** **** at *** ******* **** ***** ** **', *** ********* **** at ****** *********. ***** ******* **** ******* ** **** ** saw ** *** ** **** ****. *********, ** **** *** observed *** ****** ** *** **** ** *** ********* ***** of ****** *******, **** ** ********, ***** ***'.


** ~***' ****'* ******** ***** ** ~**%.

*******, ** ***', *** ***-***** ** ****** ** ****** *** vehicle ** ***.


*** ***** ***** ********** ******* *********** ** ******** ** *** rain:

 

Comments (7)

Figen:

Thanks for the report. Over how many hours was the data collected? I have found that over extended periods of rain >12 hours (It rains a lot in Vancouver) you should get some false positives. When calculating detecting true events accuracy, how many true events were done per test?

Also, all of the testing was using crossing line detection, which I never use. I have a belief about crossing line detection as not being as good as "appearing in ROI". Early on I was finding that trespass by people would be missed because often fencing or hoarding would have storage containers/ obstructions beside them and entry might be obscured and crossing line detection would miss them, while appearing in ROI would detect them. Any thoughts on this?

Hi Robert, we tested over about 5 hours. Results were consistent all the way through, so we felt it was long enough to see patterns. However, we do believe there could be performance differences depending on the type of rain, depending on how heavy it is, how windy, potential for hail, etc.

We typically ran 10 attempts with a human subject at each distance. Vehicles, not as many, because performance was pretty close to predictable based on the first 2-3, due to the much larger object size.

This testing was limited to cross line detection for two reasons. First and foremost, because it is the most common analytic in use in the industry. Second, this is a continuation of our previous camera analytics test, which was limited to crossline.

We will be testing other analytics in the future (starting today actually), and will be using other rules in various conditions.

I did a major analytics deployment some years ago, and in the rain we had signficant issues with directional detection in the rain, especially in evening and dark. Headlight reflections on wet pavement on opposite side of the road on a curve made the analytics think there was movement in the wrong direction consistently. On an interstate there were hundreds of alarms each morning.

Doing tripline for our first effort was a good idea, but I'll bet other functions might be more adversely affected by rain and other weather conditions. I'm sure you will do follow-on studies of more advanced features.

Too bad you didn't test that other ananlytic that would have "learned" that rain is normal...

Hi Marc, we tested virtual tripwires as a continuation of our Camera Analytics Tested: Axis vs. Bosch vs. Sony report. We will be testing AgentVi, VideoIQ, and Bosch IVA in upcoming reports, so we'll be looking at other rules in varying conditions then. It is, unfortunately, very difficult to plan rain testing in Pennsylvania, as we've had one day of steady rain in the past month. But we're getting there!

What, you don't just look at your iPhone or Android app for weather until it says "rain?"

Sounds good. I would be very interested to see performance comparisons of integrated vs. add-on analytics, edge vs. server based, and the like. I've always thought that edge should be better, but of course the algorithms are not identical either.

I had some good success with AgentVI back when they were still Aspectus. I hope you will also include Sightlogix and Puretech, both of whom include geolocation which sometimes is a good adjunct and sometimes a good alternate for radar.

My initial thought was that the rain droplets on the lens would be the issue but according to this test the reduction in analytics performance looked more like the rain itself caused visibilty to be reduced which then reduced analytics performance. This tells me that fog would have the same effect.

Which brings up another question ... what is the effect on the analytics with snow? Would slow falling snow flakes near the camera be detected?

I would think the snow flake question depends on the size and speed of the flake. We'll be testing this when winter comes, assuming we actually get snow this year.

I believe that snow will have more of an obscuration effect than rain, similar to fog, which will have more of an impact on performance that the chance of false alarms from falling snow.

Login to read this IPVM report.
Why do I need to log in?
IPVM conducts unique testing and research funded by member's payments enabling us to offer the most independent, accurate and in-depth information.

Related Reports on Video Analytics

Avigilon Announces AI-Powered H5 Camera Development on Sep 19, 2018
Avigilon will be showcasing "next-generation AI" at next week's ASIS GSX. In an atypical move, the company is not actually releasing these...
VMS Export Shootout - Avigilon, Dahua, Exacq, Genetec, Hikvision, Milestone on Sep 13, 2018
When crimes, accidents or problems occur, exporting video from one's video surveillance system is critical to proving incidents. But who does it...
US DARPA Investing $2 Billion In AI on Sep 11, 2018
The US Defense Advanced Research Projects Agency (DARPA) is granting more than $2 Billion to companies developing new AI technologies. The money...
Luxriot VMS Profile on Aug 23, 2018
Luxriot is more popular than Hikvision and Milestone products according to ASMAG which was probably even surprising to Luxriot. The company has...
Video Analytics Integration Guide on Aug 16, 2018
Video analytics is hot again (at least conceptually) but integrating video analytics with VMSes can be challenging. This is especially significant...
ISS VMS / Video Analytics Company Profile on Aug 16, 2018
Who is ISS? In the past few months, they had one of the craziest ISC West promo items in years. Then, they hired industry veteran and ex-Dahua...
IP Camera Analytics Shootout - Avigilon, Axis, Bosch, Dahua, Hanwha, Hikvision, Uniview on Jul 30, 2018
Video analytics are hot again. But whose analytics really work? IPVM bought and tested Avigilon, Axis, Bosch, Dahua, Hanwha and Hikvision cameras,...
Eagle Eye Networks Cloud VMS Tested on Jul 26, 2018
Eagle Eye has become one of the most significant players in the industry in the past few years: Eagle Eye's Owner Acquired Brivo Eagle Eye...
Avigilon "Self-Learning" Analytics And VMD Tested on Jul 23, 2018
Avigilon is a popular choice for video analytics, offering "self-learning" analytics built into their cameras as well as appliances. We tested...
Directory of Video Surveillance Startups on Jul 18, 2018
This directory provides a list of video surveillance startups to help you see and research what companies are new or not yet broadly known entity...

Most Recent Industry Reports

Alexa Guard Expands Amazon's Security Offerings, Boosts ADT's Stock on Sep 21, 2018
Amazon is expanding their security offerings yet again, this time with Alexa Guard that delivers security audio analytics and a virtual "Fake...
UTC, Owner of Lenel, Acquires S2 on Sep 20, 2018
UTC now owns two of the biggest access control providers, one of integrator's most hated access control platforms, Lenel, and one of their...
BluePoint Aims To Bring Life-Safety Mind-Set To Police Pull Stations on Sep 20, 2018
Fire alarm pull stations are commonplace but police ones are not. A self-funded startup, BluePoint Alert Solutions is aiming to make police pull...
SIA Plays Dumb On OEMs And Hikua Ban on Sep 20, 2018
OEMs widely pretend to be 'manufacturers', deceiving their customers and putting them at risk for cybersecurity attacks and, soon, violation of US...
Axis Vs. Hikvision IR PTZ Shootout on Sep 20, 2018
Hikvision has their high-end dual-sensor DarkfighterX. Axis has their high-end concealed IR Q6125-LE. Which is better? We bought both and tested...
Avigilon Announces AI-Powered H5 Camera Development on Sep 19, 2018
Avigilon will be showcasing "next-generation AI" at next week's ASIS GSX. In an atypical move, the company is not actually releasing these...
Favorite Request-to-Exit (RTE) Manufacturers 2018 on Sep 19, 2018
Request To Exit devices like motion sensors and lock releasing push-buttons are a part of almost every access install, but who makes the equipment...
25% China Tariffs Finalized For 2019, 10% Start Now, Includes Select Video Surveillance on Sep 18, 2018
A surprise move: In July, when the most recent tariff round was first announced, the tariffs were only scheduled for 10%. However, now, the US...
Central Stations Face Off Against NFPA On Fire Monitoring on Sep 18, 2018
Central stations are facing off against the NFPA over what they call anti-competitive language in NFPA 72, the standard that covers fire alarms....
Hikvision USA Starts Layoffs on Sep 18, 2018
Hikvision USA has started layoffs, just weeks after the US government ban was passed into law. Inside this note, we examine: The important...

The world's leading video surveillance information source, IPVM provides the best reporting, testing and training for 10,000+ members globally. Dedicated to independent and objective information, we uniquely refuse any and all advertisements, sponsorship and consulting from manufacturers.

About | FAQ | Contact