Analytics vs Rain: Axis, Bosch & Sony

Author: Ben Wood, Published on Oct 21, 2013

Rain. This was a key objection to our Camera Analytics Test: Axis vs. Bosch vs. Sony. Members rightfully noted it is one thing for an analytics to do well indoors and outdoors at night but what about in the rain?

In this report, we share our findings of testing the same camera analytic in moderate to heavy rain. Here's what the day looked like:

We set up crosslines / tripwires from 20' to 200' away from our cameras.

Additionally, we tested a crossline perpendicular to the camera from 20' to 200' away.

Inside, we share our test results of who performed best and worst, plus how much the rain impacted performance.

****. **** *** * *** ********* ** ********* ********* ****: **** **. ***** **. ****. ******* ********** ***** ** ** *** ***** *** ** analytics ** ** **** ******* *** ******** ** ***** *** what ***** ** *** ****?

** **** ******, ** ***** *** ******** ** ******* *** same ****** ******** ** ******** ** ***** ****. ****'* **** the *** ****** ****:

** *** ************/ ********* **** **' ** ***' **** **** *** *******.

************, ** ****** * ********* ************* ** *** ****** **** **' ** ***' ****.

******, ** ***** *** **** ******* ** *** ********* **** and *****, **** *** **** *** **** ******** ***********.

[***************]

Key ********

**** *** *** *** ******** **** **** ****:

  • ** ***** ********* **** ********* *** ** ****** **** (********-**-*****)** ******** ******* **** ****** ********.
  • ***** **** *********** *** ******** ******* ** **** ****** (** and **** ~***') ** ****, **** ******** ** *** ******* from *** ****** ***** ********* ********
  • **** * ***** *******, *** ***** ***-****'* *** *** **** Crossline ********* ******* ** **** ****** *********** ** ***' *** ********** missed *********** ** ***'. **** ** **** **********, *** **** alert **** *** ***% ***** ***' *** ***' *** **** ********* *** ***** IVA, ************. ********* ******* ***** ** **** ***-*****'* *** ********* to ** *** **** ** ***** ****** (**% ** **', 40% ** **' *** *% ** ***') ********** **** *** no-rain *******. 
  • **** *** ****** *** * *******, ** ***** *********** ** the ********* *********** ** **** ****** **** ********, ******** ** the **** ** **-**** *********.  

***************

** ******** ** *** ******* *************** ** *** ******** ******, these ***** **** **** **** *** ***** ******* ********** **** also ** ** ********* ****** ** ******** **** ********* * reliable *****-**** ********* ******. *** ******* ********* ******** **** ** reduced ** ******* ** ****** ** ***** **** *******, ******** **** ***** factors, **** ** ******** ********, *** *** ******** **********. ********** attention ** ****** **** *** ****** ****** ** ******** ** imaginary ******** ** **** ********* **** ***-***' ****.

*******, ***** ******** / ******, ***** ************** ******, *** *** demonstrate ********* ******** *** ** *** ****.

****

** ****** *** ***** **** ********* ** *** ****** ******* scenarios:

********

*****, ** ***** **** ******* * ******* ******** ***** *** parking ***/********, ** ***** ** *** ******** **** *****. 

*** ******* ******* *** ******** ** ********** ***** ** ******* distances **** **' ** ***' **** **** *** ******.

** **', **** **** ***** *** ***** ***-**** ******** **** full ********, ***** **** ***-*****'* ******* **** *** */**.

** ********* ******* **' *** **', ******* **** ** ***** decreased ************ ** ****** *** *********** ** ***'. ***** *** ***-**** continued ** ******* ******** ** **** *****.

** ***', **** *** ***** *** ***-**** ******* ** **** **** activations. **** ** ***** *********** **** **** **** ******** ** their ******** ** **** ***** ** *** ******** ** **** test.

** ***', **** ** *** ******* ********* ** ***. ** clear **********, *** ***-**** *** ***** ** ** ***% ******** 200'.

********* ******* ***** ** **** ****** **** *** ******* ******** tests **** **** ********* *****. ***** ***** **** ***% ******* rate, ******** ******* **** ******* ** ******. ***** ** **** *** observed *** ***** ****** ** **** ****, ******* **** ** based ** **** ****** ** *** ******* *****.

Vehicle ** ********

** **** ****** *** ******** ** ************ ********* ** ***** *******, ********* ******* ********* ** ***** *********; 25', ***' *** ***'. **** ***** *** ******* ** *** tripwire ****, *** ***** **** *** ** ** *** ********, across *** ***.

******* *** **** ***** ** ***** ***-**** ****** *** ** the ******* *********** ** *** ** *** ***** *********. *** Sony *****, ** *** ***** ****, *** ****** *********** **** at *** ******* **** ***** ** **', *** ********* **** at ****** *********. ***** ******* **** ******* ** **** ** saw ** *** ** **** ****. *********, ** **** *** observed *** ****** ** *** **** ** *** ********* ***** of ****** *******, **** ** ********, ***** ***'.


** ~***' ****'* ******** ***** ** ~**%.

*******, ** ***', *** ***-***** ** ****** ** ****** *** vehicle ** ***.


*** ***** ***** ********** ******* *********** ** ******** ** *** rain:

 

Comments (7)

Figen:

Thanks for the report. Over how many hours was the data collected? I have found that over extended periods of rain >12 hours (It rains a lot in Vancouver) you should get some false positives. When calculating detecting true events accuracy, how many true events were done per test?

Also, all of the testing was using crossing line detection, which I never use. I have a belief about crossing line detection as not being as good as "appearing in ROI". Early on I was finding that trespass by people would be missed because often fencing or hoarding would have storage containers/ obstructions beside them and entry might be obscured and crossing line detection would miss them, while appearing in ROI would detect them. Any thoughts on this?

Hi Robert, we tested over about 5 hours. Results were consistent all the way through, so we felt it was long enough to see patterns. However, we do believe there could be performance differences depending on the type of rain, depending on how heavy it is, how windy, potential for hail, etc.

We typically ran 10 attempts with a human subject at each distance. Vehicles, not as many, because performance was pretty close to predictable based on the first 2-3, due to the much larger object size.

This testing was limited to cross line detection for two reasons. First and foremost, because it is the most common analytic in use in the industry. Second, this is a continuation of our previous camera analytics test, which was limited to crossline.

We will be testing other analytics in the future (starting today actually), and will be using other rules in various conditions.

I did a major analytics deployment some years ago, and in the rain we had signficant issues with directional detection in the rain, especially in evening and dark. Headlight reflections on wet pavement on opposite side of the road on a curve made the analytics think there was movement in the wrong direction consistently. On an interstate there were hundreds of alarms each morning.

Doing tripline for our first effort was a good idea, but I'll bet other functions might be more adversely affected by rain and other weather conditions. I'm sure you will do follow-on studies of more advanced features.

Too bad you didn't test that other ananlytic that would have "learned" that rain is normal...

Hi Marc, we tested virtual tripwires as a continuation of our Camera Analytics Tested: Axis vs. Bosch vs. Sony report. We will be testing AgentVi, VideoIQ, and Bosch IVA in upcoming reports, so we'll be looking at other rules in varying conditions then. It is, unfortunately, very difficult to plan rain testing in Pennsylvania, as we've had one day of steady rain in the past month. But we're getting there!

What, you don't just look at your iPhone or Android app for weather until it says "rain?"

Sounds good. I would be very interested to see performance comparisons of integrated vs. add-on analytics, edge vs. server based, and the like. I've always thought that edge should be better, but of course the algorithms are not identical either.

I had some good success with AgentVI back when they were still Aspectus. I hope you will also include Sightlogix and Puretech, both of whom include geolocation which sometimes is a good adjunct and sometimes a good alternate for radar.

My initial thought was that the rain droplets on the lens would be the issue but according to this test the reduction in analytics performance looked more like the rain itself caused visibilty to be reduced which then reduced analytics performance. This tells me that fog would have the same effect.

Which brings up another question ... what is the effect on the analytics with snow? Would slow falling snow flakes near the camera be detected?

I would think the snow flake question depends on the size and speed of the flake. We'll be testing this when winter comes, assuming we actually get snow this year.

I believe that snow will have more of an obscuration effect than rain, similar to fog, which will have more of an impact on performance that the chance of false alarms from falling snow.

Login to read this IPVM report.
Why do I need to log in?
IPVM conducts unique testing and research funded by member's payments enabling us to offer the most independent, accurate and in-depth information.

Related Reports on Video Analytics

Beware Amazon Go Store Hype (Tested) on Nov 13, 2018
IPVM's trip to and testing of Amazon Go's San Francisco store shows a number of significant operational and economic issues that undermine the...
Genetec Privacy Protector Tested on Nov 12, 2018
Genetec has built Kiwi Security's Privacy Protector into Security Center, an analytic which anonymizes individuals in cameras' fields of view...
Kogniz Silicon Valley AI Startup Profile on Nov 07, 2018
Kogniz is a Silicon Valley company that aims to bring AI analytics to security and surveillance, centering on their own smart cameras: We spoke...
Avigilon Opens Up Analytics And Cameras on Nov 06, 2018
Avigilon is opening up. The company historically famous for advocating its own end-to-end solutions and making it harder for 3rd parties to...
Huawei Admits AI "Bubble" on Oct 16, 2018
A fascinating article from the Chinese government's Global Times: Huawei’s AI ambition to reshape industries. While the Global Times talks about...
Dahua Face Recognition Camera Tested on Oct 15, 2018
Dahua has been one of the industry's most vocal proponents of the value that AI creates: As part of this, Dahua has released a facial...
IACP 2018 Police Show Final Report on Oct 08, 2018
IPVM went to Orlando to cover the 2018 IACP conference, the country's largest police show (about as big as ASIS), examining the 700+...
Avigilon Announces AI-Powered H5 Camera Development on Sep 19, 2018
Avigilon will be showcasing "next-generation AI" at next week's ASIS GSX. In an atypical move, the company is not actually releasing these...
VMS Export Shootout - Avigilon, Dahua, Exacq, Genetec, Hikvision, Milestone on Sep 13, 2018
When crimes, accidents or problems occur, exporting video from one's video surveillance system is critical to proving incidents. But who does it...
US DARPA Investing $2 Billion In AI on Sep 11, 2018
The US Defense Advanced Research Projects Agency (DARPA) is granting more than $2 Billion to companies developing new AI technologies. The money...

Most Recent Industry Reports

Throughtek P2P/Cloud Solution Profile on Nov 15, 2018
Many IoT manufacturers either do not have the capabilities or the interest to develop their own cloud management software for their devices....
ASIS Offering Custom Research For Manufacturers on Nov 15, 2018
Manufacturers often want to know what industry people think about trends and, in particular, the segments and product they offer.  ASIS and its...
ISC East 2018 Mini-Show Report on Nov 15, 2018
ISC East, by its own admission, is not a national or international show, billed as the "Largest Annual Northeast U.S. Security...
Hikvision Silent on "Bad Architectural Practices" Cybersecurity Report on Nov 14, 2018
A 'significant vulnerability was found in Hikvision cameras' by VDOO, a startup cybersecurity specialist. Hikvision has fixed the specific...
French Government Threatens School with $1.7M Fine For “Excessive Video Surveillance” on Nov 14, 2018
The French government has notified a high-profile Paris coding academy that it risks a fine of up to 1.5 million euros (about $1.7m) if it...
Integrator Credit Card Alternative Divvy on Nov 13, 2018
Most security integrators are small businesses but large enough that they have various employees that need to be able to expense various charges as...
Directory of Video Intercoms on Nov 13, 2018
Video Intercoms, also known as Video Door-Phones or Video Entry Systems, have been growing in the past decade as more and more IP camera...
Beware Amazon Go Store Hype (Tested) on Nov 13, 2018
IPVM's trip to and testing of Amazon Go's San Francisco store shows a number of significant operational and economic issues that undermine the...
Magos Radar Company Profile on Nov 12, 2018
Magos America General Manager Yaron Zussman admits when he first came across Magos, he asked himself: "What's innovative about radar?" Be that as...
Genetec Privacy Protector Tested on Nov 12, 2018
Genetec has built Kiwi Security's Privacy Protector into Security Center, an analytic which anonymizes individuals in cameras' fields of view...

The world's leading video surveillance information source, IPVM provides the best reporting, testing and training for 10,000+ members globally. Dedicated to independent and objective information, we uniquely refuse any and all advertisements, sponsorship and consulting from manufacturers.

About | FAQ | Contact