Analytics vs Rain: Axis, Bosch & Sony

Author: Ben Wood, Published on Oct 21, 2013

Rain. This was a key objection to our Camera Analytics Test: Axis vs. Bosch vs. Sony. Members rightfully noted it is one thing for an analytics to do well indoors and outdoors at night but what about in the rain?

In this report, we share our findings of testing the same camera analytic in moderate to heavy rain. Here's what the day looked like:

We set up crosslines / tripwires from 20' to 200' away from our cameras.

Additionally, we tested a crossline perpendicular to the camera from 20' to 200' away.

Inside, we share our test results of who performed best and worst, plus how much the rain impacted performance.

****. **** *** * *** ********* ** ********* ********* ****: **** **. ***** **. ****. ******* ********** ***** ** ** *** ***** *** ** analytics ** ** **** ******* *** ******** ** ***** *** what ***** ** *** ****?

** **** ******, ** ***** *** ******** ** ******* *** same ****** ******** ** ******** ** ***** ****. ****'* **** the *** ****** ****:

** *** ************/ ********* **** **' ** ***' **** **** *** *******.

************, ** ****** * ********* ************* ** *** ****** **** **' ** ***' ****.

******, ** ***** *** **** ******* ** *** ********* **** and *****, **** *** **** *** **** ******** ***********.

[***************]

Key ********

**** *** *** *** ******** **** **** ****:

  • ** ***** ********* **** ********* *** ** ****** **** (********-**-*****)** ******** ******* **** ****** ********.
  • ***** **** *********** *** ******** ******* ** **** ****** (** and **** ~***') ** ****, **** ******** ** *** ******* from *** ****** ***** ********* ********
  • **** * ***** *******, *** ***** ***-****'* *** *** **** Crossline ********* ******* ** **** ****** *********** ** ***' *** ********** missed *********** ** ***'. **** ** **** **********, *** **** alert **** *** ***% ***** ***' *** ***' *** **** ********* *** ***** IVA, ************. ********* ******* ***** ** **** ***-*****'* *** ********* to ** *** **** ** ***** ****** (**% ** **', 40% ** **' *** *% ** ***') ********** **** *** no-rain *******. 
  • **** *** ****** *** * *******, ** ***** *********** ** the ********* *********** ** **** ****** **** ********, ******** ** the **** ** **-**** *********.  

***************

** ******** ** *** ******* *************** ** *** ******** ******, these ***** **** **** **** *** ***** ******* ********** **** also ** ** ********* ****** ** ******** **** ********* * reliable *****-**** ********* ******. *** ******* ********* ******** **** ** reduced ** ******* ** ****** ** ***** **** *******, ******** **** ***** factors, **** ** ******** ********, *** *** ******** **********. ********** attention ** ****** **** *** ****** ****** ** ******** ** imaginary ******** ** **** ********* **** ***-***' ****.

*******, ***** ******** / ******, ***** ************** ******, *** *** demonstrate ********* ******** *** ** *** ****.

****

** ****** *** ***** **** ********* ** *** ****** ******* scenarios:

********

*****, ** ***** **** ******* * ******* ******** ***** *** parking ***/********, ** ***** ** *** ******** **** *****. 

*** ******* ******* *** ******** ** ********** ***** ** ******* distances **** **' ** ***' **** **** *** ******.

** **', **** **** ***** *** ***** ***-**** ******** **** full ********, ***** **** ***-*****'* ******* **** *** */**.

** ********* ******* **' *** **', ******* **** ** ***** decreased ************ ** ****** *** *********** ** ***'. ***** *** ***-**** continued ** ******* ******** ** **** *****.

** ***', **** *** ***** *** ***-**** ******* ** **** **** activations. **** ** ***** *********** **** **** **** ******** ** their ******** ** **** ***** ** *** ******** ** **** test.

** ***', **** ** *** ******* ********* ** ***. ** clear **********, *** ***-**** *** ***** ** ** ***% ******** 200'.

********* ******* ***** ** **** ****** **** *** ******* ******** tests **** **** ********* *****. ***** ***** **** ***% ******* rate, ******** ******* **** ******* ** ******. ***** ** **** *** observed *** ***** ****** ** **** ****, ******* **** ** based ** **** ****** ** *** ******* *****.

Vehicle ** ********

** **** ****** *** ******** ** ************ ********* ** ***** *******, ********* ******* ********* ** ***** *********; 25', ***' *** ***'. **** ***** *** ******* ** *** tripwire ****, *** ***** **** *** ** ** *** ********, across *** ***.

******* *** **** ***** ** ***** ***-**** ****** *** ** the ******* *********** ** *** ** *** ***** *********. *** Sony *****, ** *** ***** ****, *** ****** *********** **** at *** ******* **** ***** ** **', *** ********* **** at ****** *********. ***** ******* **** ******* ** **** ** saw ** *** ** **** ****. *********, ** **** *** observed *** ****** ** *** **** ** *** ********* ***** of ****** *******, **** ** ********, ***** ***'.


** ~***' ****'* ******** ***** ** ~**%.

*******, ** ***', *** ***-***** ** ****** ** ****** *** vehicle ** ***.


*** ***** ***** ********** ******* *********** ** ******** ** *** rain:

 

Comments (7)

Figen:

Thanks for the report. Over how many hours was the data collected? I have found that over extended periods of rain >12 hours (It rains a lot in Vancouver) you should get some false positives. When calculating detecting true events accuracy, how many true events were done per test?

Also, all of the testing was using crossing line detection, which I never use. I have a belief about crossing line detection as not being as good as "appearing in ROI". Early on I was finding that trespass by people would be missed because often fencing or hoarding would have storage containers/ obstructions beside them and entry might be obscured and crossing line detection would miss them, while appearing in ROI would detect them. Any thoughts on this?

Hi Robert, we tested over about 5 hours. Results were consistent all the way through, so we felt it was long enough to see patterns. However, we do believe there could be performance differences depending on the type of rain, depending on how heavy it is, how windy, potential for hail, etc.

We typically ran 10 attempts with a human subject at each distance. Vehicles, not as many, because performance was pretty close to predictable based on the first 2-3, due to the much larger object size.

This testing was limited to cross line detection for two reasons. First and foremost, because it is the most common analytic in use in the industry. Second, this is a continuation of our previous camera analytics test, which was limited to crossline.

We will be testing other analytics in the future (starting today actually), and will be using other rules in various conditions.

I did a major analytics deployment some years ago, and in the rain we had signficant issues with directional detection in the rain, especially in evening and dark. Headlight reflections on wet pavement on opposite side of the road on a curve made the analytics think there was movement in the wrong direction consistently. On an interstate there were hundreds of alarms each morning.

Doing tripline for our first effort was a good idea, but I'll bet other functions might be more adversely affected by rain and other weather conditions. I'm sure you will do follow-on studies of more advanced features.

Too bad you didn't test that other ananlytic that would have "learned" that rain is normal...

Hi Marc, we tested virtual tripwires as a continuation of our Camera Analytics Tested: Axis vs. Bosch vs. Sony report. We will be testing AgentVi, VideoIQ, and Bosch IVA in upcoming reports, so we'll be looking at other rules in varying conditions then. It is, unfortunately, very difficult to plan rain testing in Pennsylvania, as we've had one day of steady rain in the past month. But we're getting there!

What, you don't just look at your iPhone or Android app for weather until it says "rain?"

Sounds good. I would be very interested to see performance comparisons of integrated vs. add-on analytics, edge vs. server based, and the like. I've always thought that edge should be better, but of course the algorithms are not identical either.

I had some good success with AgentVI back when they were still Aspectus. I hope you will also include Sightlogix and Puretech, both of whom include geolocation which sometimes is a good adjunct and sometimes a good alternate for radar.

My initial thought was that the rain droplets on the lens would be the issue but according to this test the reduction in analytics performance looked more like the rain itself caused visibilty to be reduced which then reduced analytics performance. This tells me that fog would have the same effect.

Which brings up another question ... what is the effect on the analytics with snow? Would slow falling snow flakes near the camera be detected?

I would think the snow flake question depends on the size and speed of the flake. We'll be testing this when winter comes, assuming we actually get snow this year.

I believe that snow will have more of an obscuration effect than rain, similar to fog, which will have more of an impact on performance that the chance of false alarms from falling snow.

Login to read this IPVM report.
Why do I need to log in?
IPVM conducts unique testing and research funded by member's payments enabling us to offer the most independent, accurate and in-depth information.

Related Reports on Video Analytics

Verint Victimized By Ransomware on Apr 18, 2019
Verint, which is best known in the physical security industry for video surveillance but has built a sizeable cybersecurity business as well, was...
ISC West 2019 Report on Apr 12, 2019
The IPVM team has finished at the Sands looking at what companies are offering and how they are changing their positioning. See below for 50+...
Bosch AI Camera Trainer Released And Tested on Apr 09, 2019
Bosch is releasing a highly unusual new AI feature - 'Camera Trainer'. Now, coming as a standard feature in Bosch IVA/EVA analytics, one can train...
Airship VMS Profile on Apr 03, 2019
Airship has been developing VMS software for over 10 years, however, with no outside investment, and minimal marketing, the company is not well...
IBM / Genetec Surveillance System Investigated Over Philippines Human Rights Abuses on Mar 22, 2019
A lengthy investigation into an IBM video surveillance project in the Philippines, raising concerns IBM helped local police conduct a bloody...
Large Hospital Security End User Interview on Mar 21, 2019
This large single-state healthcare system consists of many hospitals, and hundreds of health parks, private practices, urgent care facilities, and...
BCDVideo Expansion And Switch From HP To Dell Examined on Mar 11, 2019
BCDVideo says they have more than tripled revenue in the past 5 years and are continuing to grow, powered most recently by switching their lead...
Church Technology Director Security Interview on Mar 07, 2019
With 40+ years of experience in IT from a wide array of verticals, including US and foreign military, and large corporate and industrial settings,...
Police Department Surveillance Manager Interview on Feb 28, 2019
Former Memphis PD Surveillance Manager, Lt. Joseph Patty retired months ago, but kept busy during his decades on the force, working to build up...
BluB0x Company Profile on Feb 20, 2019
BluB0x has doubled in revenue every year since its founding in 2013, according to CEO Patrick Barry. We originally reported on them in 2015. At the...

Most Recent Industry Reports

Access Control Course Spring 2019 - Last Chance on Apr 19, 2019
This is the last chance to register for the Spring Access Control Course. IPVM offers the most comprehensive access control course in the...
Riser vs Plenum Cabling Explained on Apr 18, 2019
You could be spending twice as much for cable as you need. The difference between 'plenum' rated cable and 'riser' rated cable is subtle, but the...
Verint Victimized By Ransomware on Apr 18, 2019
Verint, which is best known in the physical security industry for video surveillance but has built a sizeable cybersecurity business as well, was...
Milestone Drops IFSEC on Apr 18, 2019
Milestone has dropped out of Europe's largest annual security trade show (IFSEC 2019), telling IPVM that they "have found that IFSEC in EMEA no...
The Fastest Growing Video Surveillance Sales Organization Ever - Verkada on Apr 17, 2019
Verkada has the fastest growing video surveillance sales organization ever. In less than 2 years, they already have more salespeople in the US...
Door Operators Access Control Tutorial on Apr 17, 2019
Doors equipped with door operators, specialty devices that automate opening and closing, tend to be quite complex. The mechanisms needed to...
Securadyne CEO: IPVM 'Entertaining For An Ignorant Few' on Apr 16, 2019
Securadyne's CEO Carey Boethel is unhappy with IPVM's report - Failed Integrator Rollup, Securadyne Sells to Guard Giant Allied. Indeed, he...
Dahua Repositionable IR Multi-Imager Camera Tested on Apr 16, 2019
Dahua has released their first repositionable multi-imager camera, the Multi-Flex 4x2MP, claiming integrated IR, true WDR, and flexible...
Strong ISC West 2019 For Manufacturers But Concerns For 2020 March Move on Apr 16, 2019
ISC West 2019 was strong for manufacturers, according to new IPVM survey results of 100+ manufacturers, consistent with 2018 results. However,...
Axis Supports HD Analog on Apr 15, 2019
In 2017, Axis declared 'Everything is IP': Now, in 2019, Axis has released support for HD analog, with their new encoders.  Why the change?...

The world's leading video surveillance information source, IPVM provides the best reporting, testing and training for 10,000+ members globally. Dedicated to independent and objective information, we uniquely refuse any and all advertisements, sponsorship and consulting from manufacturers.

About | FAQ | Contact