Analytics vs Rain: Axis, Bosch & Sony

Author: Ben Wood, Published on Oct 21, 2013

Rain. This was a key objection to our Camera Analytics Test: Axis vs. Bosch vs. Sony. Members rightfully noted it is one thing for an analytics to do well indoors and outdoors at night but what about in the rain?

In this report, we share our findings of testing the same camera analytic in moderate to heavy rain. Here's what the day looked like:

We set up crosslines / tripwires from 20' to 200' away from our cameras.

Additionally, we tested a crossline perpendicular to the camera from 20' to 200' away.

Inside, we share our test results of who performed best and worst, plus how much the rain impacted performance.

****. **** *** * *** ********* ** ********* ********* ****: **** **. ***** **. ****. ******* ********** ***** ** ** *** ***** *** ** analytics ** ** **** ******* *** ******** ** ***** *** what ***** ** *** ****?

** **** ******, ** ***** *** ******** ** ******* *** same ****** ******** ** ******** ** ***** ****. ****'* **** the *** ****** ****:

** *** ************/ ********* **** **' ** ***' **** **** *** *******.

************, ** ****** * ********* ************* ** *** ****** **** **' ** ***' ****.

******, ** ***** *** **** ******* ** *** ********* **** and *****, **** *** **** *** **** ******** ***********.

[***************]

Key ********

**** *** *** *** ******** **** **** ****:

  • ** ***** ********* **** ********* *** ** ****** **** (********-**-*****)** ******** ******* **** ****** ********.
  • ***** **** *********** *** ******** ******* ** **** ****** (** and **** ~***') ** ****, **** ******** ** *** ******* from *** ****** ***** ********* ********
  • **** * ***** *******, *** ***** ***-****'* *** *** **** Crossline ********* ******* ** **** ****** *********** ** ***' *** ********** missed *********** ** ***'. **** ** **** **********, *** **** alert **** *** ***% ***** ***' *** ***' *** **** ********* *** ***** IVA, ************. ********* ******* ***** ** **** ***-*****'* *** ********* to ** *** **** ** ***** ****** (**% ** **', 40% ** **' *** *% ** ***') ********** **** *** no-rain *******. 
  • **** *** ****** *** * *******, ** ***** *********** ** the ********* *********** ** **** ****** **** ********, ******** ** the **** ** **-**** *********.  

***************

** ******** ** *** ******* *************** ** *** ******** ******, these ***** **** **** **** *** ***** ******* ********** **** also ** ** ********* ****** ** ******** **** ********* * reliable *****-**** ********* ******. *** ******* ********* ******** **** ** reduced ** ******* ** ****** ** ***** **** *******, ******** **** ***** factors, **** ** ******** ********, *** *** ******** **********. ********** attention ** ****** **** *** ****** ****** ** ******** ** imaginary ******** ** **** ********* **** ***-***' ****.

*******, ***** ******** / ******, ***** ************** ******, *** *** demonstrate ********* ******** *** ** *** ****.

****

** ****** *** ***** **** ********* ** *** ****** ******* scenarios:

********

*****, ** ***** **** ******* * ******* ******** ***** *** parking ***/********, ** ***** ** *** ******** **** *****. 

*** ******* ******* *** ******** ** ********** ***** ** ******* distances **** **' ** ***' **** **** *** ******.

** **', **** **** ***** *** ***** ***-**** ******** **** full ********, ***** **** ***-*****'* ******* **** *** */**.

** ********* ******* **' *** **', ******* **** ** ***** decreased ************ ** ****** *** *********** ** ***'. ***** *** ***-**** continued ** ******* ******** ** **** *****.

** ***', **** *** ***** *** ***-**** ******* ** **** **** activations. **** ** ***** *********** **** **** **** ******** ** their ******** ** **** ***** ** *** ******** ** **** test.

** ***', **** ** *** ******* ********* ** ***. ** clear **********, *** ***-**** *** ***** ** ** ***% ******** 200'.

********* ******* ***** ** **** ****** **** *** ******* ******** tests **** **** ********* *****. ***** ***** **** ***% ******* rate, ******** ******* **** ******* ** ******. ***** ** **** *** observed *** ***** ****** ** **** ****, ******* **** ** based ** **** ****** ** *** ******* *****.

Vehicle ** ********

** **** ****** *** ******** ** ************ ********* ** ***** *******, ********* ******* ********* ** ***** *********; 25', ***' *** ***'. **** ***** *** ******* ** *** tripwire ****, *** ***** **** *** ** ** *** ********, across *** ***.

******* *** **** ***** ** ***** ***-**** ****** *** ** the ******* *********** ** *** ** *** ***** *********. *** Sony *****, ** *** ***** ****, *** ****** *********** **** at *** ******* **** ***** ** **', *** ********* **** at ****** *********. ***** ******* **** ******* ** **** ** saw ** *** ** **** ****. *********, ** **** *** observed *** ****** ** *** **** ** *** ********* ***** of ****** *******, **** ** ********, ***** ***'.


** ~***' ****'* ******** ***** ** ~**%.

*******, ** ***', *** ***-***** ** ****** ** ****** *** vehicle ** ***.


*** ***** ***** ********** ******* *********** ** ******** ** *** rain:

 

Comments (7)

Figen:

Thanks for the report. Over how many hours was the data collected? I have found that over extended periods of rain >12 hours (It rains a lot in Vancouver) you should get some false positives. When calculating detecting true events accuracy, how many true events were done per test?

Also, all of the testing was using crossing line detection, which I never use. I have a belief about crossing line detection as not being as good as "appearing in ROI". Early on I was finding that trespass by people would be missed because often fencing or hoarding would have storage containers/ obstructions beside them and entry might be obscured and crossing line detection would miss them, while appearing in ROI would detect them. Any thoughts on this?

Hi Robert, we tested over about 5 hours. Results were consistent all the way through, so we felt it was long enough to see patterns. However, we do believe there could be performance differences depending on the type of rain, depending on how heavy it is, how windy, potential for hail, etc.

We typically ran 10 attempts with a human subject at each distance. Vehicles, not as many, because performance was pretty close to predictable based on the first 2-3, due to the much larger object size.

This testing was limited to cross line detection for two reasons. First and foremost, because it is the most common analytic in use in the industry. Second, this is a continuation of our previous camera analytics test, which was limited to crossline.

We will be testing other analytics in the future (starting today actually), and will be using other rules in various conditions.

I did a major analytics deployment some years ago, and in the rain we had signficant issues with directional detection in the rain, especially in evening and dark. Headlight reflections on wet pavement on opposite side of the road on a curve made the analytics think there was movement in the wrong direction consistently. On an interstate there were hundreds of alarms each morning.

Doing tripline for our first effort was a good idea, but I'll bet other functions might be more adversely affected by rain and other weather conditions. I'm sure you will do follow-on studies of more advanced features.

Too bad you didn't test that other ananlytic that would have "learned" that rain is normal...

Hi Marc, we tested virtual tripwires as a continuation of our Camera Analytics Tested: Axis vs. Bosch vs. Sony report. We will be testing AgentVi, VideoIQ, and Bosch IVA in upcoming reports, so we'll be looking at other rules in varying conditions then. It is, unfortunately, very difficult to plan rain testing in Pennsylvania, as we've had one day of steady rain in the past month. But we're getting there!

What, you don't just look at your iPhone or Android app for weather until it says "rain?"

Sounds good. I would be very interested to see performance comparisons of integrated vs. add-on analytics, edge vs. server based, and the like. I've always thought that edge should be better, but of course the algorithms are not identical either.

I had some good success with AgentVI back when they were still Aspectus. I hope you will also include Sightlogix and Puretech, both of whom include geolocation which sometimes is a good adjunct and sometimes a good alternate for radar.

My initial thought was that the rain droplets on the lens would be the issue but according to this test the reduction in analytics performance looked more like the rain itself caused visibilty to be reduced which then reduced analytics performance. This tells me that fog would have the same effect.

Which brings up another question ... what is the effect on the analytics with snow? Would slow falling snow flakes near the camera be detected?

I would think the snow flake question depends on the size and speed of the flake. We'll be testing this when winter comes, assuming we actually get snow this year.

I believe that snow will have more of an obscuration effect than rain, similar to fog, which will have more of an impact on performance that the chance of false alarms from falling snow.

Login to read this IPVM report.
Why do I need to log in?
IPVM conducts unique testing and research funded by member's payments enabling us to offer the most independent, accurate and in-depth information.

Related Reports on Video Analytics

ReconaSense - The AI / Access Control / Analytics / IoT / Video Company Profile on Jun 12, 2018
One company's ISC West booth stood out for displaying a light-up tower of buzzwords. The company, ReconaSense, pledged to be 'making sense of it...
Bosch IVA Video Analytics And Motion+ VMD Tested on Jun 06, 2018
Bosch's video analytics now ship on nearly every model, from indoor domes to high-end 5MP starlight cameras.  In this test, we evaluate Bosch's...
Hanwha Wisenet X Analytics and VMD Test on May 24, 2018
Continuing our updated testing of camera analytics, we tested Hanwha's Wisenet X analytics for over two weeks in multiple scenes, indoors and out,...
The Hikvision Smart Classroom Behavior Management System on May 22, 2018
Hikvision's rapidly growing offering of analytics, which we most recently examined with Hikvision's ethnic minority analytics, is now going into...
Dahua Intrusion Analytics And VMD Tested (Poorly) on May 21, 2018
Dahua ships basic analytics on practically all their cameras, ranging from low cost to high end. To see how these analytics work in real world...
Stealth / UCIT - Remote Video Monitoring Provider Profile on May 18, 2018
Can 2 remote video monitoring companies, Stealth Monitoring from the US and UCIT from Canada combine to impact the market and compete in a changing...
Nortek Acquires IntelliVision on May 08, 2018
One of the largest intrusion manufacturers, Nortek, has acquired the self-proclaimed 'Leader in AI-Based Video Analytics Software',...
Hikvision's Minority Analytics on May 08, 2018
Hikvision has developed and is deploying minority analytics inside China, automatically identifying and categorizing individuals as ethnic...
Hikvision VMD And Intrusion Analytics Tested on May 04, 2018
VMD worked poorly on Hikvision cameras, new IPVM testing found. However, Hikvision ships analytics on practically every IP camera, with intrusion...
JCI / Tyco VideoEdge NVR Profile on May 03, 2018
JCI (formerly Tyco) is one of the largest players in the global security industry, both as a manufacturer and an integrator. The company's most...

Most Recent Industry Reports

IFSEC Show Report - Live From London on Jun 19, 2018
IPVM is live from London reporting on the IFSEC show. The Chinese have taken over the UK, centered on Hikvision, flanked by Dahua, Huawei and a...
Axis Guardian - Cloud VMS for Alarm Companies on Jun 19, 2018
Axis has struggled to deliver a cloud-based managed service video platform. Video service providers have utilized AVHS for over a decade, and have...
IPVM Vulnerability Scanner Released on Jun 18, 2018
IPVM is proud to announce video surveillance's first and only cybersecurity vulnerability scanner. This tool allows quickly and simply...
Hikvision Corrects False Cybersecurity Announcement on Jun 18, 2018
Hikvision has corrected a false cybersecurity announcement that claimed a British government-sponsored program endorsed the cybersecurity of...
July 2018 IP Networking Course on Jun 16, 2018
The last chance to save $50 on registration is this Thursday, June 21st. Register now and save. This is the only networking course designed...
The Dumb Ones: PSA's Bozeman On Cybersecurity on Jun 15, 2018
The smart ones are the hundred people who flew to Denver and spent $500+ on a 1.5-day conference featuring Dahua as a 'cyber responsible partner',...
Amazon Ring Launches $10 Monthly Professional Alarm Monitoring on Jun 15, 2018
Amazon's Ring has announced an alarm system with 24/7 professional alarm monitoring for $10 per month, a fraction of the $30+ per month traditional...
Axis Releases First New Access Controller In 5 Years (A1601) on Jun 15, 2018
It has been 5 years since Axis 2013 entry in the physical access control market, with the A1001 (IPVM test). Now, Axis has released its second...
Hikvision 12MP Fisheye Camera Tested (DS-2CD63C2F-IV) on Jun 14, 2018
Hikvision's DS-2CD63C2F-IV is their flagship panoramic camera, with a 12MP imager, 15m integrated IR, smart codec, and more. We tested the 63C2 in...
Four Major Outdoor Camera Install Problems on Jun 14, 2018
Over 140 integrators told us the top four camera installation mistakes that lead to unexpected problems and failures. Their comments often...

The world's leading video surveillance information source, IPVM provides the best reporting, testing and training for 10,000+ members globally. Dedicated to independent and objective information, we uniquely refuse any and all advertisements, sponsorship and consulting from manufacturers.

About | FAQ | Contact