Analytics vs Rain: Axis, Bosch & Sony

By: Ben Wood, Published on Oct 21, 2013

Rain. This was a key objection to our Camera Analytics Test: Axis vs. Bosch vs. Sony. Members rightfully noted it is one thing for an analytics to do well indoors and outdoors at night but what about in the rain?

In this report, we share our findings of testing the same camera analytic in moderate to heavy rain. Here's what the day looked like:

We set up crosslines / tripwires from 20' to 200' away from our cameras.

Additionally, we tested a crossline perpendicular to the camera from 20' to 200' away.

****. **** *** * key ********* ** ********* ********* ****: **** vs. ***** **. ****. ******* ********** ***** it ** *** ***** for ** ********* ** do **** ******* *** outdoors ** ***** *** what ***** ** *** rain?

** **** ******, ** share *** ******** ** testing *** **** ****** analytic ** ******** ** heavy ****. ****'* **** the *** ****** ****:

** *** ************/ ********* **** **' to ***' **** **** our *******.

************, ** ****** * ********* ************* ** *** ****** from **' ** ***' away.

******, ** ***** *** test ******* ** *** performed **** *** *****, plus *** **** *** rain ******** ***********.

[***************]

Key ********

**** *** *** *** findings **** **** ****:

  • ** ***** ********* **** generated *** ** ****** rain (********-**-*****)** ******** ******* **** camera ********.
  • ***** **** *********** *** slightly ******* ** **** ranges (** *** **** ~150') ** ****, **** compared ** *** ******* from *** ****** ***** ********* ********
  • **** * ***** *******, the ***** ***-****'* *** and **** ********* ********* ******* to **** ****** *********** at ***' *** ********** missed *********** ** ***'. With ** **** **********, the **** ***** **** was ***% ***** ***' *** 200' for **** ********* *** ***** IVA, ************. ********* ******* rates ** **** ***-*****'* VMF ********* ** ** low **** ** ***** ranges (**% ** **', 40% ** **' *** 0% ** ***') ********** with *** **-**** *******. 
  • **** *** ****** *** a *******, ** ***** differences ** *** ********* performance ** **** ****** were ********, ******** ** the **** ** **-**** condition.  

***************

** ******** ** *** overall *************** ** *** previous ******, ***** ***** show **** **** *** other ******* ********** **** also ** ** ********* factor ** ******** **** designing * ******** *****-**** detection ******. *** ******* detection ******** **** ** reduced ** ******* ** ****** ** taken **** *******, ******** with ***** *******, **** as ******** ********, *** and ******** **********. ********** attention ** ****** **** the ****** ****** ** crossing ** ********* ******** at **** ********* **** 100-150' ****.

*******, ***** ******** / alerts, ***** ************** ******, did *** *********** ********* problems *** ** *** rain.

****

** ****** *** ***** line ********* ** *** common ******* *********:

********

*****, ** ***** **** setting * ******* ******** along *** ******* ***/********, as ***** ** *** overhead **** *****. 

*** ******* ******* *** tripwire ** ********** ***** at ******* ********* **** 20' ** ***' **** from *** ******.

** **', **** **** P3354 *** ***** ***-**** detected **** **** ********, while **** ***-*****'* ******* rate *** */**.

** ********* ******* **' and **', ******* **** of ***** ********* ************ ** ****** *** activations ** ***'. ***** and ***-**** ********* ** perform ******** ** **** range.

** ***', **** *** P3354 *** ***-**** ******* ** miss **** ***********. **** is ***** *********** **** both **** ******** ** their ******** ** **** range ** *** ******** no **** ****.

** ***', **** ** the ******* ********* ** all. ** ***** **********, the ***-**** *** ***** to ** ***% ******** 200'.

********* ******* ***** ** each ****** **** *** virtual ******** ***** **** been ********* *****. ***** marks **** ***% ******* rate, ******** ******* **** ******* to ******. ***** ** have *** ******** *** false ****** ** **** test, ******* **** ** based ** **** ****** in *** ******* *****.

Vehicle ** ********

** **** ****** *** accuracy ** ************ ********* ** ***** *******, targeting ******* ********* ** three *********; **', ***' and ***'. **** ***** was ******* ** *** tripwire ****, *** ***** were *** ** ** the ********, ****** *** FOV.

******* *** **** ***** or ***** ***-**** ****** any ** *** ******* activations ** *** ** the ***** *********. *** Sony *****, ** *** other ****, *** ****** activations **** ** *** closest **** ***** ** 25', *** ********* **** at ****** *********. ***** results **** ******* ** what ** *** ** the ** **** ****. Therefore, ** **** *** observed *** ****** ** the **** ** *** detection ***** ** ****** objects, **** ** ********, until ***'.


** ~***' ****'* ******** drops ** ~**%.

*******, ** ***', *** SNC-VB600 ** ****** ** detect *** ******* ** all.


*** ***** ***** ********** overall *********** ** ******** in *** ****:

 

Comments (7)

Figen:

Thanks for the report. Over how many hours was the data collected? I have found that over extended periods of rain >12 hours (It rains a lot in Vancouver) you should get some false positives. When calculating detecting true events accuracy, how many true events were done per test?

Also, all of the testing was using crossing line detection, which I never use. I have a belief about crossing line detection as not being as good as "appearing in ROI". Early on I was finding that trespass by people would be missed because often fencing or hoarding would have storage containers/ obstructions beside them and entry might be obscured and crossing line detection would miss them, while appearing in ROI would detect them. Any thoughts on this?

Hi Robert, we tested over about 5 hours. Results were consistent all the way through, so we felt it was long enough to see patterns. However, we do believe there could be performance differences depending on the type of rain, depending on how heavy it is, how windy, potential for hail, etc.

We typically ran 10 attempts with a human subject at each distance. Vehicles, not as many, because performance was pretty close to predictable based on the first 2-3, due to the much larger object size.

This testing was limited to cross line detection for two reasons. First and foremost, because it is the most common analytic in use in the industry. Second, this is a continuation of our previous camera analytics test, which was limited to crossline.

We will be testing other analytics in the future (starting today actually), and will be using other rules in various conditions.

I did a major analytics deployment some years ago, and in the rain we had signficant issues with directional detection in the rain, especially in evening and dark. Headlight reflections on wet pavement on opposite side of the road on a curve made the analytics think there was movement in the wrong direction consistently. On an interstate there were hundreds of alarms each morning.

Doing tripline for our first effort was a good idea, but I'll bet other functions might be more adversely affected by rain and other weather conditions. I'm sure you will do follow-on studies of more advanced features.

Too bad you didn't test that other ananlytic that would have "learned" that rain is normal...

Hi Marc, we tested virtual tripwires as a continuation of our Camera Analytics Tested: Axis vs. Bosch vs. Sony report. We will be testing AgentVi, VideoIQ, and Bosch IVA in upcoming reports, so we'll be looking at other rules in varying conditions then. It is, unfortunately, very difficult to plan rain testing in Pennsylvania, as we've had one day of steady rain in the past month. But we're getting there!

What, you don't just look at your iPhone or Android app for weather until it says "rain?"

Sounds good. I would be very interested to see performance comparisons of integrated vs. add-on analytics, edge vs. server based, and the like. I've always thought that edge should be better, but of course the algorithms are not identical either.

I had some good success with AgentVI back when they were still Aspectus. I hope you will also include Sightlogix and Puretech, both of whom include geolocation which sometimes is a good adjunct and sometimes a good alternate for radar.

My initial thought was that the rain droplets on the lens would be the issue but according to this test the reduction in analytics performance looked more like the rain itself caused visibilty to be reduced which then reduced analytics performance. This tells me that fog would have the same effect.

Which brings up another question ... what is the effect on the analytics with snow? Would slow falling snow flakes near the camera be detected?

I would think the snow flake question depends on the size and speed of the flake. We'll be testing this when winter comes, assuming we actually get snow this year.

I believe that snow will have more of an obscuration effect than rain, similar to fog, which will have more of an impact on performance that the chance of false alarms from falling snow.

Login to read this IPVM report.
Why do I need to log in?
IPVM conducts unique testing and research funded by member's payments enabling us to offer the most independent, accurate and in-depth information.

Related Reports on Video Analytics

Anyvision Facial Recognition Tested on Aug 21, 2019
Anyvision is aiming for $1 billion in revenue by 2022, backed by $74 million in funding. But does their performance live up to the hype they have...
Verkada People And Face Analytics Tested on Aug 16, 2019
This week, Verkada released "People Analytics", including face analytics that they describe is a "game-changing feature" that "pushes the...
Dahua Analytics+ Tested on Aug 07, 2019
Dahua's analytics have performed poorly in past shootouts. But now, they claim their new Analytics+ "algorithms significantly improve accuracy and...
Honeywell Speaks On NDAA Ban, New Non-Banned Cameras and Cybersecurity on Aug 06, 2019
For years, Honeywell has depended on Dahua, a company with a poor cybersecurity track record and now banned by the US NDAA, for the development and...
History of Video Surveillance on Jul 19, 2019
The video surveillance market has changed significantly since 2000, going from VCRs to ab emerging AI cloud era.  The goal of this history is to...
Wyze AI Analytics Tested - Beats Axis and Hikvision on Jul 17, 2019
$20 camera disruptor Wyze has released free person detection deep learning analytics to all of their users, claiming users will "Only get notified...
Ivideon Russian VSaaS Profile on Jun 27, 2019
Ivideon was an early VSaaS entrant, initially focusing on the consumer market, claiming massive growth to IPVM in 2014. We spoke to Ivideon, to...
Directory of 60 Video Surveillance Startups on Jun 25, 2019
This directory provides a list of video surveillance startups to help you see and research what companies are new or not yet broadly known. 2019...
Repositionable Multi-Imager Camera Shootout - Avigilon, Axis, Dahua, Hanwha, Hikvision, Panasonic, Vivotek on Jun 19, 2019
Repositionable multi-imager cameras are one of the fastest growing segments in video surveillance, with a slew of new offerings being recently...
Startup Vaion Launching End-to-End AI Solution Backed with $20 Million Funding on Jun 17, 2019
An EU / USA video surveillance startup, Vaion, founded by ex-Cisco Senior Directors is launching an end-to-end VSaaS platform with $20 million in...

Most Recent Industry Reports

China Dahua Replaces Their Software With US Pepper on Aug 22, 2019
What does a US government banned company do to improve its security positioning in the US? Well, Dahua is unveiling a novel solution, partnering...
Security Integrators Outlook On Remaining Integrators In 2025 on Aug 22, 2019
The industry has changed substantially in the last decade, with the rise of IP cameras and the race to the bottom. Indeed, more changes may be...
First GDPR Facial Recognition Fine For Sweden School on Aug 22, 2019
A school in Sweden has been fined $20,000 for using facial recognition to keep attendance in what is Sweden's first GDPR fine. Notably, the fine is...
Anyvision Facial Recognition Tested on Aug 21, 2019
Anyvision is aiming for $1 billion in revenue by 2022, backed by $74 million in funding. But does their performance live up to the hype they have...
Dahua 4K Camera Shootout on Aug 20, 2019
Dahua's new Pro Series 4K N85CL5Z claims to "deliver superior images in all lighting and environmental conditions", but how does this compare to...
ZK Teco Atlas Access Control Tested on Aug 20, 2019
Who needs access specialists? China-based ZKTeco claims its newest access panel 'makes it very easy for anyone to learn and install access control...
Uniview Beats Intel In Trademark Lawsuit on Aug 19, 2019
Uniview has won a long-running trademark lawsuit brought by Intel, with Beijing's highest court reversing an earlier Intel win, centered on...
Suprema Biometric Mass Leak Examined on Aug 19, 2019
While Suprema is rarely discussed even within the physical security market, the South Korean biometrics manufacturer made global news this past...
Verkada People And Face Analytics Tested on Aug 16, 2019
This week, Verkada released "People Analytics", including face analytics that they describe is a "game-changing feature" that "pushes the...

The world's leading video surveillance information source, IPVM provides the best reporting, testing and training for 10,000+ members globally. Dedicated to independent and objective information, we uniquely refuse any and all advertisements, sponsorship and consulting from manufacturers.

About | FAQ | Contact