Camera Analytics Tested: Axis vs. Bosch vs. Sony

By: Ethan Ace, Published on Sep 17, 2013

With the move to IP and HD now commonplace, companies are looking for new ways to compete. While analytics has long disappointed, many still hold out hope for it being the next big thing.

Many major camera manufacturers offer analytics built-in or for a minimal additional charge. However, skepticism of their performance is commonplace.

The Test

In this report, we tested 3 camera manufacturers analytics - Axis cross line detection, Bosch's Intelligent Video Analytics [link no longer available] and Sony's Video Motion Filter (VMF).

The test focuses on tripwire / perimeter violation analytics as this is most common type used in security to alert against intruders entering into a restricted area.

We tested in 3 different scenes.

To start, we did a simple indoor tripwire at an entrance:

Then we progressed to an outdoor scene of moderate complexity and distance:

Get Notified of Video Surveillance Breaking News
Get Notified of Video Surveillance Breaking News

Finally, we tried a very long area with vegetation and shadows to maximize the challenge:

Key Findings

Here are our key findings from these tests:

General Considerations

  • Cross line detection performs best when the angle of incidence is as high as possible. This reduces false or early activations due to shadows or shallow perspective. 
  • Cross lines perform best when configured for outside-in crossing, not inside-out, with the object moving toward the camera. 
  • False alerts from background objects such as moving foliage and headlights were possible, but not common. Lines should be drawn to avoid such areas where possible.
  • All of the above combined mean that typical camera locations may not always provide best performance for cross line analytics.

 Competitive Performance

  • Axis P3354 Cross Line Detection detected objects at more ranges in more scenes under varying lighting than either Bosch NBN-733V or Sony SNC-VB600. However, it suffered from false activations, caused by shadows or shallow angle of incidence, and often activated multiple times.
  • Bosch NBN-733V's IVA triggered reliably in well lit scenes, far better than either Sony SNC-VB600 or Axis P3354 , with no false or multiple activations. However, in low light detection range was drastically reduced, and missed activations increased.
  • Sony SNC-VB600's built-in VMF lagged in our tests, only able to reliably detect people at very close range. Vehicles were detectable at longer distances, though short of Bosch and Axis. This is due at least in part to object size settings in Sony SNC-VB600's analytics not required in other cameras, which failed to detect objects if set to large, and detected single objects as multiple if set too small, missing detections.

Price

Cost of the built in analytics in this test vary by manufacturer:

  • Axis P3354  Cross Line Detection: ~$30 USD, with discounts on 10-packs of licenses.
  • Bosch NBN-733V IVA: 900 series cameras ship with IVA installed at no additional charge. 700 and 800 series models may be licensed at an additional cost of ~$100.
  • Sony SNC-VB600 VMF: Sony SNC-VB600's built-in analytics are available at no additional cost in 6th generation cameras. Fifth generation cameras previously shipped with DEPA analytics.

Recommendations

Cross line analytics may offer some useful advantages, but must be applied carefully, as mounting locations and FOV of typical cameras may not always be ideal. Users should carefully consider these factors and test analytics in place before relying upon them in the field.

In most scenes, with the exception of very low light, Bosch NBN-733V's IVA was most accurate, with almost no false or missed activations. Users should be aware of its low light limitations before deploying, however.

Axis P3354  Cross Line Detection was reliable in most scenes, and performed better in very low light than Bosch NBN-733V. However, users should be aware of its false activations due to headlights, shadows, and moving foliage, before deploying.

We do not recommend Sony SNC-VB600's VMF analytics in scenes aside from well lit indoor areas due to its issues detecting objects.

Configuration 

In these screencasts, we review configuration for each of the three manufacturers tested.

First, Axis P3354 Cross Line Detection. Configuration is very simple, only including the line (1 or 2 segments) and activation direction, with no other setup.

Next, Bosch NBN-733V IVA. The NBN-733V offered the most configuration options, including multiple lines, fine tuning of speed, size, contrast, and other parameters. It also presented information regarding objects detected and their path through the scene, making troubleshooting easier compared to others which did not display this.

Finally, Sony SNC-VB600's VMF setup contains adjustment for object size and speed, as well as standard settings such as line length and direction. Up to three lines can be set per camera, and combined into events. Users should be careful when configuration object size, as it can drastically affect detection performance.

Outdoor Applications

We tested the cross line analytics in several common outdoor scenarios.

Driveway/Parking Lot

First, we began with a parking lot/driveway scene at ranges up to 250', shown in this overhead view:

During the day, beginning at 25', all cameras are accurate.

Moving about halfway down the parking lot to ~120', Axis P3354 and Bosch NBN-733V once again trigger reliably on the vehicle. At this distance and beyond, Axis P3354 occasionally tirggered twice or more in quick succession, as the vehicle approached and crossed the line. Bosch NBN-733V triggered only once. The VB600 was unable to reliably detect objects at this range, so did not trigger.

Finally, with the line located all the way at the entrance to the lot from the road, both Axis P3354 and Bosch NBN-733V were still accurate, though Axis P3354  still activated multiple times as it did at 120'. 

At night, at close range, Axis P3354 and Bosch NBN-733V both trigger, while Sony SNC-VB600 does not. Bosch NBN-733V performed better when the vehicle was moving away from the camera, not toward, as it detected the object much more easily in the foreground.

At ~120' and beyond, only Axis P3354  triggers. Some activations were slightly early due to the reflection of headlights moving toward the camera.

At long distance, ~250', Axis P3354 still triggers. However, occasional false alarms were caused by cars passing on the roadway beyond the parking lot. Moving the line back from the road partway remedies this.

Long Range Tripwire

We also tested the cameras in the rear of our office, drawing a virtual tripwire inside the vegetation at the perimeter, another common application for cross line analytics.

 

At 100', both Axis P3354  and Bosch NBN-733V activated, while Sony SNC-VB600 did not (nor did it at any longer ranges):

 

At 200', Bosch NBN-733V continued to track and alert on our subject, while Axis P3354  was unable to:

 

At 300', none of the cameras triggered, with the subject simply too small to activate:

 At night, with the outdoor lighting on, approximately 2-3 lux, both Axis P3354 and Bosch NBN-733V triggered reliably at 50' range.

At 75', Bosch NBN-733V still triggered, while Axis P3354  did not. Beyond this, no cameras were able to detect our subject.

Switching off the outdoor lighting drops light levels below 1 lux. Detection range of Bosch NBN-733V drops to only 20', shown here.

Axis P3354 was able to detect our subject reliably up to 75', however.

Indoor Applications

One key application often cited for crossline analytics is detection of subjects entering a doorway. We set up our cameras in front of the office doors to simulate this, drawing our tripwire at the bottom of the door, with the cameras about 12' away. In this setup, Axis P3354  and Sony SNC-VB600 worked reliably, but Bosch NBN-733V did not trigger:

This is due to the way Bosch NBN-733V tracks objects, with the object path drawn higher, between the subject's knees and waist. At the above angle, the subject's path simply never crossed the line. Moving it partway up the doorway, Bosch NBN-733V triggers reliably, as does Axis P3354 . However, depending on the exact installation scenario, this line location may cause false triggers on object moving outside the door. Users should be careful to avoid this.

Conference Room

Thought not a common use for cross line analytics, we performed some tests in our controlled conference room in order to demonstrate some issues common throughout our tests.

First, this image shows Axis P3354  Cross Line Detection's sensitivity to shadows. In this case, though the line was set to trigger only when the subject moved toward the camera, the motion of his shadow on the floor caused false activations as he moved across the line in the opposite direction:

 This occurred as he moved across the room, as well. 

In this case, the line was drawn to trigger as the subject moved out of the door at the other end of the room. However, all three cameras failed to activate if he moved straight out of the door perpendicular to the cameras.

Detection was more reliable if he moved at more of an angle or toward the camera. 

However, users are better served using a single vertical line next to the door, so the subject crosses no matter what their path once in the room.

Test Parameters

We used the following cameras and firmware versions for this test:

  • Axis P3354, firmware 5.40.17
  • Bosch NBN-733V, firmware 5.80 build 73
  • Sony SNC-VB600, firmware 1.5.1

All cameras used default settings, with exposures standardized to 1/30s maximum.

 

4 reports cite this report:

Bosch IVA Video Analytics And Motion+ VMD Tested on Jun 06, 2018
Bosch's video analytics now ship on nearly every model, from indoor domes to high-end 5MP starlight cameras.  In this test, we evaluate Bosch's...
Axis vs Sony Super Size Shootout (Q1659 vs SNC-VB770) on Sep 11, 2017
Super low light, super sized sensor cameras are a growing trend. In the past year, 2 of the most notable entrants for these IP cameras have been...
Hikvision vs Exacq VMS on Jun 15, 2016
Hikvision and Exacq are two of the most commonly used recorders in the market, but how do they really match up? With Hikvision so much less...
Bosch Person / Car / Bike / Truck Analytics Tested on Jun 05, 2015
Automatically identify what objects are persons vs cars vs bikes vs trucks. That is the new feature and claim from Bosch in their newest IVA...
Comments (19) : PRO Members only. Login. or Join.

Related Reports

Color Low Light Mega Camera Shootout - Avigilon, Axis, Bosch, Dahua, Hanwha, Hikvision, Panasonic, Speco, Sony, Vivotek on Nov 12, 2019
This is the biggest color low light shootout ever, testing 20+ super low light models from 10 manufacturers: Increasingly, each manufacturer...
Biggest Low Light Problems 2019 on Nov 08, 2019
Over 150 integrators responded to our survey question: "What are the biggest problems you face getting good low-light images?" Inside, we share...
Rhombus Cameras, VMS and Analytics Tested on Nov 06, 2019
Rhombus boasts they have created "the new standard in Enterprise, cloud-managed video security" and told IPVM in January 2019 they offer twice the...
90+ Companies Profile Directory on Nov 05, 2019
While IPVM covers the largest companies in the industry regularly (like Axis, Dahua, Hikvision, etc.), IPVM strives to do a profile post on each...
Avigilon Appearance Search Tested on Oct 30, 2019
Avigilon Appearance Search claims that it "sorts through hours of video with ease, to quickly locate a specific person or vehicle of interest...
nFlux AI Startup Profile on Oct 28, 2019
nFlux, an Amazon, Google, and Microsoft funded AI startup told IPVM their mission is to build the most intelligent video analytics platform in the...
Illustra Pro Gen3 4K Camera Tested on Oct 23, 2019
Johnson Controls has released the latest generation in their Illustra Pro line, the Pro Mini-Dome Gen 3, a non-OEM model claiming to "optimize...
Integrated IR Camera Usage Statistics 2019 on Oct 21, 2019
Virtually every IP camera now comes with integrated IR but how many actually make use of IR or choose 'super' low light cameras without IR? In...
Pelco Sarix Pro3 Camera Tested on Oct 16, 2019
Pelco has released their Sarix Professional Series 3 cameras, claiming "more security detail in challenging scenes with excellent low light and...
Avigilon H5A Analytic Cameras Tested on Oct 07, 2019
Avigilon has released its H5A analytic cameras, claiming to "detect more objects with greater accuracy even in crowded scenes." We tested the...

Most Recent Industry Reports

Wireless / WiFi Access Lock Guide on Nov 12, 2019
For some access openings, running wires can add thousands in cost, and wireless alternatives that avoid it becomes appealing. But using wireless...
Open vs End-to-End Systems: Integrator Statistics 2019 on Nov 11, 2019
Preference for open systems is on the decline, according to new IPVM statistics. We asked integrators: For video surveillance systems, do you...
Biggest Low Light Problems 2019 on Nov 08, 2019
Over 150 integrators responded to our survey question: "What are the biggest problems you face getting good low-light images?" Inside, we share...
US Issues Criminal Charges For Fraudulently Selling Hikvision And Other China Products on Nov 07, 2019
The US government has made an unprecedented move on the video surveillance supply chain, charging a US company, Aventura for "having conspired with...
The Access Control Codes Guide: IBC, NFPA 72, 80 & 101 on Nov 07, 2019
For access, there is one basic maxim: Life safety above all else. But how do you know if all applicable codes are being followed? While the...
Rhombus Cameras, VMS and Analytics Tested on Nov 06, 2019
Rhombus boasts they have created "the new standard in Enterprise, cloud-managed video security" and told IPVM in January 2019 they offer twice the...
"Stress in the Residential Market" - Major Lender Exits on Nov 06, 2019
The residential security / 'alarm' market is getting worse, at least for traditional players. Now, one of the biggest lenders in the industry has...
Aiphone Video Intercom Tested (IX Series 2) on Nov 05, 2019
Aiphone was one of integrator's favorite intercom manufacturers but how well do their products work? The company's newest offering, the IX Series 2...
90+ Companies Profile Directory on Nov 05, 2019
While IPVM covers the largest companies in the industry regularly (like Axis, Dahua, Hikvision, etc.), IPVM strives to do a profile post on each...