100% Video Analytics Accuracy?

Author: Ethan Ace, Published on Feb 01, 2012

In January 2012, analytic provider SightLogix announced its SightSensor intrusion detection analytics had been validated by the Transportation Safety Administration, providing "100% video analytics accuracy", a bold claim.  In this note, we look at this announcement, the TSA test, the integrator's experience and what users should look for when reviewing manufacturer test reports.

Overview 

While the report was prepared by the TSA, the evaluation of the SightLogix system was performed by the Safe Skies Alliance, which is a non-profit formed in 1997 to support testing of aviation security technology. The SSA receives funding from the FAA, and support from the TSA. This testing was performed at Buffalo International Airport in March 2011.

SightLogix has said that this report will be available to other airport security departments, but not to the public. While we understand the need for redaction, and keeping some of these measures secret, any number of other industries could benefit from this test, as well. As one commentor pointed out, though, it would take TSA involvement to distribute this to even industries outside of aviation, let alone the security industry at large.

Evaluating Claims

SightLogix's perimeter detection system is well respected and higher cost than many other options in the industry. It has also been developed for the specific purpose of intrusion detection, unlike many all-in-one analytic packages (see our review of their new 2012 camera series). Given this, we would not be surprised if detection rates are high, with false alarm rates low. However, users should be careful when reading this test report and press release, as some claims may be misleading.

  • Location and coverage information is redacted, so we're left with no knowledge of how wide the area covered is, nor are detection distances shared.
  • Intrusion techniques (the scenarios) were redacted, so we've no idea if subjects were detected while walking or crawling, from which direction they approached, at what speed, etc. Results for each of these scenarios were also redacted, so performance by intrusion type is unknown.
  • All nuisance and false alarm reports are redacted. It is extremely unlikely that any systems can be free of false and nuisance alarms, so this lack of detail removes information regarding potential system weaknesses.

Integrator Response

In a LinkedIn discussion regarding this release, one of our members who was involved with the integration shared the following clarification:

  • Regarding false alarms: "We had a number of alarms in the beginning and by a process of adjusmtments we were able to tune the system for the customer to an acceptable level. That is the key when installing an analytic system is setting the expectations of the customer correctly. 99% of the alarms are caused be animals and dependant on the size and quantity may or may not be a concern for the airport."
  • Regarding Nuisance alarms: "It does track objects that are not humans, but that was an added benefit to certain airport operations as birds, deer and other animals are not wanted guests on a runway. The biggest benefit to the system is that if there is an alarm it is automatically time stamped and logged, a monitor displays the target and the police officer can, by not touching anything, view to see if the alarm is something that requires attention."
  • Regarding performance: "I can say that the system after being in service for over a year is still providing a useful tool that is reliable and paying for itself."

He also stated that the system has been expanded to a second airport, bus garages, and a light rail system, which would lead us to believe the system is performing satisfactorily.

Conclusion

We found the integrator's insights to be far more useful than the study itself. A key factor was that the integrator was able to share experiences in the optimization stage and how it is used in daily operations. The lesson to take away - If you want a good reference, stay away from 'studies' and find an engineer who was involved in the details.

Comments : PRO Members only. Login. or Join.

Related Reports

2Gig Gun Lock / Motion Detector Tested on Aug 17, 2018
Safer guns for families and an opportunity for security dealers to sell more services? That is the aim of Nortek's 2GIG 'Gun Motion Detector'...
Video Analytics Integration Guide on Aug 16, 2018
Video analytics is hot again (at least conceptually) but integrating video analytics with VMSes can be challenging. This is especially significant...
ISS VMS / Video Analytics Company Profile on Aug 16, 2018
Who is ISS? In the past few months, they had one of the craziest ISC West promo items in years. Then, they hired industry veteran and ex-Dahua...
SimpliSafe Violating California, Florida, and Texas Licensing Laws on Aug 14, 2018
IPVM has verified that DIY security system provider SimpliSafe, founded in 2006 and acquired in June of 2018 at a billion dollar valuation, is...
Nortek Sues SDS, Battle Over Unpaid Bill and Cancelled Lines on Aug 13, 2018
Nortek and SDS legal battle continues. As IPVM reported, SDS sued Nortek alleging bribery and antitrust violation. However, Wave fired back at SDS,...
Uniview Intrusion Analytics and VMD Tested on Aug 13, 2018
IPVM's IP Camera Analytics Shootout featuring Avigilon, Axis, Bosch, Dahua, Hanwha, Hikvision created some ill will with a Uniview distributor who...
IP Camera Analytics Shootout - Avigilon, Axis, Bosch, Dahua, Hanwha, Hikvision, Uniview on Jul 30, 2018
Video analytics are hot again. But whose analytics really work? IPVM bought and tested Avigilon, Axis, Bosch, Dahua, Hanwha and Hikvision cameras,...
Eagle Eye Networks Cloud VMS Tested on Jul 26, 2018
Eagle Eye has become one of the most significant players in the industry in the past few years: Eagle Eye's Owner Acquired Brivo Eagle Eye...
Avigilon "Self-Learning" Analytics And VMD Tested on Jul 23, 2018
Avigilon is a popular choice for video analytics, offering "self-learning" analytics built into their cameras as well as appliances. We tested...
Directory of Video Surveillance Startups on Jul 18, 2018
This directory provides a list of video surveillance startups to help you see and research what companies are new or not yet broadly known entity...

Most Recent Industry Reports

2Gig Gun Lock / Motion Detector Tested on Aug 17, 2018
Safer guns for families and an opportunity for security dealers to sell more services? That is the aim of Nortek's 2GIG 'Gun Motion Detector'...
Video Analytics Integration Guide on Aug 16, 2018
Video analytics is hot again (at least conceptually) but integrating video analytics with VMSes can be challenging. This is especially significant...
Hikvision IP Camera Critical Vulnerability 2018 Disclosed on Aug 16, 2018
The same day that the US government passed a prohibition on Hikvision cameras, Hikvision disclosed a critical vulnerability for its IP...
ISS VMS / Video Analytics Company Profile on Aug 16, 2018
Who is ISS? In the past few months, they had one of the craziest ISC West promo items in years. Then, they hired industry veteran and ex-Dahua...
Chinese OEM Avycon Gets ADI Push on Aug 15, 2018
Who is Avycon? An American company? A Korean company? A couple of guys relabelling Chinese products? The latter is the best explanation. While...
Backboxes for Video Surveillance Tutorial on Aug 15, 2018
Backboxes are a necessity in surveillance, whether for managing cable whips, recessing cameras, adding wireless radios. But it can be confusing to...
Genetec Stratocast / Comcast 'Motion Insights' Examined on Aug 15, 2018
Comcast recently announced "SmartOffice Motion Insights", an extension to their Genetec OEMed cloud video service (covered by IPVM here). This...
SimpliSafe Violating California, Florida, and Texas Licensing Laws on Aug 14, 2018
IPVM has verified that DIY security system provider SimpliSafe, founded in 2006 and acquired in June of 2018 at a billion dollar valuation, is...
Ban of Dahua and Hikvision Is Now US Gov Law on Aug 13, 2018
The US President has signed the 2019 NDAA into law, banning the use of Dahua and Hikvision (and their OEMs) for the US government, for US...
Cut Milestone Licensing Costs 80% By Using Hikvision and Dahua NVRs (Tested) on Aug 13, 2018
Enterprise VMS licensing can be quite expensive, with $200 or more per channel common, meaning a 100 camera system can cost $20,000 in VMS...

The world's leading video surveillance information source, IPVM provides the best reporting, testing and training for 10,000+ members globally. Dedicated to independent and objective information, we uniquely refuse any and all advertisements, sponsorship and consulting from manufacturers.

About | FAQ | Contact