100% Video Analytics Accuracy?

Author: Ethan Ace, Published on Feb 01, 2012

In January 2012, analytic provider SightLogix announced its SightSensor intrusion detection analytics had been validated by the Transportation Safety Administration, providing "100% video analytics accuracy", a bold claim.  In this note, we look at this announcement, the TSA test, the integrator's experience and what users should look for when reviewing manufacturer test reports.

Overview 

While the report was prepared by the TSA, the evaluation of the SightLogix system was performed by the Safe Skies Alliance, which is a non-profit formed in 1997 to support testing of aviation security technology. The SSA receives funding from the FAA, and support from the TSA. This testing was performed at Buffalo International Airport in March 2011.

SightLogix has said that this report will be available to other airport security departments, but not to the public. While we understand the need for redaction, and keeping some of these measures secret, any number of other industries could benefit from this test, as well. As one commentor pointed out, though, it would take TSA involvement to distribute this to even industries outside of aviation, let alone the security industry at large.

Evaluating Claims

SightLogix's perimeter detection system is well respected and higher cost than many other options in the industry. It has also been developed for the specific purpose of intrusion detection, unlike many all-in-one analytic packages (see our review of their new 2012 camera series). Given this, we would not be surprised if detection rates are high, with false alarm rates low. However, users should be careful when reading this test report and press release, as some claims may be misleading.

  • Location and coverage information is redacted, so we're left with no knowledge of how wide the area covered is, nor are detection distances shared.
  • Intrusion techniques (the scenarios) were redacted, so we've no idea if subjects were detected while walking or crawling, from which direction they approached, at what speed, etc. Results for each of these scenarios were also redacted, so performance by intrusion type is unknown.
  • All nuisance and false alarm reports are redacted. It is extremely unlikely that any systems can be free of false and nuisance alarms, so this lack of detail removes information regarding potential system weaknesses.

Integrator Response

In a LinkedIn discussion regarding this release, one of our members who was involved with the integration shared the following clarification:

  • Regarding false alarms: "We had a number of alarms in the beginning and by a process of adjusmtments we were able to tune the system for the customer to an acceptable level. That is the key when installing an analytic system is setting the expectations of the customer correctly. 99% of the alarms are caused be animals and dependant on the size and quantity may or may not be a concern for the airport."
  • Regarding Nuisance alarms: "It does track objects that are not humans, but that was an added benefit to certain airport operations as birds, deer and other animals are not wanted guests on a runway. The biggest benefit to the system is that if there is an alarm it is automatically time stamped and logged, a monitor displays the target and the police officer can, by not touching anything, view to see if the alarm is something that requires attention."
  • Regarding performance: "I can say that the system after being in service for over a year is still providing a useful tool that is reliable and paying for itself."

He also stated that the system has been expanded to a second airport, bus garages, and a light rail system, which would lead us to believe the system is performing satisfactorily.

Conclusion

We found the integrator's insights to be far more useful than the study itself. A key factor was that the integrator was able to share experiences in the optimization stage and how it is used in daily operations. The lesson to take away - If you want a good reference, stay away from 'studies' and find an engineer who was involved in the details.

Comments : PRO Members only. Login. or Join.

Related Reports

IACP 2018 Police Show Final Report on Oct 08, 2018
IPVM went to Orlando to cover the 2018 IACP conference, the country's largest police show (about as big as ASIS), examining the 700+...
Fall 2018 New Products Directory on Oct 03, 2018
Fall 2018 New Products Directory We have compiled a directory of new products released fall 2018, or at least since our Spring 2018 Directory. The...
Alexa Guard Expands Amazon's Security Offerings, Boosts ADT's Stock on Sep 21, 2018
Amazon is expanding their security offerings yet again, this time with Alexa Guard that delivers security audio analytics and a virtual "Fake...
Avigilon Announces AI-Powered H5 Camera Development on Sep 19, 2018
Avigilon will be showcasing "next-generation AI" at next week's ASIS GSX. In an atypical move, the company is not actually releasing these...
Central Stations Face Off Against NFPA On Fire Monitoring on Sep 18, 2018
Central stations are facing off against the NFPA over what they call anti-competitive language in NFPA 72, the standard that covers fire alarms....
Ascent / MONI Faces Lender Lawsuit and Debt Crisis on Sep 13, 2018
ASCMA, aka Ascent, aka Brinks Home Security, aka MONI, aka Monitronics is being sued by a group of their lenders who allege: As of June 30,...
VMS Export Shootout - Avigilon, Dahua, Exacq, Genetec, Hikvision, Milestone on Sep 13, 2018
When crimes, accidents or problems occur, exporting video from one's video surveillance system is critical to proving incidents. But who does it...
US DARPA Investing $2 Billion In AI on Sep 11, 2018
The US Defense Advanced Research Projects Agency (DARPA) is granting more than $2 Billion to companies developing new AI technologies. The money...
ADT CEO Is Out on Sep 04, 2018
ADT's CEO Tim Whall is out, 'retiring' in less than 3 months, ADT announced. This comes amidst financial challenges, with the company's stock...
Why Vivint / Best Buy Failed on Aug 31, 2018
DIY has bested Vivint. In 2017, Best Buy and Vivint partnered with Vivint employees on the floor of 400+ Best Buy stores, helping customers with...

Most Recent Industry Reports

Higher Power PoE 802.3bt Ratified, Impact on Security Products Examined on Oct 12, 2018
Power over Ethernet has become one of the most popular features of many video, access, and other security products. See our PoE for IP Video...
"New Zealand Govt Uses Chinese Cameras Banned In US", Considers Security Audit on Oct 12, 2018
Newsroom NZ has issued a report: "NZ Govt uses Chinese cameras banned in US": This comes after the US federal government banned purchases of...
Mysterious Patent Troll 'Secure Cam' Targets Industry, Sues Hanwha, Hikvison, JCI, Panasonic, More on Oct 11, 2018
A company named "Secure Cam," who is actively hiding their ownership, has acquired a slew of video patents and is systematically suing video...
Unfixed Critical Vulnerability In Millions of XiongMai Devices Disclosed on Oct 10, 2018
XiongMai, one of the biggest OEMs alongside Dahua and Hikvision, has suffered a critical vulnerability impacting millions of their devices. This...
Door Hinges Guide on Oct 10, 2018
Some of the trickiest access control problems are caused by bad door hinges. From doors not closing right, to locks not locking, worn or warped...
Default Passwords Outlawed in California, US To Follow on Oct 09, 2018
UPDATES A new California bill aimed at improving security for connected devices has been signed into law. The law takes aim especially at...
Security System Health Monitoring Usage Statistics 2018 on Oct 09, 2018
How well and quickly do integrators know if devices are offline or broken? New IPVM statistics show that typically no health monitoring is...
IP Camera Installability Shootout - Avigilon, Axis, Bosch, Dahua, Hanwha, Hikvision, Uniview, Vivotek on Oct 08, 2018
What are the best and worst cameras from an installation standpoint? Which manufacturers make it harder or easier to install their cameras? We...
IACP 2018 Police Show Final Report on Oct 08, 2018
IPVM went to Orlando to cover the 2018 IACP conference, the country's largest police show (about as big as ASIS), examining the 700+...

The world's leading video surveillance information source, IPVM provides the best reporting, testing and training for 10,000+ members globally. Dedicated to independent and objective information, we uniquely refuse any and all advertisements, sponsorship and consulting from manufacturers.

About | FAQ | Contact