100% Video Analytics Accuracy?

By: Ethan Ace, Published on Feb 01, 2012

In January 2012, analytic provider SightLogix announced its SightSensor intrusion detection analytics had been validated by the Transportation Safety Administration, providing "100% video analytics accuracy", a bold claim.  In this note, we look at this announcement, the TSA test, the integrator's experience and what users should look for when reviewing manufacturer test reports.

Overview 

While the report [link no longer available] was prepared by the TSA, the evaluation of the SightLogix system was performed by the Safe Skies Alliance, which is a non-profit formed in 1997 to support testing of aviation security technology. The SSA receives funding from the FAA, and support from the TSA. This testing was performed at Buffalo International Airport in March 2011.

SightLogix has said that this report will be available to other airport security departments, but not to the public. While we understand the need for redaction, and keeping some of these measures secret, any number of other industries could benefit from this test, as well. As one commentor pointed out, though, it would take TSA involvement to distribute this to even industries outside of aviation, let alone the security industry at large.

Evaluating Claims

SightLogix's perimeter detection system is well respected and higher cost than many other options in the industry. It has also been developed for the specific purpose of intrusion detection, unlike many all-in-one analytic packages (see our review of their new 2012 camera series). Given this, we would not be surprised if detection rates are high, with false alarm rates low. However, users should be careful when reading this test report and press release, as some claims may be misleading.

  • Location and coverage information is redacted, so we're left with no knowledge of how wide the area covered is, nor are detection distances shared.
  • Intrusion techniques (the scenarios) were redacted, so we've no idea if subjects were detected while walking or crawling, from which direction they approached, at what speed, etc. Results for each of these scenarios were also redacted, so performance by intrusion type is unknown.
  • All nuisance and false alarm reports are redacted. It is extremely unlikely that any systems can be free of false and nuisance alarms, so this lack of detail removes information regarding potential system weaknesses.

Integrator Response

In a LinkedIn discussion [link no longer available] regarding this release, one of our members who was involved with the integration shared the following clarification:

  • Regarding false alarms: "We had a number of alarms in the beginning and by a process of adjusmtments we were able to tune the system for the customer to an acceptable level. That is the key when installing an analytic system is setting the expectations of the customer correctly. 99% of the alarms are caused be animals and dependant on the size and quantity may or may not be a concern for the airport."
  • Regarding Nuisance alarms: "It does track objects that are not humans, but that was an added benefit to certain airport operations as birds, deer and other animals are not wanted guests on a runway. The biggest benefit to the system is that if there is an alarm it is automatically time stamped and logged, a monitor displays the target and the police officer can, by not touching anything, view to see if the alarm is something that requires attention."
  • Regarding performance: "I can say that the system after being in service for over a year is still providing a useful tool that is reliable and paying for itself."

He also stated that the system has been expanded to a second airport, bus garages, and a light rail system, which would lead us to believe the system is performing satisfactorily.

Conclusion

We found the integrator's insights to be far more useful than the study itself. A key factor was that the integrator was able to share experiences in the optimization stage and how it is used in daily operations. The lesson to take away - If you want a good reference, stay away from 'studies' and find an engineer who was involved in the details.

Comments : Members only. Login. or Join.

Related Reports

Euklis Presents AI Analytics on May 05, 2020
Euklis presented its AI facial recognition, LPR, and object recognition...
ISS Presents Face As A Credential and UVSS on Apr 30, 2020
ISS presented its security platform, including access control integration,...
SAFR Presents AI Facial Recognition on May 18, 2020
SAFR presented its AI facial recognition at the April 2020 IPVM New Products...
Seek Scan Thermal Temperature Screening System Tested on May 28, 2020
Now that IPVM has tested Dahua, Hikvision, and Sunell, we are returning to...
Avigilon ACC Cloud Tested on Jul 08, 2020
Avigilon merged Blue and ACC, adding VSaaS features to its on-premise VMS,...
Anyvision Layoffs on Mar 19, 2020
Anyvision has conducted a layoff, citing the impact of coronavirus, joining a...
Hanwha Presents AI Analytics and First 8K Camera on May 11, 2020
Hanwha presented their AI Analytics and the "Industry's First 8K" camera at...
Avigilon Presents Cloud Connected ACC on May 13, 2020
Avigilon presented Avigilon Cloud Services, which supports cloud-connected...
VSaaS 101 on Mar 25, 2020
Video Surveillance as a Service (VSaaS) is the common industry term for cloud...
Milestone Presents XProtect On AWS on May 04, 2020
Milestone presented its XProtect on AWS offering at the April 2020 IPVM New...
SafeZone Tech Presents AI Gunfire Detection on Jun 15, 2020
Safe Zone presented its AI gunfire sensor the May 2020 IPVM Startups...
Meditherm Medical Thermal Cameras Examined on May 07, 2020
US-based Meditherm emphasized to IPVM its 20+ year development history and...
Milestone Launches Multiple Cloud Solutions on Feb 18, 2020
Milestone is going to the cloud, becoming one of the last prominent VMSes to...
Video Analytics 101 on Mar 16, 2020
This guide teaches the fundamentals of video surveillance...
Omnisense "Mass Fever Screening and Pandemic Control System" Examined on Jun 11, 2020
Omnisense Systems is marketing a "Mass Fever Screening and Pandemic Control"...

Recent Reports

Dangerous Hikvision Fever Camera Showcased by Chilean City on Aug 07, 2020
Deploying a fever camera outdoors, in the rain, with no black body, is...
"Grand Slam" For Pelco's PE Firm, A Risk For Motorola on Aug 07, 2020
The word "Pelco" and "grand slam" have not been said together for many years....
FLIR Stock Falls, Admits 'Decelerating' Demand For Temperature Screening on Aug 07, 2020
Is the boom going to bust for temperature screening? FLIR disappointed...
VSaaS Will Hurt Integrators on Aug 06, 2020
VSaaS will hurt integrators, there is no question about that. How much...
Dogs For Coronavirus Screening Examined on Aug 06, 2020
While thermal temperature screening is the surveillance industry's most...
ADT Slides Back, Disappointing Results, Poor Commercial Performance on Aug 06, 2020
While ADT had an incredible start to the week, driven by the Google...
AHJ / Authority Having Jurisdiction Tutorial on Aug 06, 2020
One of the most powerful yet often underappreciated characters in all of the...
SIA Coaches Sellers on NDAA 889B Blacklist Workarounds on Aug 05, 2020
Last month SIA demanded that NDAA 899B "must be delayed". Now that they have...
ADI Returns To Growth, Back To 'Pre-COVID Levels' on Aug 05, 2020
While ADI was hit hard in April, with revenue declining 21%, the company's...
Exposing Fever Tablet Suppliers and 40+ Relabelers on Aug 05, 2020
IPVM has found 40+ USA and EU companies relabeling fever tablets designed,...
Directory of 201 "Fever" Camera Suppliers on Aug 04, 2020
This directory provides a list of "Fever" scanning thermal camera providers...
Face Masks Increase Face Recognition Errors Says NIST on Aug 04, 2020
COVID-19 has led to widespread facemask use, which as IPVM testing has shown...
Dahua Loses Australian Medical Device Approval on Aug 04, 2020
Dahua has cancelled its medical device registration after "discussions" with...
Google Invests in ADT, ADT Stock Soars on Aug 03, 2020
Google has announced a $450 million investment in the Florida-based security...