100% Video Analytics Accuracy?

By: Ethan Ace, Published on Feb 01, 2012

In January 2012, analytic provider SightLogix announced its SightSensor intrusion detection analytics had been validated by the Transportation Safety Administration, providing "100% video analytics accuracy", a bold claim.  In this note, we look at this announcement, the TSA test, the integrator's experience and what users should look for when reviewing manufacturer test reports.

Overview 

While the report was prepared by the TSA, the evaluation of the SightLogix system was performed by the Safe Skies Alliance, which is a non-profit formed in 1997 to support testing of aviation security technology. The SSA receives funding from the FAA, and support from the TSA. This testing was performed at Buffalo International Airport in March 2011.

SightLogix has said that this report will be available to other airport security departments, but not to the public. While we understand the need for redaction, and keeping some of these measures secret, any number of other industries could benefit from this test, as well. As one commentor pointed out, though, it would take TSA involvement to distribute this to even industries outside of aviation, let alone the security industry at large.

Evaluating Claims

SightLogix's perimeter detection system is well respected and higher cost than many other options in the industry. It has also been developed for the specific purpose of intrusion detection, unlike many all-in-one analytic packages (see our review of their new 2012 camera series). Given this, we would not be surprised if detection rates are high, with false alarm rates low. However, users should be careful when reading this test report and press release, as some claims may be misleading.

  • Location and coverage information is redacted, so we're left with no knowledge of how wide the area covered is, nor are detection distances shared.
  • Intrusion techniques (the scenarios) were redacted, so we've no idea if subjects were detected while walking or crawling, from which direction they approached, at what speed, etc. Results for each of these scenarios were also redacted, so performance by intrusion type is unknown.
  • All nuisance and false alarm reports are redacted. It is extremely unlikely that any systems can be free of false and nuisance alarms, so this lack of detail removes information regarding potential system weaknesses.

Integrator Response

In a LinkedIn discussion regarding this release, one of our members who was involved with the integration shared the following clarification:

  • Regarding false alarms: "We had a number of alarms in the beginning and by a process of adjusmtments we were able to tune the system for the customer to an acceptable level. That is the key when installing an analytic system is setting the expectations of the customer correctly. 99% of the alarms are caused be animals and dependant on the size and quantity may or may not be a concern for the airport."
  • Regarding Nuisance alarms: "It does track objects that are not humans, but that was an added benefit to certain airport operations as birds, deer and other animals are not wanted guests on a runway. The biggest benefit to the system is that if there is an alarm it is automatically time stamped and logged, a monitor displays the target and the police officer can, by not touching anything, view to see if the alarm is something that requires attention."
  • Regarding performance: "I can say that the system after being in service for over a year is still providing a useful tool that is reliable and paying for itself."

He also stated that the system has been expanded to a second airport, bus garages, and a light rail system, which would lead us to believe the system is performing satisfactorily.

Conclusion

We found the integrator's insights to be far more useful than the study itself. A key factor was that the integrator was able to share experiences in the optimization stage and how it is used in daily operations. The lesson to take away - If you want a good reference, stay away from 'studies' and find an engineer who was involved in the details.

Comments : PRO Members only. Login. or Join.

Related Reports

Repositionable Multi-Imager Camera Shootout - Avigilon, Axis, Dahua, Hanwha, Hikvision, Panasonic, Vivotek on Jun 19, 2019
Repositionable multi-imager cameras are one of the fastest growing segments in video surveillance, with a slew of new offerings being recently...
Startup Vaion Launching End-to-End AI Solution Backed with $20 Million Funding on Jun 17, 2019
An EU / USA video surveillance startup, Vaion, founded by ex-Cisco Senior Directors is launching an end-to-end VSaaS platform with $20 million in...
Sighthound Transforms Into Enterprise AI Provider Profile on Jun 14, 2019
Sighthound is now rapidly expanding its R&D team, building an enterprise AI service. This may come as a surprise given their origins 6 years...
ADT Eliminating Acquired Brands, Unifying Under 'Commercial' Brand on Jun 14, 2019
ADT is eliminating the brands of the many integrators it has acquired over the past few years, including Red Hawk, Aronson Security Group (ASG),...
Embattled $400 Million China Funded Philippines Surveillance System Proceeds on Jun 13, 2019
An embattled 12,000 camera surveillance system project that will cost ~$400 million will proceed.  The project contract was awarded, had its...
Carnegie Mellon AI Startup Zensors Profile on Jun 11, 2019
Zensors is a startup formed by Carnegie Mellon graduates from a Carnegie Mellon research project, offering customized models per camera that they...
Dumber Techs, Bad Box Movers, Says Australian Distributor on Jun 10, 2019
Techs today are "dumber" than they used to be, despite better education and training and that makes a typical day "frustrating" for one...
Directory of 30+ VSaaS / Cloud Video Surveillance Providers on Jun 07, 2019
This directory provides a list of VSaaS / cloud video surveillance providers to help you see and research what options are available. 2019 State...
Hikvision DeepinMind 2019 Test on Jun 06, 2019
In 2018, Hikvision's DeepinMind AI NVR performed terribly, recognizing vehicles, animals, and other objects as humans, misclassifying demographics,...
Startup Rhombus Systems Says Twice the Features, Half the Price of Verkada on Jun 04, 2019
Closed cloud systems may be the fastest growing segment of video surveillance with Meraki and Verkada. Now another California company is joining...

Most Recent Industry Reports

Repositionable Multi-Imager Camera Shootout - Avigilon, Axis, Dahua, Hanwha, Hikvision, Panasonic, Vivotek on Jun 19, 2019
Repositionable multi-imager cameras are one of the fastest growing segments in video surveillance, with a slew of new offerings being recently...
Genetec Synergis Cloud Link - Complex, Costly and Confusing on Jun 18, 2019
Genetec's Synergis Cloud Link is complex, costly and confusing compared to competitor access control architectures. Inside this note, we examine...
Startup Vaion Launching End-to-End AI Solution Backed with $20 Million Funding on Jun 17, 2019
An EU / USA video surveillance startup, Vaion, founded by ex-Cisco Senior Directors is launching an end-to-end VSaaS platform with $20 million in...
Biometrics Usage Statistics 2019 on Jun 17, 2019
While face and fingerprint recognition are used regularly for smartphones, it is not as common in physical security. In this note, we examine...
The Scheme Hikvision and China Importers Use To Avoid Tariffs on Jun 17, 2019
Hikvision and numerous China importers are avoiding 25% tariffs by including an SD card slot in their IP cameras to claim they are 'digital still...
Sighthound Transforms Into Enterprise AI Provider Profile on Jun 14, 2019
Sighthound is now rapidly expanding its R&D team, building an enterprise AI service. This may come as a surprise given their origins 6 years...
ADT Eliminating Acquired Brands, Unifying Under 'Commercial' Brand on Jun 14, 2019
ADT is eliminating the brands of the many integrators it has acquired over the past few years, including Red Hawk, Aronson Security Group (ASG),...
NSA Director Keynoting Dahua and Hikvision Sponsored Cybersecurity Conference [Canceled] on Jun 13, 2019
The technical director for the NSA’s Cybersecurity Threat Operations Center will be keynoting a physical security cybersecurity conference that is...
Farpointe Data Conekt Mobile Access Reader Tested on Jun 13, 2019
California based Farpointe Data has been a significant OEM supplier of conventional access readers for years to companies including DMP, RS2, DSX,...
Embattled $400 Million China Funded Philippines Surveillance System Proceeds on Jun 13, 2019
An embattled 12,000 camera surveillance system project that will cost ~$400 million will proceed.  The project contract was awarded, had its...

The world's leading video surveillance information source, IPVM provides the best reporting, testing and training for 10,000+ members globally. Dedicated to independent and objective information, we uniquely refuse any and all advertisements, sponsorship and consulting from manufacturers.

About | FAQ | Contact