100% Video Analytics Accuracy?

By: Ethan Ace, Published on Feb 01, 2012

In January 2012, analytic provider SightLogix announced its SightSensor intrusion detection analytics had been validated by the Transportation Safety Administration, providing "100% video analytics accuracy", a bold claim.  In this note, we look at this announcement, the TSA test, the integrator's experience and what users should look for when reviewing manufacturer test reports.

Overview 

While the report was prepared by the TSA, the evaluation of the SightLogix system was performed by the Safe Skies Alliance, which is a non-profit formed in 1997 to support testing of aviation security technology. The SSA receives funding from the FAA, and support from the TSA. This testing was performed at Buffalo International Airport in March 2011.

SightLogix has said that this report will be available to other airport security departments, but not to the public. While we understand the need for redaction, and keeping some of these measures secret, any number of other industries could benefit from this test, as well. As one commentor pointed out, though, it would take TSA involvement to distribute this to even industries outside of aviation, let alone the security industry at large.

Evaluating Claims

SightLogix's perimeter detection system is well respected and higher cost than many other options in the industry. It has also been developed for the specific purpose of intrusion detection, unlike many all-in-one analytic packages (see our review of their new 2012 camera series). Given this, we would not be surprised if detection rates are high, with false alarm rates low. However, users should be careful when reading this test report and press release, as some claims may be misleading.

  • Location and coverage information is redacted, so we're left with no knowledge of how wide the area covered is, nor are detection distances shared.
  • Intrusion techniques (the scenarios) were redacted, so we've no idea if subjects were detected while walking or crawling, from which direction they approached, at what speed, etc. Results for each of these scenarios were also redacted, so performance by intrusion type is unknown.
  • All nuisance and false alarm reports are redacted. It is extremely unlikely that any systems can be free of false and nuisance alarms, so this lack of detail removes information regarding potential system weaknesses.

Integrator Response

In a LinkedIn discussion regarding this release, one of our members who was involved with the integration shared the following clarification:

  • Regarding false alarms: "We had a number of alarms in the beginning and by a process of adjusmtments we were able to tune the system for the customer to an acceptable level. That is the key when installing an analytic system is setting the expectations of the customer correctly. 99% of the alarms are caused be animals and dependant on the size and quantity may or may not be a concern for the airport."
  • Regarding Nuisance alarms: "It does track objects that are not humans, but that was an added benefit to certain airport operations as birds, deer and other animals are not wanted guests on a runway. The biggest benefit to the system is that if there is an alarm it is automatically time stamped and logged, a monitor displays the target and the police officer can, by not touching anything, view to see if the alarm is something that requires attention."
  • Regarding performance: "I can say that the system after being in service for over a year is still providing a useful tool that is reliable and paying for itself."

He also stated that the system has been expanded to a second airport, bus garages, and a light rail system, which would lead us to believe the system is performing satisfactorily.

Conclusion

We found the integrator's insights to be far more useful than the study itself. A key factor was that the integrator was able to share experiences in the optimization stage and how it is used in daily operations. The lesson to take away - If you want a good reference, stay away from 'studies' and find an engineer who was involved in the details.

Comments : PRO Members only. Login. or Join.

Related Reports

Anyvision Facial Recognition Tested on Aug 21, 2019
Anyvision is aiming for $1 billion in revenue by 2022, backed by $74 million in funding. But does their performance live up to the hype they have...
Verkada People And Face Analytics Tested on Aug 16, 2019
This week, Verkada released "People Analytics", including face analytics that they describe is a "game-changing feature" that "pushes the...
Dahua Analytics+ Tested on Aug 07, 2019
Dahua's analytics have performed poorly in past shootouts. But now, they claim their new Analytics+ "algorithms significantly improve accuracy and...
Honeywell Speaks On NDAA Ban, New Non-Banned Cameras and Cybersecurity on Aug 06, 2019
For years, Honeywell has depended on Dahua, a company with a poor cybersecurity track record and now banned by the US NDAA, for the development and...
Hikvision 4K Camera Shootout on Aug 02, 2019
With their latest Smart Series 5 cameras, Hikvision is claiming cameras "fully loaded" with "state-of-the-art technology for high performance and...
History of Video Surveillance on Jul 19, 2019
The video surveillance market has changed significantly since 2000, going from VCRs to ab emerging AI cloud era.  The goal of this history is to...
Wyze AI Analytics Tested - Beats Axis and Hikvision on Jul 17, 2019
$20 camera disruptor Wyze has released free person detection deep learning analytics to all of their users, claiming users will "Only get notified...
Ivideon Russian VSaaS Profile on Jun 27, 2019
Ivideon was an early VSaaS entrant, initially focusing on the consumer market, claiming massive growth to IPVM in 2014. We spoke to Ivideon, to...
Directory of 60 Video Surveillance Startups on Jun 25, 2019
This directory provides a list of video surveillance startups to help you see and research what companies are new or not yet broadly known. 2019...
Repositionable Multi-Imager Camera Shootout - Avigilon, Axis, Dahua, Hanwha, Hikvision, Panasonic, Vivotek on Jun 19, 2019
Repositionable multi-imager cameras are one of the fastest growing segments in video surveillance, with a slew of new offerings being recently...

Most Recent Industry Reports

TMA Apologizes to Amazon / Ring on Aug 23, 2019
Not only is Amazon / Ring making major incursions into the residential security market, the organization representing the biggest incumbents, The...
China Dahua Replaces Their Software With US Pepper on Aug 22, 2019
What does a US government banned company do to improve its security positioning in the US? Well, Dahua is unveiling a novel solution, partnering...
Security Integrators Outlook On Remaining Integrators In 2025 on Aug 22, 2019
The industry has changed substantially in the last decade, with the rise of IP cameras and the race to the bottom. Indeed, more changes may be...
First GDPR Facial Recognition Fine For Sweden School on Aug 22, 2019
A school in Sweden has been fined $20,000 for using facial recognition to keep attendance in what is Sweden's first GDPR fine. Notably, the fine is...
Anyvision Facial Recognition Tested on Aug 21, 2019
Anyvision is aiming for $1 billion in revenue by 2022, backed by $74 million in funding. But does their performance live up to the hype they have...
JCI Sues Wyze on Aug 21, 2019
The mega manufacturer / integrator JCI has sued the fast-growing $20 camera Seattle startup Wyze. Inside this note: Share the court...
Dahua 4K Camera Shootout on Aug 20, 2019
Dahua's new Pro Series 4K N85CL5Z claims to "deliver superior images in all lighting and environmental conditions", but how does this compare to...
ZK Teco Atlas Access Control Tested on Aug 20, 2019
Who needs access specialists? China-based ZKTeco claims its newest access panel 'makes it very easy for anyone to learn and install access control...
Uniview Beats Intel In Trademark Lawsuit on Aug 19, 2019
Uniview has won a long-running trademark lawsuit brought by Intel, with Beijing's highest court reversing an earlier Intel win, centered on...
Suprema Biometric Mass Leak Examined on Aug 19, 2019
While Suprema is rarely discussed even within the physical security market, the South Korean biometrics manufacturer made global news this past...

The world's leading video surveillance information source, IPVM provides the best reporting, testing and training for 10,000+ members globally. Dedicated to independent and objective information, we uniquely refuse any and all advertisements, sponsorship and consulting from manufacturers.

About | FAQ | Contact