100% Video Analytics Accuracy?

By Ethan Ace, Published on Feb 01, 2012

In January 2012, analytic provider SightLogix announced its SightSensor intrusion detection analytics had been validated by the Transportation Safety Administration, providing "100% video analytics accuracy", a bold claim.  In this note, we look at this announcement, the TSA test, the integrator's experience and what users should look for when reviewing manufacturer test reports.

Overview 

While the report [link no longer available] was prepared by the TSA, the evaluation of the SightLogix system was performed by the Safe Skies Alliance, which is a non-profit formed in 1997 to support testing of aviation security technology. The SSA receives funding from the FAA, and support from the TSA. This testing was performed at Buffalo International Airport in March 2011.

SightLogix has said that this report will be available to other airport security departments, but not to the public. While we understand the need for redaction, and keeping some of these measures secret, any number of other industries could benefit from this test, as well. As one commentor pointed out, though, it would take TSA involvement to distribute this to even industries outside of aviation, let alone the security industry at large.

Evaluating Claims

SightLogix's perimeter detection system is well respected and higher cost than many other options in the industry. It has also been developed for the specific purpose of intrusion detection, unlike many all-in-one analytic packages (see our review of their new 2012 camera series). Given this, we would not be surprised if detection rates are high, with false alarm rates low. However, users should be careful when reading this test report and press release, as some claims may be misleading.

  • Location and coverage information is redacted, so we're left with no knowledge of how wide the area covered is, nor are detection distances shared.
  • Intrusion techniques (the scenarios) were redacted, so we've no idea if subjects were detected while walking or crawling, from which direction they approached, at what speed, etc. Results for each of these scenarios were also redacted, so performance by intrusion type is unknown.
  • All nuisance and false alarm reports are redacted. It is extremely unlikely that any systems can be free of false and nuisance alarms, so this lack of detail removes information regarding potential system weaknesses.

Integrator Response

In a LinkedIn discussion [link no longer available] regarding this release, one of our members who was involved with the integration shared the following clarification:

  • Regarding false alarms: "We had a number of alarms in the beginning and by a process of adjusmtments we were able to tune the system for the customer to an acceptable level. That is the key when installing an analytic system is setting the expectations of the customer correctly. 99% of the alarms are caused be animals and dependant on the size and quantity may or may not be a concern for the airport."
  • Regarding Nuisance alarms: "It does track objects that are not humans, but that was an added benefit to certain airport operations as birds, deer and other animals are not wanted guests on a runway. The biggest benefit to the system is that if there is an alarm it is automatically time stamped and logged, a monitor displays the target and the police officer can, by not touching anything, view to see if the alarm is something that requires attention."
  • Regarding performance: "I can say that the system after being in service for over a year is still providing a useful tool that is reliable and paying for itself."

He also stated that the system has been expanded to a second airport, bus garages, and a light rail system, which would lead us to believe the system is performing satisfactorily.

Conclusion

We found the integrator's insights to be far more useful than the study itself. A key factor was that the integrator was able to share experiences in the optimization stage and how it is used in daily operations. The lesson to take away - If you want a good reference, stay away from 'studies' and find an engineer who was involved in the details.

Comments : Members only. Login. or Join.

Related Reports

Video Analytics 101 on Mar 16, 2020
This guide teaches the fundamentals of video surveillance...
Euklis Presents AI Analytics on May 05, 2020
Euklis presented its AI facial recognition, LPR, and object recognition...
Hanwha Presents AI Analytics and First 8K Camera on May 11, 2020
Hanwha presented their AI Analytics and the "Industry's First 8K" camera at...
Favorite Video Analytic Manufacturers 2020 on Feb 25, 2020
Video analytics is now as hot as ever, driven by the excitement of advancing...
Bosch / Milestone Forensic Search Tested on Jan 08, 2020
Bosch's Forensic Search Milestone plugin integrates Bosch IVA and EVA...
BriefCam Video Analytics Tested on Jan 06, 2020
BriefCam, acquired by Canon in 2018, is one of the most commonly used video...
Video Surveillance Cameras 101 on Feb 25, 2020
Cameras come in many shapes, sizes and specifications. This 101 examines the...
SafeZone Tech Presents AI Gunfire Detection on Jun 15, 2020
Safe Zone presented its AI gunfire sensor the May 2020 IPVM Startups...
VSaaS 101 on Mar 25, 2020
Video Surveillance as a Service (VSaaS) is the common industry term for cloud...
Seek Scan Thermal Temperature Screening System Tested on May 28, 2020
Now that IPVM has tested Dahua, Hikvision, and Sunell, we are returning to...
ZeroEyes Presents Firearm Detection Video Analytics on Jul 09, 2020
ZeroEyes presented its Firearm detection Video Analytics system at the May...
Anyvision Layoffs on Mar 19, 2020
Anyvision has conducted a layoff, citing the impact of coronavirus, joining a...
Axxon Presents VMS 4.4 and AI Behavior Analytics on May 20, 2020
AxxonSoft presented its VMS 4.4 and AI behavior analytics at the April 2020...
Vivotek Presents AI Analytics and LPR on May 19, 2020
Vivotek presented its AI Analytics and LPR at the April 2020 IPVM New...
Video Surveillance Business 101 on Mar 30, 2020
This report explains the fundamental elements of the video surveillance...

Recent Reports

Recruiters Online Show LIVE Today! on Oct 29, 2020
IPVM's 7th online show resumes today with 12 recruiters presenting themselves...
Hikvision AcuSense G2 Camera Test on Oct 29, 2020
Hikvision has released their next generation of AcuSense analytic cameras...
Biggest Problems Selling Access Control 2020 on Oct 29, 2020
Access control can cause integrators big headaches. What practical issues do...
Taiwan Geovision AI Analytics and NDAA Examined on Oct 29, 2020
Taiwan manufacturer Geovision's revenue has been falling for years. However,...
Bedside Cough and Sneeze Detector (Sound Intelligence and CLB) on Oct 28, 2020
Coronavirus has increased interest in detecting symptoms such as fever and...
Fever Tablet Thermal Sensors Examined (Melexis) on Oct 28, 2020
Fever tablet suppliers heavily rely on the accuracy and specs of...
Verkada Fires 3 on Oct 28, 2020
Verkada has fired three employees over an incident where female colleagues...
Eagle Eye Networks Raises $40 Million on Oct 27, 2020
Eagle Eye has raised $40 million aiming to "reinvent video...
Hikvision Q3 2020 Global Revenue Rises, US Revenue Falls on Oct 27, 2020
While Hikvision's global revenue rises driven by domestic recovery, its US...
VICE Investigates Verkada's Harassing "RawVerkadawgz" on Oct 26, 2020
This month, IPVM investigated Verkada's sexism, discrimination, and cultural...
Six Flags' FDA Violating Outdoor Dahua Fever Cameras on Oct 26, 2020
As Six Flags scrambled to reopen parks amid plummeting revenues caused by the...
ISC Brasil Digital Experience 2020 Report on Oct 23, 2020
ISC Brasil 2020 rebranded itself to ISC Digital Experience and, like its...
Top Video Surveillance Service Call Problems 2020 on Oct 23, 2020
3 primary and 4 secondary issues stood out as causing the most problems when...
GDPR Impact On Temperature / Fever Screening Explained on Oct 22, 2020
What impact does GDPR have on temperature screening? Do you risk a GDPR fine...
Security And Safety Things (S&ST) Tested on Oct 22, 2020
S&ST, a Bosch spinout, is spending tens of millions of dollars aiming to...