Facial Surveillance Used On US Politicians, Technical Challenges Remain

Author: John Honovich, Published on Jun 12, 2018

Many fear that facial surveillance will be used against people by governments, such as the vast facial surveillance apparatus being built in China as well as the ACLU's complaints about US pilots.

Now, the New York Times has built their own facial surveillance system that they are using on US politicians as they seek to report on political issues in the US capital.

Inside this note, we examine the NY Times system, including problems they have faced, in particular with facial recognition matching issues.

Facial Recognition of US Congresspeople

As explained here, "How The New York Times Uses Software To Recognize Members of Congress", the NY Times built an application on top of Amazon's Rekognition service, which lets reporters take a picture of potential Congresspeople with their phones, text it to their service, let Amazon run facial recognition on it, returning potential face matches, as the demo image below shows:

The NY Times need was that, with 500+ Congresspeople, it is hard for every reporter to remember the face and name of every Congressperson, so that they could use this service to identify Congresspeople to help in pursuing stories. For example, they credited this application for their "Trump Groups Raised Millions, Then Paid It Out to Loyalists and a Trump Hotel" article.

Open Sourced

The NY Times has open sourced the application (see Github: Who The Hill: An MMS-based facial recognition service for members of Congress).

Amazon Rekognition Central Role

Get Video Surveillance News In Your Inbox
Get Video Surveillance News In Your Inbox

Like the ACLU case recently showed, Amazon's Rekognition service plays a central role in this application. Amazon makes it relatively cheap and easy to build your own 'DIY' facial surveillance system, as there are no expensive license fees up front, you pay as you go and it offers well-documented APIs for integration.

Moreover, Amazon offers a specific API for recognizing celebrities. This means no work needs to be done by the user for this as tens of thousands of celebrities (actors, politicians, etc.) have been pre-indexed and trained by the Amazon system.

Indeed, Amazon offers a demo of the celebrity recognition service (for those with an AWS account). However, it is not always accurate as this IPVM experiment showed:

Doppelgangers / False Match Issues

Indeed, the NY Times faced a more serious false matching issue. They found certain false matches so significant that they had to hardcode their application with a 'doppelganger' feature to exclude known look-alikes. Below is the list with a sample of a few of the common false matches (Congressperson vs Doppelganger):

Here is another one, politician vs actor:

Problems With Facial Recognition

The problem is, fundamentally, lots of people look alike each other, especially when you get into larger populations of people (retail, public facilities, etc.) and not small offices.

Moreover, as the NY Times themselves learned in their application's usage:

our hit rate on photographs was very low because the halls of the Capitol are poorly lit and the photographs we took for testing were consistently marred by shadow and blur. [emphasis added]

That type of lamentation should be familiar to experienced video surveillance users where such conditions are commonplace.

On The Rise But Problems Too

On the plus side for facial recognition, this example of Amazon usage shows facial surveillance getting easier to do. On the negative side, these results also show the need to temper expectations about accuracy. For the NY Times use case where a reporter will only be periodically submitting requests and manually reviewing responses (looking just for Congresspeople), they should be able to fairly and quickly assess false matches. To the contrary, such results on a real-time video surveillance system would likely cause an overload of false alerts.

Comments (2) : PRO Members only. Login. or Join.

Related Reports on Facial Recognition

Video Analytics Integration Guide on Aug 16, 2018
Video analytics is hot again (at least conceptually) but integrating video analytics with VMSes can be challenging. This is especially significant...
RealNetworks Free School Facial Recognition on Aug 03, 2018
The company that created RealPlayer is moving beyond media delivery and into the security space with a new facial recognition platform they have...
99.9997% Accurate Amazon Facial Rekognition Falsely Matches 28 on Jul 27, 2018
A new facial recognition controversy has arisen, with the ACLU cleverly using Amazon's Rekognition service to falsely match 28 Congresspeople to...
AI Startup Anyvision Raises $28 Million Led By Bosch on Jul 20, 2018
Anyvision is the most ambitious heavy-spending video surveillance startup in many years. And, now, the startup has raised $28 million led by...
FST Fails on Jul 17, 2018
FST was one of the hottest startups of the decade, selected as the best new product at ISC West 2011 and backed with tens of millions in...
Hikvision Wins Chinese Government Forced Facial Recognition Project Across 967 Mosques on Jul 16, 2018
Hikvision has won a Chinese government tender which requires that facial recognition cameras be set up at the entrance of every single mosque...
Belgium Bans Private Facial Surveillance on Jul 06, 2018
Belgium has effectively banned the use of facial recognition and other biometrics-based video analytics in surveillance cameras for private,...
GDPR / ICO Complaint Filed Against IFSEC Show Facial Recognition on Jun 20, 2018
IPVM has filed a complaint against IFSEC’s parent company UBM based on our concern that the conference violates core GDPR principles on...
IFSEC 2018 Final Show Report on Jun 20, 2018
IPVM attended the IFSEC show for the first time this year. The Chinese took over the show, centered on Hikvision, flanked by Dahua, Huawei and a...
China Public Video Surveillance Guide: From Skynet to Sharp Eyes on Jun 14, 2018
China is expanding its video surveillance network to achieve “100%” nationwide coverage by 2020, including facial recognition capabilities and a...

Most Recent Industry Reports

Backup Power For Maglocks Guide on Aug 20, 2018
When the main power fails, many believe maglocks must leave doors unlocked. However, battery backed up maglocks are allowed according to IBC /...
Axis Strong Q2 2018 Results on Aug 20, 2018
Axis reported strong Q2 2018 financial results and, for the first time in many years, Axis grew faster than Hikvision. However, the company's...
2Gig Gun Lock / Motion Detector Tested on Aug 17, 2018
Safer guns for families and an opportunity for security dealers to sell more services? That is the aim of Nortek's 2GIG 'Gun Motion Detector'...
Video Analytics Integration Guide on Aug 16, 2018
Video analytics is hot again (at least conceptually) but integrating video analytics with VMSes can be challenging. This is especially significant...
Hikvision IP Camera Critical Vulnerability 2018 Disclosed on Aug 16, 2018
The same day that the US government passed a prohibition on Hikvision cameras, Hikvision disclosed a critical vulnerability for its IP...
ISS VMS / Video Analytics Company Profile on Aug 16, 2018
Who is ISS? In the past few months, they had one of the craziest ISC West promo items in years. Then, they hired industry veteran and ex-Dahua...
Chinese OEM Avycon Gets ADI Push on Aug 15, 2018
Who is Avycon? An American company? A Korean company? A couple of guys relabelling Chinese products? The latter is the best explanation. While...
Backboxes for Video Surveillance Tutorial on Aug 15, 2018
Backboxes are a necessity in surveillance, whether for managing cable whips, recessing cameras, adding wireless radios. But it can be confusing to...
Genetec Stratocast / Comcast 'Motion Insights' Examined on Aug 15, 2018
Comcast recently announced "SmartOffice Motion Insights", an extension to their Genetec OEMed cloud video service (covered by IPVM here). This...

The world's leading video surveillance information source, IPVM provides the best reporting, testing and training for 10,000+ members globally. Dedicated to independent and objective information, we uniquely refuse any and all advertisements, sponsorship and consulting from manufacturers.

About | FAQ | Contact