Facial Surveillance Used On US Politicians, Technical Challenges Remain

Author: John Honovich, Published on Jun 12, 2018

Many fear that facial surveillance will be used against people by governments, such as the vast facial surveillance apparatus being built in China as well as the ACLU's complaints about US pilots.

Now, the New York Times has built their own facial surveillance system that they are using on US politicians as they seek to report on political issues in the US capital.

Inside this note, we examine the NY Times system, including problems they have faced, in particular with facial recognition matching issues.

Facial Recognition of US Congresspeople

As explained here, "How The New York Times Uses Software To Recognize Members of Congress", the NY Times built an application on top of Amazon's Rekognition service, which lets reporters take a picture of potential Congresspeople with their phones, text it to their service, let Amazon run facial recognition on it, returning potential face matches, as the demo image below shows:

The NY Times need was that, with 500+ Congresspeople, it is hard for every reporter to remember the face and name of every Congressperson, so that they could use this service to identify Congresspeople to help in pursuing stories. For example, they credited this application for their "Trump Groups Raised Millions, Then Paid It Out to Loyalists and a Trump Hotel" article.

Open Sourced

The NY Times has open sourced the application (see Github: Who The Hill: An MMS-based facial recognition service for members of Congress).

Amazon Rekognition Central Role

Get Video Surveillance News In Your Inbox
Get Video Surveillance News In Your Inbox

Like the ACLU case recently showed, Amazon's Rekognition service plays a central role in this application. Amazon makes it relatively cheap and easy to build your own 'DIY' facial surveillance system, as there are no expensive license fees up front, you pay as you go and it offers well-documented APIs for integration.

Moreover, Amazon offers a specific API for recognizing celebrities. This means no work needs to be done by the user for this as tens of thousands of celebrities (actors, politicians, etc.) have been pre-indexed and trained by the Amazon system.

Indeed, Amazon offers a demo of the celebrity recognition service (for those with an AWS account). However, it is not always accurate as this IPVM experiment showed:

Doppelgangers / False Match Issues

Indeed, the NY Times faced a more serious false matching issue. They found certain false matches so significant that they had to hardcode their application with a 'doppelganger' feature to exclude known look-alikes. Below is the list with a sample of a few of the common false matches (Congressperson vs Doppelganger):

Here is another one, politician vs actor:

Problems With Facial Recognition

The problem is, fundamentally, lots of people look alike each other, especially when you get into larger populations of people (retail, public facilities, etc.) and not small offices.

Moreover, as the NY Times themselves learned in their application's usage:

our hit rate on photographs was very low because the halls of the Capitol are poorly lit and the photographs we took for testing were consistently marred by shadow and blur. [emphasis added]

That type of lamentation should be familiar to experienced video surveillance users where such conditions are commonplace.

On The Rise But Problems Too

On the plus side for facial recognition, this example of Amazon usage shows facial surveillance getting easier to do. On the negative side, these results also show the need to temper expectations about accuracy. For the NY Times use case where a reporter will only be periodically submitting requests and manually reviewing responses (looking just for Congresspeople), they should be able to fairly and quickly assess false matches. To the contrary, such results on a real-time video surveillance system would likely cause an overload of false alerts.

Comments (2) : PRO Members only. Login. or Join.

Related Reports on Facial Recognition

Massive Leak Of Chinese VMS Provider Exposes Xinjiang Surveillance on Feb 20, 2019
A subsidiary of China’s claimed largest VMS provider is tracking the precise location and ethnicity of millions in China’s Xinjiang region,...
Casino Surveillance Pro Interview: James Lathrop on Feb 15, 2019
James Lathrop has been working in casinos for almost 25 years. During that time, he says he has held "just about every job you can do in the...
Intersec 2019 Show Report on Jan 23, 2019
The 2019 Intersec show, held annually in Dubai, is now complete. IPVM attended for 3 days, interviewing numerous Chinese and Western video...
NYPD Refutes False SCMP Hikvision Story on Jan 18, 2019
The NYPD has refuted the SCMP Hikvision story, the Voice of America has reported. On January 11, 2018, the SCMP alleged that the NYPD was using...
The False SCMP Story on Hikvision NYC AI on Jan 14, 2019
In the past week, one of Asia's largest publications, the South China Morning Post (SCMP), posted an article about "Chinese [facial recognition]...
CES 2019 Show Report on Jan 10, 2019
IPVM was at CES 2019 for the second year (see our CES 2018 Show Final Report) and is reporting on announcements and interesting news from the show...
IPVM Best New Products 2019 Opened - 70+ Entrants on Jan 07, 2019
The inaugural IPVM Best New Product Awards has been opened - the industry's first and only program where the awards are not pay-to-play and the...
CyberExtruder Face Recognition Profile on Jan 04, 2019
CyberExtruder offers 3D modeling face recognition software that they say provides quicker and more accurate matches than other 2D face recognition...
Foolish Strategy: OEMing Facial Recognition on Dec 13, 2018
Almost as 'hot' as face recognition marketing right now is OEMing facial recognition. Last year, they were a who's who of company's with...
Akuvox Intercom Profile on Dec 06, 2018
Akuvox, a Chinese manufacturer of VoIP products, is expanding heavily into Video Intercom products with disruptive pricing targeted for commercial...

Most Recent Industry Reports

From The Basement To Buried Behind Chinatown: ISC West Emerging Technology Zone on Feb 22, 2019
What does ISC West think about 'Emerging Technology'? Well, last year, they put those companies in the basement. This year, they moved them up to...
Private School IT Manager Surveillance Interview on Feb 22, 2019
This IT manager describes himself as the "oft-maligned IT person" whose "opinions may not always be appreciated by the integrator crowd." But he is...
Outdoor Camera Mounting Hardware Guide on Feb 21, 2019
Mounting cameras outdoors can be challenging, requiring understanding different types of equipment and methods. In this guide, we teach this...
HID Favorability Results 2019 on Feb 21, 2019
HID favorability results were strong, in the 2019 IPVM integrator study of 200+ integrators, with a net +62% and low negativity as the table below...
First US State, Vermont, Bans Dahua and Hikvision on Feb 21, 2019
The first US state, Vermont, has issued a ban on a number of Chinese and Russian manufacturers including the world's 2 largest video surveillance...
ADI 'SAVE BIG' On FLIR And Hikvision Examined on Feb 20, 2019
One is a major US defense supplier. The other is owned by the Chinese government. But you can "SAVE BIG" on both at ADI. In this note, we...
BluB0x Company Profile on Feb 20, 2019
BluB0x has doubled in revenue every year since its founding in 2013, according to CEO Patrick Barry. We originally reported on them in 2015. At the...
Massive Leak Of Chinese VMS Provider Exposes Xinjiang Surveillance on Feb 20, 2019
A subsidiary of China’s claimed largest VMS provider is tracking the precise location and ethnicity of millions in China’s Xinjiang region,...
Security Installation Tools Guide - 22 Tools Listed on Feb 19, 2019
In this guide, we cover 22 tools that security installers frequently use. This is one part of our upcoming Video Surveillance...
Sales Cuts At Rasilient on Feb 19, 2019
Over the past 2 years, video surveillance storage specialist Rasilient has expanded its workforce significantly, aiming to build its own branded...

The world's leading video surveillance information source, IPVM provides the best reporting, testing and training for 10,000+ members globally. Dedicated to independent and objective information, we uniquely refuse any and all advertisements, sponsorship and consulting from manufacturers.

About | FAQ | Contact