Facial Surveillance Used On US Politicians, Technical Challenges Remain

Author: John Honovich, Published on Jun 12, 2018

Many fear that facial surveillance will be used against people by governments, such as the vast facial surveillance apparatus being built in China as well as the ACLU's complaints about US pilots.

Now, the New York Times has built their own facial surveillance system that they are using on US politicians as they seek to report on political issues in the US capital.

Inside this note, we examine the NY Times system, including problems they have faced, in particular with facial recognition matching issues.

Facial Recognition of US Congresspeople

As explained here, "How The New York Times Uses Software To Recognize Members of Congress", the NY Times built an application on top of Amazon's Rekognition service, which lets reporters take a picture of potential Congresspeople with their phones, text it to their service, let Amazon run facial recognition on it, returning potential face matches, as the demo image below shows:

The NY Times need was that, with 500+ Congresspeople, it is hard for every reporter to remember the face and name of every Congressperson, so that they could use this service to identify Congresspeople to help in pursuing stories. For example, they credited this application for their "Trump Groups Raised Millions, Then Paid It Out to Loyalists and a Trump Hotel" article.

Open Sourced

The NY Times has open sourced the application (see Github: Who The Hill: An MMS-based facial recognition service for members of Congress).

Amazon Rekognition Central Role

Get Video Surveillance News In Your Inbox
Get Video Surveillance News In Your Inbox

Like the ACLU case recently showed, Amazon's Rekognition service plays a central role in this application. Amazon makes it relatively cheap and easy to build your own 'DIY' facial surveillance system, as there are no expensive license fees up front, you pay as you go and it offers well-documented APIs for integration.

Moreover, Amazon offers a specific API for recognizing celebrities. This means no work needs to be done by the user for this as tens of thousands of celebrities (actors, politicians, etc.) have been pre-indexed and trained by the Amazon system.

Indeed, Amazon offers a demo of the celebrity recognition service (for those with an AWS account). However, it is not always accurate as this IPVM experiment showed:

Doppelgangers / False Match Issues

Indeed, the NY Times faced a more serious false matching issue. They found certain false matches so significant that they had to hardcode their application with a 'doppelganger' feature to exclude known look-alikes. Below is the list with a sample of a few of the common false matches (Congressperson vs Doppelganger):

Here is another one, politician vs actor:

Problems With Facial Recognition

The problem is, fundamentally, lots of people look alike each other, especially when you get into larger populations of people (retail, public facilities, etc.) and not small offices.

Moreover, as the NY Times themselves learned in their application's usage:

our hit rate on photographs was very low because the halls of the Capitol are poorly lit and the photographs we took for testing were consistently marred by shadow and blur. [emphasis added]

That type of lamentation should be familiar to experienced video surveillance users where such conditions are commonplace.

On The Rise But Problems Too

On the plus side for facial recognition, this example of Amazon usage shows facial surveillance getting easier to do. On the negative side, these results also show the need to temper expectations about accuracy. For the NY Times use case where a reporter will only be periodically submitting requests and manually reviewing responses (looking just for Congresspeople), they should be able to fairly and quickly assess false matches. To the contrary, such results on a real-time video surveillance system would likely cause an overload of false alerts.

Comments (2) : PRO Members only. Login. or Join.

Related Reports on Facial Recognition

Dahua Face Recognition Camera Tested on Oct 15, 2018
Dahua has been one of the industry's most vocal proponents of the value that AI creates: As part of this, Dahua has released a facial...
Evidence Of Hikvision's Involvement With Xinjiang IJOP And Re-Education Camps on Oct 02, 2018
IPVM reveals as-yet unreported details about Hikvision’s activities in Xinjiang - a region of China where massive human rights abuses are being...
ASIS GSX 2018 Show Report on Sep 25, 2018
In the first major US show since the US government ban of Dahua and Hikvision was passed into law, the mega Chinese companies were in retreat and...
Congressional Letter Urges Sanctions Against Dahua and Hikvision For Human Rights Abuses on Sep 04, 2018
17 US Congresspeople sent a letter to the Secretary of State and Treasury urging sanctions against Chinese officials plus Dahua and Hikvision,...
Video Analytics Integration Guide on Aug 16, 2018
Video analytics is hot again (at least conceptually) but integrating video analytics with VMSes can be challenging. This is especially significant...
RealNetworks Free School Facial Recognition on Aug 03, 2018
The company that created RealPlayer is moving beyond media delivery and into the security space with a new facial recognition platform they have...
99.9997% Accurate Amazon Facial Rekognition Falsely Matches 28 on Jul 27, 2018
A new facial recognition controversy has arisen, with the ACLU cleverly using Amazon's Rekognition service to falsely match 28 Congresspeople to...
AI Startup Anyvision Raises $28 Million Led By Bosch on Jul 20, 2018
Anyvision is the most ambitious heavy-spending video surveillance startup in many years. And, now, the startup has raised $28 million led by...
FST Fails on Jul 17, 2018
FST was one of the hottest startups of the decade, selected as the best new product at ISC West 2011 and backed with tens of millions in...
Hikvision Wins Chinese Government Forced Facial Recognition Project Across 967 Mosques on Jul 16, 2018
Hikvision has won a Chinese government tender which requires that facial recognition cameras be set up at the entrance of every single mosque...

Most Recent Industry Reports

Best Alternatives to Banned Dahua and Hikvision on Oct 17, 2018
With the US government ban and a growing number of users banning Dahua and Hikvision, one key question is what to use for low cost? While Dahua and...
Video Quality / Compression Tutorial on Oct 17, 2018
While CODECs, like H.264, H.265, and MJPEG, get a lot of attention, a camera's 'quality' or compression setting has a big impact on overall...
Knightscope Winning Investors, Struggling With Growth on Oct 16, 2018
While Knightscope's new financials show the company only winning 11 new customers in the past 12 months, the company continues to win new...
Integrator Laptop Guide on Oct 16, 2018
This 18-page guide provides guidance and statistics about integrator laptop use. 150 integrators explained to IPVM in detail about their laptops,...
Huawei Admits AI "Bubble" on Oct 16, 2018
A fascinating article from the Chinese government's Global Times: Huawei’s AI ambition to reshape industries. While the Global Times talks about...
ADI's Financials Revealed + W-Box Growth Priority on Oct 15, 2018
  ADI is one of the most powerful distributors in the security industry but how big are they? How much profit do they make? How much do they sell...
Dahua Face Recognition Camera Tested on Oct 15, 2018
Dahua has been one of the industry's most vocal proponents of the value that AI creates: As part of this, Dahua has released a facial...
Amazon Touts Home Security Market Disruption on Oct 15, 2018
Amazon is coming for ADT and all of home security. Indeed, Amazon is advertising this as, in their own words, calling home security a: Inside...
Higher Power PoE 802.3bt Ratified, Impact on Security Products Examined on Oct 12, 2018
Power over Ethernet has become one of the most popular features of many video, access, and other security products. See our PoE for IP Video...
"New Zealand Govt Uses Chinese Cameras Banned In US", Considers Security Audit on Oct 12, 2018
Newsroom NZ has issued a report: "NZ Govt uses Chinese cameras banned in US": This comes after the US federal government banned purchases of...

The world's leading video surveillance information source, IPVM provides the best reporting, testing and training for 10,000+ members globally. Dedicated to independent and objective information, we uniquely refuse any and all advertisements, sponsorship and consulting from manufacturers.

About | FAQ | Contact