Facial Surveillance Used On US Politicians, Technical Challenges Remain

Author: John Honovich, Published on Jun 12, 2018

Many fear that facial surveillance will be used against people by governments, such as the vast facial surveillance apparatus being built in China as well as the ACLU's complaints about US pilots.

Now, the New York Times has built their own facial surveillance system that they are using on US politicians as they seek to report on political issues in the US capital.

Inside this note, we examine the NY Times system, including problems they have faced, in particular with facial recognition matching issues.

Facial Recognition of US Congresspeople

As explained here, "How The New York Times Uses Software To Recognize Members of Congress", the NY Times built an application on top of Amazon's Rekognition service, which lets reporters take a picture of potential Congresspeople with their phones, text it to their service, let Amazon run facial recognition on it, returning potential face matches, as the demo image below shows:

The NY Times need was that, with 500+ Congresspeople, it is hard for every reporter to remember the face and name of every Congressperson, so that they could use this service to identify Congresspeople to help in pursuing stories. For example, they credited this application for their "Trump Groups Raised Millions, Then Paid It Out to Loyalists and a Trump Hotel" article.

Open Sourced

The NY Times has open sourced the application (see Github: Who The Hill: An MMS-based facial recognition service for members of Congress).

Amazon Rekognition Central Role

Get Video Surveillance News In Your Inbox
Get Video Surveillance News In Your Inbox

Like the ACLU case recently showed, Amazon's Rekognition service plays a central role in this application. Amazon makes it relatively cheap and easy to build your own 'DIY' facial surveillance system, as there are no expensive license fees up front, you pay as you go and it offers well-documented APIs for integration.

Moreover, Amazon offers a specific API for recognizing celebrities. This means no work needs to be done by the user for this as tens of thousands of celebrities (actors, politicians, etc.) have been pre-indexed and trained by the Amazon system.

Indeed, Amazon offers a demo of the celebrity recognition service (for those with an AWS account). However, it is not always accurate as this IPVM experiment showed:

Doppelgangers / False Match Issues

Indeed, the NY Times faced a more serious false matching issue. They found certain false matches so significant that they had to hardcode their application with a 'doppelganger' feature to exclude known look-alikes. Below is the list with a sample of a few of the common false matches (Congressperson vs Doppelganger):

Here is another one, politician vs actor:

Problems With Facial Recognition

The problem is, fundamentally, lots of people look alike each other, especially when you get into larger populations of people (retail, public facilities, etc.) and not small offices.

Moreover, as the NY Times themselves learned in their application's usage:

our hit rate on photographs was very low because the halls of the Capitol are poorly lit and the photographs we took for testing were consistently marred by shadow and blur. [emphasis added]

That type of lamentation should be familiar to experienced video surveillance users where such conditions are commonplace.

On The Rise But Problems Too

On the plus side for facial recognition, this example of Amazon usage shows facial surveillance getting easier to do. On the negative side, these results also show the need to temper expectations about accuracy. For the NY Times use case where a reporter will only be periodically submitting requests and manually reviewing responses (looking just for Congresspeople), they should be able to fairly and quickly assess false matches. To the contrary, such results on a real-time video surveillance system would likely cause an overload of false alerts.

Comments (2) : PRO Members only. Login. or Join.

Related Reports on Facial Recognition

Akuvox Intercom Profile on Dec 06, 2018
Akuvox, a Chinese manufacturer of VoIP products, is expanding heavily into Video Intercom products with disruptive pricing targeted for commercial...
No GDPR Penalties For UK Swann 'Spying Hack' on Nov 20, 2018
The UK’s data protection agency has closed its investigation into Infinova-owned Swann Security UK, the ICO confirmed to IPVM, deciding to take “no...
ISC East 2018 Mini-Show Final Report on Nov 16, 2018
This is our second (updated) and final show report from ISC East. ISC East, by its own admission, is not a national or international show, billed...
Facial Detection Tested on Nov 16, 2018
Facial detection and recognition are increasingly offered by video surveillance manufacturers. Facial detection detects faces in an image/video...
Huawei To Be “Top Three Global Position” in Video Surveillance on Oct 29, 2018
Huawei - China’s biggest networking gear and smartphone maker - plans to become one of the top three global video surveillance manufacturers in...
Axxon Face Search Tested on Oct 26, 2018
AxxonSoft has brought facial recognition to their Axxon Next VMS for free with the simply named Face Search, claiming to allow users to find...
When Your Salesperson Creates An International Incident (Amazon / ICE) on Oct 23, 2018
Having your salespeople be aggressive and go after big accounts is a virtue, mostly. On occasions, it can cause problems. A new case involves...
Dahua Face Recognition Camera Tested on Oct 15, 2018
Dahua has been one of the industry's most vocal proponents of the value that AI creates: As part of this, Dahua has released a facial...
Evidence Of Hikvision's Involvement With Xinjiang IJOP And Re-Education Camps on Oct 02, 2018
IPVM reveals as-yet unreported details about Hikvision’s activities in Xinjiang - a region of China where massive human rights abuses are being...
ASIS GSX 2018 Show Report on Sep 25, 2018
In the first major US show since the US government ban of Dahua and Hikvision was passed into law, the mega Chinese companies were in retreat and...

Most Recent Industry Reports

2019 Access Control Book Released on Dec 12, 2018
This is the best, most comprehensive access control book in the world, based on our unprecedented research and testing has been significantly...
Huawei Hisilicon Quietly Powering Tens of Millions of Western IoT Devices on Dec 12, 2018
Huawei Hisilicon chips are powering, at least, tens of millions of Western IoT devices, such as IP cameras and surveillance recorders, a fact that...
FLIR Launches Body Cameras Unified With VMS (TruWitness) on Dec 11, 2018
While FLIR is best known for their thermal cameras, now they have expanded into body cameras, launching TruWITNESS, a public safety focused body...
Startup Sunflower Labs' Autonomous Drone Security System on Dec 11, 2018
Startup Sunflower Labs is claiming a unique design on a home security system, combining autonomous drones and 'Sunflower' sensors. Imagine an...
The 2019 Video Surveillance Industry Guide on Dec 10, 2018
The 300 page, 2019 Video Surveillance Industry Guide, covers the key events and the future of the video surveillance market, is now available,...
Multi-Factor Access Control Authentication Guide on Dec 10, 2018
Can a stranger use your credentials? One of the oldest problems facing access control is making credentials as easy to use as keys, but restricting...
Top 2019 Trend - AI Video Analytics on Dec 10, 2018
160+ Integrators answered: What do you think the top industry trend will be in 2019? Why? AI / video analytics was the run-away winner with...
AV Tech Company Profile on Dec 07, 2018
Taiwanese manufacturer AV Tech's revenue declined ~70% since 2012. Planning a comeback, AV Tech spoke to IPVM about their opportunities and...
Ubiquiti $79 Flex IP Camera Tested on Dec 07, 2018
U.S. Manufacturer Ubiquiti has released a 1080p, integrated IR IP camera, selling it directly for $79, making this one of the least expensive IP...
Infinova's Xinjiang Business Examined on Dec 07, 2018
As pressure mounts for companies to stop doing business in China’s Xinjiang region amid a severe human rights crisis, IPVM has found Infinova sold...

The world's leading video surveillance information source, IPVM provides the best reporting, testing and training for 10,000+ members globally. Dedicated to independent and objective information, we uniquely refuse any and all advertisements, sponsorship and consulting from manufacturers.

About | FAQ | Contact