Bias In Facial Recognition Varies By Country, NIST Report Shows

By: Zach Segal, Published on Jul 15, 2020

While many argue that face recognition is inherently racist, results from one of the most extensive studies done on demographic bias in AI, the Facial Recognition Vendor Test (FRVT) Part 3 by the National Institute of Standards and Technolgy (NIST) analyzing over 100 algorithms, has shown that bias varied across the country of development.

IPVM Image

In particular, they observed that several algorithms developed by China groups performed better on East Asian faces than Eastern European faces while the vast majority of algorithms performed Eastern European faces than East Asians.

***** **** ***** **** face *********** ** ********** racist, ******* **** *** of *** **** ********* studies **** ** *********** bias ** **, ********* *********** ****** **** (FRVT) **** *** *** ******** ********* of ********* *** ********* (NIST) ********* **** *** algorithms, *** ***** **** bias ****** ****** *** country ** ***********.

IPVM Image

** **********, **** ******** that ******* ********** ********* by ***** ****** ********* better ** **** ***** faces **** ******* ******** faces ***** *** **** majority ** ********** ********* Eastern ******** ***** **** East ******.

[***************]

Executive *******

***** **** **% ** western ********* ********* ********** that ********* ****** ** Eastern ******** *** **** East ***** **** (***** 10-100 ***** ******), ~**% of ***** ********* ********* algorithms **** ********* ****** on **** ***** *** than ******* ******** ****.

**** ****** ******* ** all **** ***** *****, although *** ** ********, with ~**% ** ********** developed ** **** ***** groups ********** ****** ** Eastern ***** ***** **** European.

Training **** *****

**** ********* ** ***** of ****** ******** ****:

**** ******** **** ******** data, ** ******* **** other ****** ********* ** the ***********, *** ** effective ** ******** ********** false ******** *************. ****, the ******-**** ********** ***** be *** ********** ** investigate *** ******* ** more *******, ******** *******, training ****.

*******, **** *** *** an ********** ******** ** establish ***** *** **** did *** **** ** training ****. ***, **** result *** *** **** with ***** ***** ****** performance ** ***** ***** faces, ** ******* ****** may ******* *** *******.

****, ***** ** *** NIST ********* *******:

IPVM Image

Study ********

**** ****** **** *********** disparities ** ****** *********** algorithms ** ***** ****** facial *********** ****** ****, analyzing ********** **** **** 100 ******. *** ****** of ***** ****, ***** we **** ***** **, they ******** *** ********** in ***** ***** ***** (FMR) ** ****** ***** from ****-******* ** ***** Department *********** ******** (***/*** 39794-5 **************). ** ******* of * ***** ***** would ** ** * friend ******** **** ****** using ***** ****, ******* of *****. ** **** case, **** ******** ***,*** visa *********** ****** **** 441,517 ********* *********** ******, selecting ****** *** ******** to * ****** ** countries ****** *** *** historical ***********. *** **** researchers **** **** ***** of *** ****** ** false ******* **** ******** (all ******* **** ***** because *** **** **** disjoint).

**** ***** *********** ** false ***** *****, **** false ***** ***** ****** in ******* ********* (******, Ukraine, *** ******) *** highest **** **** ******** and **** ****** (*****, Japan, *****, ***********, ********, and *******). *******, *** study ***** ****

**** * ****** ** algorithms ********* ** ***** this ****** ** ********, with *** *****-******** ***** on **** ***** *****.

******** **** ******** **** or ********* *********** ***** be ****** * *** of *** ****** *** ethnic **** ******* ** most ** *** ********** they ******.

IPVM Image

How ** ********* *********** ***********

***** **-*** ***** **** accurate *** *. ********* than *. ******, ****** like * ***** **********, this *** ** **********. The *********** ** ********** can ** **** (**** algorithms *** ***** ***** of ~.***** *** *. Asians *** ~.****** *** E. *********), *** ** some *****, *, ** close ** * ****** would ** ********. ** an ********* ** **** for ************/*:* ********, **** as ***** ****** *********** to ****** **** ******, mistakes ***** ** ********** infrequent *** *** ********* would ** ****** ************. However, ** **** *** identification/1:n *********, **** ** scanning ** **** *** anyone **** ** *********** warrant, *** ***** **** increases ************* **** *, and *** ********* *** have * **** ****** (even **** **** ***) if * ** ***** enough.

**, ***** *** *********** differences *** ***, *** difference *** *** ** an ***** **** *:* verification *** *** ** a **** ***** (**** disparity ****** **** ***) if **** *** *:* identification.

Effect ** ********* ******

**** **% ** ***-****-***** made ********** ********* ****** (a ***** ***** ***** rate) ** ******* ******** faces **** ** **** Asian ****

***, ~**% ** ******* firms (********* *****, *********, and ******) *** ~**% of **** ***** ***** (expanded ** ******* ********* like ****** **** *** in **** **** *** not ** *** ***** of ********* ****** *** the **** *****) ********* better ** **** ***** faces **** ******* ********, as *** ***** ***** highlights:

IPVM Image

********

***** *** ******** ** widespread *********** **** ** a *******, *** **** that ********** ********* ** Chinese *** **** ***** teams ****** * ********* bias **** **** ** other *****, ******* **** much ** *** *********** disparity ** **** *********** accuracy *** ** ****** by *** *** ** diverse ******** ****. **** means **** ********** ****** ensure **** **** *** diverse ******** ****, ******* of **** ******* ** what ** ******* ** use. ** **** ***** that ********* ** ** teams *** **** ** better **********.

** ********, ***-***** ****** ask ***** *********** *********** in *** ********** **** are ******** ******* *** take ********* ****** **** consideration. ***** ** ********* may ******* ** ********** on *** **** ***********, it *** ************ ** the *********** ** ** used **.

***********

***** *** ***** *** rigorously *** **** ***, it *** *** ******* the ******** **** **** or **** ** ********* cause *** ******. **** means **** ***** ** can **** ******** ******* based ** *** *****'* findings, ** ****** ************ state ******** ** ****.

****, **** *****:

*** *** *** * corresponding *********** *** ***** Asian ***** *** *** few ********** [****] **** submitted ** ********** [** India *** **********].

*******, ** ** ***** possible **** ***** ***** Asian **** **** **** that *** *** ******* a ****** ********** ** South ***** **** ****** than ***-***** ***** ***** did. ** ** **** to **** ** ******* data **** ** ***** that *** ***** ****** to ****** ****** *********** neural ********, ** **** teams *** **** ****** on *** **** ****** accessible ********.

*** *** ** *** same ******** *** ******* why **** ***** **** clustered ******** *** *** bias *** ***** **** uniform, **** **** ********** performing ****** ** *** same ******** *** ****** groups.

Poll / ****

Comments (6)

John Honovich

IPVM | 07/15/20 04:19pm

****, **** ***!

***** *** **** ****** was **** * ****** ago, ****, **** ****, is ******* ** **** coverage ** ****** *********** technology. *** *********** *** topics ** *****, ****** share.

Avatar

Salvatore D'Agostino

IDmachines | 07/15/20 04:45pm

** **** ******* ****** out *** **** ****** did *** **** ** training ****. ** *** FR ******** ************ ******* on ******* ********, **** is ***** ****** ***** seem ** ** **** and **** ********. ******* a ****** **** **** training **** *** ******* a **** ** **** is *** ***** ** open ****** ****. **** will, ** ** *******, be ** *********** ***** to ****** **** ** not **** *** ****** to ***** **** ****** acquisition *** ******** **** becomes * *** ******** block ** ******** *** also *** ****** ** which **** ****** ******** sets. *** *********** ****** in **** ******** *** the ******* ************** ****** amassing * ***** **** set ** ******'* *****. This ***** ** *** question **, ** **** extent ***** * ********** training *** ** ******* that ***** ******* *** challenges ** *********** *** bias.

Anthony Hackett

In reply to Salvatore D'Agostino | 07/15/20 05:33pm

**** ***** ** *** question **, ** **** extent ***** * ********* training *** ** ******* that ***** ******* *** challenges ** *********** *** bias.

***** ***** *****, * can *** * ********* type ** ************* *** all ********** ************. **** would *** **** ***** to ** *** *** AI ** **** *** general ****** *** ******* entities *** ****** ** ethical *********** ** *** algorithms ***** ****.

Zach Segal

In reply to Anthony Hackett | 07/15/20 06:54pm

* ***** ** ******* (Timnit *****, ***** ***********, Briana *********, ******** ******* Vaughan, ***** *******, *** Daumé ***, **** ********) wrote ** *********** ******* proposing ********* ***** ***** lines ********,********** *** ********. **** ***** **** all ******** ****** **** with ********** *** ***** use *** *********** ***** how *** **** *** collected *** **** ** contains.

** ***** ** *********** if *** ********** *** a "**** **** ***** dataset" ***** *** ****** accessible *********** ***** **** data.

***, *********, ********* **** is ** *********** ****, but ** ***** **** lead ** ********* *********** in ******** ******* * machine ******** ********* **** invariably **** *** ********* data ***** ** **** data. *******, ** ***** definitely ***** ******* ******.

Avatar

Salvatore D'Agostino

IDmachines | 07/15/20 07:05pm

** ****, ***** *********. I **** *** ****, its **** ******. ***** is * *** ***** on ***** *** ****** the ** *** *****. My ***** ** ***** will ** * *** more ** **** ** this *****. ******* **** is ******** ** ******* AI, *** **** * human *********** ***'* **** this ***** ** *** fore. ***** ** **** interesting ******* **** ** de-identification, *** ***** ******* related *******.

Zach Segal

In reply to Salvatore D'Agostino | 07/15/20 07:14pm

***** *** *** * agree. ** ** * very ********* ***** *** only ******* **** ********.

Login to read this IPVM report.

Related Reports

Integrated IR Camera Shootout 2020 - Avigilon, Axis, Bosch, Dahua, Hanwha, Hikvision, Panasonic, Uniview, Vivotek on Jan 30, 2020
The best and worst cameras tested in this IPVM shootout showed major...
Facial Recognition 101 on Mar 18, 2020
Facial recognition interest, use and fear is increasing. This guide aims to...
TVT / InVid Facial Recognition Tested on Mar 25, 2020
Facial recognition is frequently sold for thousands of dollars per channel...
TVT Temperature Measurement Terminal Tested on Jul 23, 2020
While Dahua and Hikvision get the most attention for China temp products,...
IronYun AI Analytics Tested on Feb 17, 2020
Taiwan / US startup IronYun has raised tens of millions for its "mission to...
Hikvision Fever Screening Thermal Solutions Examined on Apr 13, 2020
Hikvision is marketing "safer, faster, smarter" with their Fever Screening...
Dahua, Hikvision, ZKTeco Face Mask Detection Shootout on Jun 19, 2020
Temperature tablets with face mask detection are one of the hottest trends in...
Spectron IR Thermal Fever Screening System Examined on Apr 14, 2020
Most are quick to avoid "fever screening" and "medical" labels, but...
FLIR Screen-EST Screening Software Tested on Jun 30, 2020
In our FLIR A Series Test, the cameras' biggest drawback was their lack of...
Cheap Camera Problems at Night on Feb 19, 2020
Cheap cameras generally have problems at night, despite the common perception...
FLIR A Series Temperature Screening Cameras Tested on Jun 04, 2020
FLIR is one of the biggest names in thermal and one of the most conservative....
30 Million Criminal Face Database Tested (Captis Intelligence) on Apr 27, 2020
30 million criminal mugshots are now available for facial recognition...
Face Shields Impact On Temperature Measurement And Mask Detection on Jul 27, 2020
First, the use of face masks, and now, plastic face shields are rising...
Directory of 201 "Fever" Camera Suppliers on Aug 04, 2020
This directory provides a list of "Fever" scanning thermal camera providers...
The Problem With Fever Detecting Thermal Sunglasses on Apr 15, 2020
While the media has promoted using thermal sunglasses to detect fevers, this...

Recent Reports

Dangerous Hikvision Fever Camera Showcased by Chilean City on Aug 07, 2020
Deploying a fever camera outdoors, in the rain, with no black body, is...
"Grand Slam" For Pelco's PE Firm, A Risk For Motorola on Aug 07, 2020
The word "Pelco" and "grand slam" have not been said together for many years....
FLIR Stock Falls, Admits 'Decelerating' Demand For Temperature Screening on Aug 07, 2020
Is the boom going to bust for temperature screening? FLIR disappointed...
VSaaS Will Hurt Integrators on Aug 06, 2020
VSaaS will hurt integrators, there is no question about that. How much...
Dogs For Coronavirus Screening Examined on Aug 06, 2020
While thermal temperature screening is the surveillance industry's most...
ADT Slides Back, Disappointing Results, Poor Commercial Performance on Aug 06, 2020
While ADT had an incredible start to the week, driven by the Google...
AHJ / Authority Having Jurisdiction Tutorial on Aug 06, 2020
One of the most powerful yet often underappreciated characters in all of the...
SIA Coaches Sellers on NDAA 889B Blacklist Workarounds on Aug 05, 2020
Last month SIA demanded that NDAA 899B "must be delayed". Now that they have...
ADI Returns To Growth, Back To 'Pre-COVID Levels' on Aug 05, 2020
While ADI was hit hard in April, with revenue declining 21%, the company's...
Exposing Fever Tablet Suppliers and 40+ Relabelers on Aug 05, 2020
IPVM has found 40+ USA and EU companies relabeling fever tablets designed,...
Directory of 201 "Fever" Camera Suppliers on Aug 04, 2020
This directory provides a list of "Fever" scanning thermal camera providers...
Face Masks Increase Face Recognition Errors Says NIST on Aug 04, 2020
COVID-19 has led to widespread facemask use, which as IPVM testing has shown...
Dahua Loses Australian Medical Device Approval on Aug 04, 2020
Dahua has cancelled its medical device registration after "discussions" with...
Google Invests in ADT, ADT Stock Soars on Aug 03, 2020
Google has announced a $450 million investment in the Florida-based security...