Bias In Facial Recognition Varies By Country, NIST Report Shows

By Zach Segal, Published Jul 15, 2020, 12:17pm EDT

While many argue that face recognition is inherently racist, results from one of the most extensive studies done on demographic bias in AI, the Facial Recognition Vendor Test (FRVT) Part 3 by the National Institute of Standards and Technolgy (NIST) analyzing over 100 algorithms, has shown that bias varied across the country of development.

IPVM Image

In particular, they observed that several algorithms developed by China groups performed better on East Asian faces than Eastern European faces while the vast majority of algorithms performed Eastern European faces than East Asians.

Executive *******

***** **** **% ** western ********* ********* ********** that ********* ****** ** Eastern ******** *** **** East ***** **** (***** 10-100 ***** ******), ~**% of ***** ********* ********* algorithms **** ********* ****** on **** ***** *** than ******* ******** ****.

**** ****** ******* ** all **** ***** *****, although *** ** ********, with ~**% ** ********** developed ** **** ***** groups ********** ****** ** Eastern ***** ***** **** European.

Training **** *****

**** ********* ** ***** of ****** ******** ****:

**** ******** **** ******** data, ** ******* **** other ****** ********* ** the ***********, *** ** effective ** ******** ********** false ******** *************. ****, the ******-**** ********** ***** be *** ********** ** investigate *** ******* ** more *******, ******** *******, training ****.

*******, **** *** *** an ********** ******** ** establish ***** *** **** did *** **** ** training ****. ***, **** result *** *** **** with ***** ***** ****** performance ** ***** ***** faces, ** ******* ****** may ******* *** *******.

****, ***** ** *** NIST ********* *******:

IPVM Image

Study ********

**** ****** **** *********** disparities ** ****** *********** algorithms ** ***** ****** facial *********** ****** ****, analyzing ********** **** **** 100 ******. *** ****** of ***** ****, ***** we **** ***** **, they ******** *** ********** in ***** ***** ***** (FMR) ** ****** ***** from ****-******* ** ***** Department *********** ******** (***/*** 39794-5 **************). ** ******* of * ***** ***** would ** ** * friend ******** **** ****** using ***** ****, ******* of *****. ** **** case, **** ******** ***,*** visa *********** ****** **** 441,517 ********* *********** ******, selecting ****** *** ******** to * ****** ** countries ****** *** *** historical ***********. *** **** researchers **** **** ***** of *** ****** ** false ******* **** ******** (all ******* **** ***** because *** **** **** disjoint).

**** ***** *********** ** false ***** *****, **** false ***** ***** ****** in ******* ********* (******, Ukraine, *** ******) *** highest **** **** ******** and **** ****** (*****, Japan, *****, ***********, ********, and *******). *******, *** study ***** ****

**** * ****** ** algorithms ********* ** ***** this ****** ** ********, with *** *****-******** ***** on **** ***** *****.

******** **** ******** **** or ********* *********** ***** be ****** * *** of *** ****** *** ethnic **** ******* ** most ** *** ********** they ******.

IPVM Image

How ** ********* *********** ***********

***** **-*** ***** **** accurate *** *. ********* than *. ******, ****** like * ***** **********, this *** ** **********. The *********** ** ********** can ** **** (**** algorithms *** ***** ***** of ~.***** *** *. Asians *** ~.****** *** E. *********), *** ** some *****, *, ** close ** * ****** would ** ********. ** an ********* ** **** for ************/*:* ********, **** as ***** ****** *********** to ****** **** ******, mistakes ***** ** ********** infrequent *** *** ********* would ** ****** ************. However, ** **** *** identification/1:n *********, **** ** scanning ** **** *** anyone **** ** *********** warrant, *** ***** **** increases ************* **** *, and *** ********* *** have * **** ****** (even **** **** ***) if * ** ***** enough.

**, ***** *** *********** differences *** ***, *** difference *** *** ** an ***** **** *:* verification *** *** ** a **** ***** (**** disparity ****** **** ***) if **** *** *:* identification.

Effect ** ********* ******

**** **% ** ***-****-***** made ********** ********* ****** (a ***** ***** ***** rate) ** ******* ******** faces **** ** **** Asian ****

***, ~**% ** ******* firms (********* *****, *********, and ******) *** ~**% of **** ***** ***** (expanded ** ******* ********* like ****** **** *** in **** **** *** not ** *** ***** of ********* ****** *** the **** *****) ********* better ** **** ***** faces **** ******* ********, as *** ***** ***** highlights:

IPVM Image

********

***** *** ******** ** widespread *********** **** ** a *******, *** **** that ********** ********* ** Chinese *** **** ***** teams ****** * ********* bias **** **** ** other *****, ******* **** much ** *** *********** disparity ** **** *********** accuracy *** ** ****** by *** *** ** diverse ******** ****. **** means **** ********** ****** ensure **** **** *** diverse ******** ****, ******* of **** ******* ** what ** ******* ** use. ** **** ***** that ********* ** ** teams *** **** ** better **********.

** ********, ***-***** ****** ask ***** *********** *********** in *** ********** **** are ******** ******* *** take ********* ****** **** consideration. ***** ** ********* may ******* ** ********** on *** **** ***********, it *** ************ ** the *********** ** ** used **.

***********

***** *** ***** *** rigorously *** **** ***, it *** *** ******* the ******** **** **** or **** ** ********* cause *** ******. **** means **** ***** ** can **** ******** ******* based ** *** *****'* findings, ** ****** ************ state ******** ** ****.

****, **** *****:

*** *** *** * corresponding *********** *** ***** Asian ***** *** *** few ********** [****] **** submitted ** ********** [** India *** **********].

*******, ** ** ***** possible **** ***** ***** Asian **** **** **** that *** *** ******* a ****** ********** ** South ***** **** ****** than ***-***** ***** ***** did. ** ** **** to **** ** ******* data **** ** ***** that *** ***** ****** to ****** ****** *********** neural ********, ** **** teams *** **** ****** on *** **** ****** accessible ********.

*** *** ** *** same ******** *** ******* why **** ***** **** clustered ******** *** *** bias *** ***** **** uniform, **** **** ********** performing ****** ** *** same ******** *** ****** groups.

Poll / ****

Comments (6)

John Honovich

IPVM | 07/15/20 04:19pm

****, **** ***!

***** *** **** ****** was **** * ****** ago, ****, **** ****, is ******* ** **** coverage ** ****** *********** technology. *** *********** *** topics ** *****, ****** share.

Agree: 3
Disagree
Informative
Unhelpful
Funny
Avatar

Salvatore D'Agostino

IDmachines | 07/15/20 04:45pm

** **** ******* ****** out *** **** ****** did *** **** ** training ****. ** *** FR ******** ************ ******* on ******* ********, **** is ***** ****** ***** seem ** ** **** and **** ********. ******* a ****** **** **** training **** *** ******* a **** ** **** is *** ***** ** open ****** ****. **** will, ** ** *******, be ** *********** ***** to ****** **** ** not **** *** ****** to ***** **** ****** acquisition *** ******** **** becomes * *** ******** block ** ******** *** also *** ****** ** which **** ****** ******** sets. *** *********** ****** in **** ******** *** the ******* ************** ****** amassing * ***** **** set ** ******'* *****. This ***** ** *** question **, ** **** extent ***** * ********** training *** ** ******* that ***** ******* *** challenges ** *********** *** bias.

Agree
Disagree
Informative: 1
Unhelpful
Funny

Anthony Hackett

In reply to Salvatore D'Agostino | 07/15/20 05:33pm

**** ***** ** *** question **, ** **** extent ***** * ********* training *** ** ******* that ***** ******* *** challenges ** *********** *** bias.

***** ***** *****, * can *** * ********* type ** ************* *** all ********** ************. **** would *** **** ***** to ** *** *** AI ** **** *** general ****** *** ******* entities *** ****** ** ethical *********** ** *** algorithms ***** ****.

Agree
Disagree
Informative
Unhelpful
Funny

Zach Segal

IPVM | In reply to Anthony Hackett | 07/15/20 06:54pm

* ***** ** ******* (Timnit *****, ***** ***********, Briana *********, ******** ******* Vaughan, ***** *******, *** Daumé ***, **** ********) wrote ** *********** ******* proposing ********* ***** ***** lines ********,********** *** ********. **** ***** **** all ******** ****** **** with ********** *** ***** use *** *********** ***** how *** **** *** collected *** **** ** contains.

** ***** ** *********** if *** ********** *** a "**** **** ***** dataset" ***** *** ****** accessible *********** ***** **** data.

***, *********, ********* **** is ** *********** ****, but ** ***** **** lead ** ********* *********** in ******** ******* * machine ******** ********* **** invariably **** *** ********* data ***** ** **** data. *******, ** ***** definitely ***** ******* ******.

Agree
Disagree
Informative: 1
Unhelpful
Funny
Avatar

Salvatore D'Agostino

IDmachines | 07/15/20 07:05pm

** ****, ***** *********. I **** *** ****, its **** ******. ***** is * *** ***** on ***** *** ****** the ** *** *****. My ***** ** ***** will ** * *** more ** **** ** this *****. ******* **** is ******** ** ******* AI, *** **** * human *********** ***'* **** this ***** ** *** fore. ***** ** **** interesting ******* **** ** de-identification, *** ***** ******* related *******.

Agree: 2
Disagree
Informative
Unhelpful
Funny

Zach Segal

IPVM | In reply to Salvatore D'Agostino | 07/15/20 07:14pm

***** *** *** * agree. ** ** * very ********* ***** *** only ******* **** ********.

Agree
Disagree
Informative
Unhelpful
Funny
Read this IPVM report for free.

This article is part of IPVM's 6,904 reports, 921 tests and is only available to members. To get a one-time preview of our work, enter your work email to access the full article.

Already a member? Login here | Join now
Loading Related Reports