Bias In Facial Recognition Varies By Country, NIST Report Shows

By Zach Segal, Published Jul 15, 2020, 12:17pm EDT

While many argue that face recognition is inherently racist, results from one of the most extensive studies done on demographic bias in AI, the Facial Recognition Vendor Test (FRVT) Part 3 by the National Institute of Standards and Technolgy (NIST) analyzing over 100 algorithms, has shown that bias varied across the country of development.

IPVM Image

In particular, they observed that several algorithms developed by China groups performed better on East Asian faces than Eastern European faces while the vast majority of algorithms performed Eastern European faces than East Asians.

Executive *******

***** **** **% ** western ********* ********* ********** that ********* ****** ** Eastern ******** *** **** East ***** **** (***** 10-100 ***** ******), ~**% of ***** ********* ********* algorithms **** ********* ****** on **** ***** *** than ******* ******** ****.

**** ****** ******* ** all **** ***** *****, although *** ** ********, with ~**% ** ********** developed ** **** ***** groups ********** ****** ** Eastern ***** ***** **** European.

Training **** *****

**** ********* ** ***** of ****** ******** ****:

**** ******** **** ******** data, ** ******* **** other ****** ********* ** the ***********, *** ** effective ** ******** ********** false ******** *************. ****, the ******-**** ********** ***** be *** ********** ** investigate *** ******* ** more *******, ******** *******, training ****.

*******, **** *** *** an ********** ******** ** establish ***** *** **** did *** **** ** training ****. ***, **** result *** *** **** with ***** ***** ****** performance ** ***** ***** faces, ** ******* ****** may ******* *** *******.

****, ***** ** *** NIST ********* *******:

IPVM Image

Study ********

**** ****** **** *********** disparities ** ****** *********** algorithms ** ***** ****** facial *********** ****** ****, analyzing ********** **** **** 100 ******. *** ****** of ***** ****, ***** we **** ***** **, they ******** *** ********** in ***** ***** ***** (FMR) ** ****** ***** from ****-******* ** ***** Department *********** ******** (***/*** 39794-5 **************). ** ******* of * ***** ***** would ** ** * friend ******** **** ****** using ***** ****, ******* of *****. ** **** case, **** ******** ***,*** visa *********** ****** **** 441,517 ********* *********** ******, selecting ****** *** ******** to * ****** ** countries ****** *** *** historical ***********. *** **** researchers **** **** ***** of *** ****** ** false ******* **** ******** (all ******* **** ***** because *** **** **** disjoint).

**** ***** *********** ** false ***** *****, **** false ***** ***** ****** in ******* ********* (******, Ukraine, *** ******) *** highest **** **** ******** and **** ****** (*****, Japan, *****, ***********, ********, and *******). *******, *** study ***** ****

**** * ****** ** algorithms ********* ** ***** this ****** ** ********, with *** *****-******** ***** on **** ***** *****.

******** **** ******** **** or ********* *********** ***** be ****** * *** of *** ****** *** ethnic **** ******* ** most ** *** ********** they ******.

IPVM Image

How ** ********* *********** ***********

***** **-*** ***** **** accurate *** *. ********* than *. ******, ****** like * ***** **********, this *** ** **********. The *********** ** ********** can ** **** (**** algorithms *** ***** ***** of ~.***** *** *. Asians *** ~.****** *** E. *********), *** ** some *****, *, ** close ** * ****** would ** ********. ** an ********* ** **** for ************/*:* ********, **** as ***** ****** *********** to ****** **** ******, mistakes ***** ** ********** infrequent *** *** ********* would ** ****** ************. However, ** **** *** identification/1:n *********, **** ** scanning ** **** *** anyone **** ** *********** warrant, *** ***** **** increases ************* **** *, and *** ********* *** have * **** ****** (even **** **** ***) if * ** ***** enough.

**, ***** *** *********** differences *** ***, *** difference *** *** ** an ***** **** *:* verification *** *** ** a **** ***** (**** disparity ****** **** ***) if **** *** *:* identification.

Effect ** ********* ******

**** **% ** ***-****-***** made ********** ********* ****** (a ***** ***** ***** rate) ** ******* ******** faces **** ** **** Asian ****

***, ~**% ** ******* firms (********* *****, *********, and ******) *** ~**% of **** ***** ***** (expanded ** ******* ********* like ****** **** *** in **** **** *** not ** *** ***** of ********* ****** *** the **** *****) ********* better ** **** ***** faces **** ******* ********, as *** ***** ***** highlights:

IPVM Image

********

***** *** ******** ** widespread *********** **** ** a *******, *** **** that ********** ********* ** Chinese *** **** ***** teams ****** * ********* bias **** **** ** other *****, ******* **** much ** *** *********** disparity ** **** *********** accuracy *** ** ****** by *** *** ** diverse ******** ****. **** means **** ********** ****** ensure **** **** *** diverse ******** ****, ******* of **** ******* ** what ** ******* ** use. ** **** ***** that ********* ** ** teams *** **** ** better **********.

** ********, ***-***** ****** ask ***** *********** *********** in *** ********** **** are ******** ******* *** take ********* ****** **** consideration. ***** ** ********* may ******* ** ********** on *** **** ***********, it *** ************ ** the *********** ** ** used **.

***********

***** *** ***** *** rigorously *** **** ***, it *** *** ******* the ******** **** **** or **** ** ********* cause *** ******. **** means **** ***** ** can **** ******** ******* based ** *** *****'* findings, ** ****** ************ state ******** ** ****.

****, **** *****:

*** *** *** * corresponding *********** *** ***** Asian ***** *** *** few ********** [****] **** submitted ** ********** [** India *** **********].

*******, ** ** ***** possible **** ***** ***** Asian **** **** **** that *** *** ******* a ****** ********** ** South ***** **** ****** than ***-***** ***** ***** did. ** ** **** to **** ** ******* data **** ** ***** that *** ***** ****** to ****** ****** *********** neural ********, ** **** teams *** **** ****** on *** **** ****** accessible ********.

*** *** ** *** same ******** *** ******* why **** ***** **** clustered ******** *** *** bias *** ***** **** uniform, **** **** ********** performing ****** ** *** same ******** *** ****** groups.

Poll / ****

Comments (6)

****, **** ***!

***** *** **** ****** was **** * ****** ago, ****, **** ****, is ******* ** **** coverage ** ****** *********** technology. *** *********** *** topics ** *****, ****** share.

Agree: 3
Disagree
Informative
Unhelpful
Funny

** **** ******* ****** out *** **** ****** did *** **** ** training ****. ** *** FR ******** ************ ******* on ******* ********, **** is ***** ****** ***** seem ** ** **** and **** ********. ******* a ****** **** **** training **** *** ******* a **** ** **** is *** ***** ** open ****** ****. **** will, ** ** *******, be ** *********** ***** to ****** **** ** not **** *** ****** to ***** **** ****** acquisition *** ******** **** becomes * *** ******** block ** ******** *** also *** ****** ** which **** ****** ******** sets. *** *********** ****** in **** ******** *** the ******* ************** ****** amassing * ***** **** set ** ******'* *****. This ***** ** *** question **, ** **** extent ***** * ********** training *** ** ******* that ***** ******* *** challenges ** *********** *** bias.

Agree
Disagree
Informative: 1
Unhelpful
Funny

**** ***** ** *** question **, ** **** extent ***** * ********* training *** ** ******* that ***** ******* *** challenges ** *********** *** bias.

***** ***** *****, * can *** * ********* type ** ************* *** all ********** ************. **** would *** **** ***** to ** *** *** AI ** **** *** general ****** *** ******* entities *** ****** ** ethical *********** ** *** algorithms ***** ****.

Agree
Disagree
Informative
Unhelpful
Funny

* ***** ** ******* (Timnit *****, ***** ***********, Briana *********, ******** ******* Vaughan, ***** *******, *** Daumé ***, **** ********) wrote ** *********** ******* proposing ********* ***** ***** lines ********,********** *** ********. **** ***** **** all ******** ****** **** with ********** *** ***** use *** *********** ***** how *** **** *** collected *** **** ** contains.

** ***** ** *********** if *** ********** *** a "**** **** ***** dataset" ***** *** ****** accessible *********** ***** **** data.

***, *********, ********* **** is ** *********** ****, but ** ***** **** lead ** ********* *********** in ******** ******* * machine ******** ********* **** invariably **** *** ********* data ***** ** **** data. *******, ** ***** definitely ***** ******* ******.

Agree
Disagree
Informative: 1
Unhelpful
Funny

** ****, ***** *********. I **** *** ****, its **** ******. ***** is * *** ***** on ***** *** ****** the ** *** *****. My ***** ** ***** will ** * *** more ** **** ** this *****. ******* **** is ******** ** ******* AI, *** **** * human *********** ***'* **** this ***** ** *** fore. ***** ** **** interesting ******* **** ** de-identification, *** ***** ******* related *******.

Agree: 2
Disagree
Informative
Unhelpful
Funny

***** *** *** * agree. ** ** * very ********* ***** *** only ******* **** ********.

Agree
Disagree
Informative
Unhelpful
Funny
Read this IPVM report for free.

This article is part of IPVM's 7,205 reports and 959 tests and is only available to subscribers. To get a one-time preview of our work, enter your work email to access the full article.

Already a subscriber? Login here | Join now
Loading Related Reports