Bias In Facial Recognition Varies By Country, NIST Report Shows

By Zach Segal, Published on Jul 15, 2020

While many argue that face recognition is inherently racist, results from one of the most extensive studies done on demographic bias in AI, the Facial Recognition Vendor Test (FRVT) Part 3 by the National Institute of Standards and Technolgy (NIST) analyzing over 100 algorithms, has shown that bias varied across the country of development.

IPVM Image

In particular, they observed that several algorithms developed by China groups performed better on East Asian faces than Eastern European faces while the vast majority of algorithms performed Eastern European faces than East Asians.

Executive *******

***** **** **% ** western ********* ********* ********** that ********* ****** ** Eastern ******** *** **** East ***** **** (***** 10-100 ***** ******), ~**% of ***** ********* ********* algorithms **** ********* ****** on **** ***** *** than ******* ******** ****.

**** ****** ******* ** all **** ***** *****, although *** ** ********, with ~**% ** ********** developed ** **** ***** groups ********** ****** ** Eastern ***** ***** **** European.

Training **** *****

**** ********* ** ***** of ****** ******** ****:

**** ******** **** ******** data, ** ******* **** other ****** ********* ** the ***********, *** ** effective ** ******** ********** false ******** *************. ****, the ******-**** ********** ***** be *** ********** ** investigate *** ******* ** more *******, ******** *******, training ****.

*******, **** *** *** an ********** ******** ** establish ***** *** **** did *** **** ** training ****. ***, **** result *** *** **** with ***** ***** ****** performance ** ***** ***** faces, ** ******* ****** may ******* *** *******.

****, ***** ** *** NIST ********* *******:

IPVM Image

Study ********

**** ****** **** *********** disparities ** ****** *********** algorithms ** ***** ****** facial *********** ****** ****, analyzing ********** **** **** 100 ******. *** ****** of ***** ****, ***** we **** ***** **, they ******** *** ********** in ***** ***** ***** (FMR) ** ****** ***** from ****-******* ** ***** Department *********** ******** (***/*** 39794-5 **************). ** ******* of * ***** ***** would ** ** * friend ******** **** ****** using ***** ****, ******* of *****. ** **** case, **** ******** ***,*** visa *********** ****** **** 441,517 ********* *********** ******, selecting ****** *** ******** to * ****** ** countries ****** *** *** historical ***********. *** **** researchers **** **** ***** of *** ****** ** false ******* **** ******** (all ******* **** ***** because *** **** **** disjoint).

**** ***** *********** ** false ***** *****, **** false ***** ***** ****** in ******* ********* (******, Ukraine, *** ******) *** highest **** **** ******** and **** ****** (*****, Japan, *****, ***********, ********, and *******). *******, *** study ***** ****

**** * ****** ** algorithms ********* ** ***** this ****** ** ********, with *** *****-******** ***** on **** ***** *****.

******** **** ******** **** or ********* *********** ***** be ****** * *** of *** ****** *** ethnic **** ******* ** most ** *** ********** they ******.

IPVM Image

How ** ********* *********** ***********

***** **-*** ***** **** accurate *** *. ********* than *. ******, ****** like * ***** **********, this *** ** **********. The *********** ** ********** can ** **** (**** algorithms *** ***** ***** of ~.***** *** *. Asians *** ~.****** *** E. *********), *** ** some *****, *, ** close ** * ****** would ** ********. ** an ********* ** **** for ************/*:* ********, **** as ***** ****** *********** to ****** **** ******, mistakes ***** ** ********** infrequent *** *** ********* would ** ****** ************. However, ** **** *** identification/1:n *********, **** ** scanning ** **** *** anyone **** ** *********** warrant, *** ***** **** increases ************* **** *, and *** ********* *** have * **** ****** (even **** **** ***) if * ** ***** enough.

**, ***** *** *********** differences *** ***, *** difference *** *** ** an ***** **** *:* verification *** *** ** a **** ***** (**** disparity ****** **** ***) if **** *** *:* identification.

Effect ** ********* ******

**** **% ** ***-****-***** made ********** ********* ****** (a ***** ***** ***** rate) ** ******* ******** faces **** ** **** Asian ****

***, ~**% ** ******* firms (********* *****, *********, and ******) *** ~**% of **** ***** ***** (expanded ** ******* ********* like ****** **** *** in **** **** *** not ** *** ***** of ********* ****** *** the **** *****) ********* better ** **** ***** faces **** ******* ********, as *** ***** ***** highlights:

IPVM Image

********

***** *** ******** ** widespread *********** **** ** a *******, *** **** that ********** ********* ** Chinese *** **** ***** teams ****** * ********* bias **** **** ** other *****, ******* **** much ** *** *********** disparity ** **** *********** accuracy *** ** ****** by *** *** ** diverse ******** ****. **** means **** ********** ****** ensure **** **** *** diverse ******** ****, ******* of **** ******* ** what ** ******* ** use. ** **** ***** that ********* ** ** teams *** **** ** better **********.

** ********, ***-***** ****** ask ***** *********** *********** in *** ********** **** are ******** ******* *** take ********* ****** **** consideration. ***** ** ********* may ******* ** ********** on *** **** ***********, it *** ************ ** the *********** ** ** used **.

***********

***** *** ***** *** rigorously *** **** ***, it *** *** ******* the ******** **** **** or **** ** ********* cause *** ******. **** means **** ***** ** can **** ******** ******* based ** *** *****'* findings, ** ****** ************ state ******** ** ****.

****, **** *****:

*** *** *** * corresponding *********** *** ***** Asian ***** *** *** few ********** [****] **** submitted ** ********** [** India *** **********].

*******, ** ** ***** possible **** ***** ***** Asian **** **** **** that *** *** ******* a ****** ********** ** South ***** **** ****** than ***-***** ***** ***** did. ** ** **** to **** ** ******* data **** ** ***** that *** ***** ****** to ****** ****** *********** neural ********, ** **** teams *** **** ****** on *** **** ****** accessible ********.

*** *** ** *** same ******** *** ******* why **** ***** **** clustered ******** *** *** bias *** ***** **** uniform, **** **** ********** performing ****** ** *** same ******** *** ****** groups.

Poll / ****

Comments (6)

John Honovich

IPVM | 07/15/20 04:19pm

****, **** ***!

***** *** **** ****** was **** * ****** ago, ****, **** ****, is ******* ** **** coverage ** ****** *********** technology. *** *********** *** topics ** *****, ****** share.

Avatar

Salvatore D'Agostino

IDmachines | 07/15/20 04:45pm

** **** ******* ****** out *** **** ****** did *** **** ** training ****. ** *** FR ******** ************ ******* on ******* ********, **** is ***** ****** ***** seem ** ** **** and **** ********. ******* a ****** **** **** training **** *** ******* a **** ** **** is *** ***** ** open ****** ****. **** will, ** ** *******, be ** *********** ***** to ****** **** ** not **** *** ****** to ***** **** ****** acquisition *** ******** **** becomes * *** ******** block ** ******** *** also *** ****** ** which **** ****** ******** sets. *** *********** ****** in **** ******** *** the ******* ************** ****** amassing * ***** **** set ** ******'* *****. This ***** ** *** question **, ** **** extent ***** * ********** training *** ** ******* that ***** ******* *** challenges ** *********** *** bias.

Anthony Hackett

In reply to Salvatore D'Agostino | 07/15/20 05:33pm

**** ***** ** *** question **, ** **** extent ***** * ********* training *** ** ******* that ***** ******* *** challenges ** *********** *** bias.

***** ***** *****, * can *** * ********* type ** ************* *** all ********** ************. **** would *** **** ***** to ** *** *** AI ** **** *** general ****** *** ******* entities *** ****** ** ethical *********** ** *** algorithms ***** ****.

Zach Segal

IPVM | In reply to Anthony Hackett | 07/15/20 06:54pm

* ***** ** ******* (Timnit *****, ***** ***********, Briana *********, ******** ******* Vaughan, ***** *******, *** Daumé ***, **** ********) wrote ** *********** ******* proposing ********* ***** ***** lines ********,********** *** ********. **** ***** **** all ******** ****** **** with ********** *** ***** use *** *********** ***** how *** **** *** collected *** **** ** contains.

** ***** ** *********** if *** ********** *** a "**** **** ***** dataset" ***** *** ****** accessible *********** ***** **** data.

***, *********, ********* **** is ** *********** ****, but ** ***** **** lead ** ********* *********** in ******** ******* * machine ******** ********* **** invariably **** *** ********* data ***** ** **** data. *******, ** ***** definitely ***** ******* ******.

Avatar

Salvatore D'Agostino

IDmachines | 07/15/20 07:05pm

** ****, ***** *********. I **** *** ****, its **** ******. ***** is * *** ***** on ***** *** ****** the ** *** *****. My ***** ** ***** will ** * *** more ** **** ** this *****. ******* **** is ******** ** ******* AI, *** **** * human *********** ***'* **** this ***** ** *** fore. ***** ** **** interesting ******* **** ** de-identification, *** ***** ******* related *******.

Zach Segal

IPVM | In reply to Salvatore D'Agostino | 07/15/20 07:14pm

***** *** *** * agree. ** ** * very ********* ***** *** only ******* **** ********.

Read this IPVM report for free.

This article is part of IPVM's 6,582 reports, 886 tests and is only available to members. To get a one-time preview of our work, enter your work email to access the full article.

Already a member? Login here | Join now

Related Reports

Face Masks Increase Face Recognition Errors Says NIST on Aug 04, 2020
COVID-19 has led to widespread facemask use, which as IPVM testing has shown...
Ear Recognition Examined on Oct 12, 2020
Will ear recognition become a viable biometric? Facial, fingerprint, and iris...
Exacq "Facial Matching" Facial Recognition Tested on Sep 03, 2020
Exacq is claiming "Accurate Recall of Every Noteworthy Person of Interest"...
Herta Facial Recognition Plus Masks Tested on Aug 19, 2020
Masks increase face recognition errors, but facial recognition developer...
Face Detection Shootout - Dahua, Hanwha, Hikvision, Uniview, Vivotek on Jul 30, 2020
Face detection analytics are available from a number of manufactures...
The US Fight Over Facial Recognition Explained on Jul 08, 2020
The controversy around facial recognition has grown significantly in 2020,...
TVT Temperature Measurement Terminal Tested on Jul 23, 2020
While Dahua and Hikvision get the most attention for China temp products,...
Beware Rigged China Fever Cameras on Sep 08, 2020
Many China fever camera manufacturers have rigged algorithms dynamically...
FaceFirst Problems And Layoff on Oct 01, 2020
FaceFirst, a US company and one of the oldest ongoing facial recognition...
U.S. Government Accountability Office Urges Facial Recognition Regulation on Aug 27, 2020
The US Government Accountability Office (GAO) is urging facial recognition...
Gait Recognition Examined on Sep 14, 2020
Facial recognition faces increasing ethical and political criticisms while...
Verkada Video Analytics Tested 2020 on Oct 08, 2020
In 2019, Verkada released people and face analytics but our testing then...
FLIR Screen-EST Screening Software Tested on Jun 30, 2020
In our FLIR A Series Test, the cameras' biggest drawback was their lack of...
Hanwha AI Object Detection Tested on Sep 28, 2020
Hanwha has added detection and classification of people, cars, clothing...
UK Court Rules Police Facial Recognition Needs Reform on Sep 01, 2020
A UK court has ruled that the South Wales Police use of facial recognition is...

Recent Reports

Panasonic Presents i-PRO Cameras and Video Analytics on Oct 19, 2020
Panasonic presented its i-PRO X-Series cameras and AI video analytics at the...
Augmented Reality (AR) Cameras From Hikvision and Dahua Examined on Oct 19, 2020
Hikvision, Dahua, and other China companies are marketing augmented reality...
18 TB Video Surveillance Drives (WD and Seagate) on Oct 19, 2020
Both Seagate and Western Digital recently announced 18TB hard drives...
Watrix Gait Recognition Profile on Oct 16, 2020
Watrix is the world's only gait recognition surveillance provider IPVM has...
Intel Presents Edge-to-Cloud Ecosystem for Video Analytics on Oct 16, 2020
Intel presented its processors and software toolkit for computer vision at...
Best Manufacturer Technical Support 2020 on Oct 16, 2020
5 manufacturers stood out as providing the best technical support to ~200...
Microsoft Azure Presents Live Video Analytics on Oct 15, 2020
Microsoft Azure presented its Live Video Analytics offering at the September...
Worst Manufacturer Technical Support 2020 on Oct 15, 2020
4 manufacturers stood out as providing the worst technical support to ~200...
Clorox Announces, Then Pulls, Fever Camera on Oct 15, 2020
For almost one week, Clorox was marketing fever cameras. The booming...
Faulty Hikvision Fever Cam Setup at Mexico City Basilica and Cathedral on Oct 14, 2020
Donated Hikvision fever cameras (claiming screening of 1,800 people/min. with...
Directory of 209 "Fever" Camera Suppliers on Oct 14, 2020
This directory provides a list of "Fever" scanning thermal camera providers...
Avigilon UMD / UAD Tested on Oct 14, 2020
Avigilon's Unusual Activity Detection and Unusual Motion Detection claim to...
Longse Promoting Hikvision Partner Fullhan Chip Based Cameras on Oct 14, 2020
With Huawei HiSilicon production being shut down at TSMC, camera...
Meridian & Goodview (BEMS Relabeller) Temperature Screening Tested on Oct 13, 2020
A lot of temperature tablets look exactly alike and that is because they use...
Monitoring Alarm Systems From Home - Innovation or Danger? on Oct 13, 2020
Remote monitoring by alarm companies since COVID-19 is bringing cost savings...