NIST Facial Recognition Mask Accuracy Nov 2020 Results Analyzed

By Zach Segal, Published Dec 03, 2020, 09:51am EST (Info+)

In July, NIST found that face masks negatively impacted face recognition created before coronavirus significantly. Now, NIST released a report on recent algorithms developed after Mid-March for testing on face masked subjects.

IPVM Image

Do these new algorithms perform better with face masks? IPVM investigates.

Executive *******

**** ***** ******** ** ************* ****** the ******** ** **** *********** ********** despite ************* ****** **** ** ****** algorithms *** ****** ********. ***** *** an *********** *******, *** *** *** best-performing ********** **** ********* ***-******** *** several *** ********** *** ***** ***** close ** ***%. **** ***** *** a *********** ********* *** **** *********** algorithms *** ***-***** ****** ** ***** that *** ******** ** ***** ******* may ** ******* ******** ** ****.

Real-World ************

***-*****, ********** ***** ***** ************ ******* or ***** ***-**-**** ***********, *** ***-**-*** verification, **** ** ********** ** ***** face *********** ********* *** ****** **** widespread **** **** ***. **** **** was ******* **** **** *** *****, because ******* **** ***, ***** ******** at *****, *** ******** ********** **** shots, *** ******** *** ***** ******* impacted **** **** ********** ******* ** even ********* ****** ***** ** *****.

IPVM Image

** ********, *** ********** ********* **** no *********** ** ********** ***** ** time *** *** *** *********** ********** algorithms ** ****-***** ********* *** ******* even ***** ** ****** ********. ***-*****, especially **** ***** **** *********** *** tasks ***** **** ****** *******, ****** be ****.

New ***** ********

************************** ******** *** ****** ** ******* face ***** ** ********. **** ****~* ******* ****** ******** ****** ***** with ******* ** ******* **** ~* million **** *********** ****** *** ************/ 1:1 ******** (*.*. ** *** ****** crossing ***** ******* *** **** *****). The **** *********** ****** **** ****-******* and ***** ** *********** ******* **** a ***** **********. **** *** ******* as ***/*** ***** *.*. **** *** are *** * *** ******. *** border ******** ****** *** *** ****** with *** ***/*** *****-* ****-******* ***** standards *** ** **** ***** **** distorted. ***** ****** **** ***** ** trained ********* ** *********** ********** ******* harsh ******, ***** ***** **** *********** much ****** **** ************ ***-***** *** more ******* ** ****** ******* ******* WDR ******.

IPVM Image

**** ***** ******* “*****” ** *** border ******** ******. **** ****** * colors ** *** ***** ***** (*****-**** and *****) *** ***** * **** in *** ****** (*** *** *****), 2 ********** (**** *** *****), *** 3 ****** ** **** ******** (***/**** unobstructed, ******/**** ********* *******, ****/**** ****** up ** ***-*****, ********** ******** ****).

IPVM Image

**** ****** ** *** ***-**-*** ************ accuracy ** ***** **********/**** ***** **** **** *** ***** match **** *** ** */**,***. **** is **** ******* **** ***-**-**** (*.*., video ************) ***** ** ** ************* harder *******.

Not **** *** *******

**** *** ********** *** ****** **** but ********** *** ***** ******** ** masks, ** ****** ******* *******, *** the **** ********* *** ** *** first *****. * ** *** ** new ********** *** ********** ** **+% with *** ******-****-****-**** ***** ******** ** only * ** *** ******** ** algorithms, *******, ***** * *** ***** the *** **********.**** ***** ********* ********** **** **** greatly ********, **** ******* *&* **** went **** **% ** **% ********, but ***** *** ********** ********* ****** (up ** ***% ***** *****) ***, in *** **** ** ********* *+*, their *** ********* ********* ******** ****** than ***** *** ***.

IPVM Image

********

**** ***** ******** ******** ** ******* degrees, **** ****** ***** ****** **** impact, *** ** **** *****, ****** masks ****** * ****** ******.******** ******* ** *** *****, *** the ****** ** ****** ********. *** new **** *** ********* ********* ******* from **.*% ** **.*% **** * mask, ******* **** **.*% ** **.*%, and *** ********* ********* **** *** first ***** **** ******* **** **.*% to **.*%.

IPVM Image

**** **** **** *********** ** ****** subjects ** ******* ** ******** ** 2017 **** *********** ******* *****:

*** ******* *********** ** **** *********** with **** ***** ** ********** ** the *****-**-***-*** ** ******** ****** ** mid-2017

*** ******* ****** *** **** ********, which ** **** *****, *********** ** over **** *** *** ******** ** true ************ ********. ***** ***** *** a ****** ****** **** ***** **** (N95 *****). ****, ***** *******, **** darker ***** (***** *** ***) ****** a ******** ****** ****** **** ******* (white *** ****) *******, **** *** not **********.

Changing ********** *** *****

******** ********** *** **** ***** *** effect ** ****-***** ** ********** *** true ************ **** **** **** ******* effects ** ***** ***********, ** ***** in *** ******* *****.

IPVM Image

Exposing *****

** ** ** ****, ** *** make ***** *** ***** ** ***** their ***** ******** ***** ****, ** this *** ******* ******** *** **** verification ****.

*******

*** *** ********** *** *** *********** designed *** ***, *** *** **** processing ** ******* **********, *** ***** be ******** ** *** *********, ** the **** ** * *** **** algorithm ******* **** ***** ***** **** a *********** ********* ** ********* ********** that **** *** **** ****** ****** by *** **** ** *** ***** despite ** ********* ** ** **.

Comments (4)

Continued NIST testing is certainly valuable, but (as Zach noted) tests are...tests. I recently observed that some agencies/customers may choose to perform tests using their own data, rather than a third party's (NIST's) data. Of course, many agencies aren't going to have millions of pictures that are going to be used for such tests.

I'm curious about the usual database size that is maintained by IPVM readers. Are your clients' database sizes in the hundreds? Thousands? Millions? This could potentially impact the threshold settings that you use.

I hope that NIST continues to update these test results, in the same way that it is updating other FRVT test results. Perhaps in the future NIST may incorporate some other measurements into the mask tests, such as required processing power.

Agree
Disagree
Informative: 1
Unhelpful
Funny

Of course, many agencies aren't going to have millions of pictures that are going to be used for such tests.

That is one issue. It's a lot easier to do well on a companies employees or a few days of footage than over a long-continuous deployment with many times more faces.

I'm curious about the usual database size that is maintained by IPVM readers.

I am too. The discussion tool is a great place to ask questions of the wider IPVM membership.

I hope that NIST continues to update these test results, in the same way that it is updating other FRVT test results.

It looks like NIST is continuing to test newly submitted algorithms. And we will continue to update our coverage of the testing.

Agree
Disagree
Informative
Unhelpful
Funny

Imagine if NIST could use a database like Facebook, Google Photo or Shutterfly has of images across the whole spectrum for their testing.

I know I help Google refine their aging algorithms often, and for free.

Agree
Disagree
Informative
Unhelpful
Funny

NIST has access to some pretty labeled and extensive data courtesy of the State Department and other government organizations. They don't even need to do "10-year challenges", because visas, passports, and drivers-licenses all need to be renewed.

But they do lack surveillance style footage: taken from above head height, possibly at a distance, with various lighting conditions, etc. One big question is how well the tests predict performance taken from above with a large vertical angle.

Agree
Disagree
Informative
Unhelpful
Funny
Login to read this IPVM report.
Why do I need to log in?
IPVM conducts reporting, tutorials and software funded by subscriber's payments enabling us to offer the most independent, accurate and in-depth information.
Loading Related Reports