Debunking "99%+ Facial Recognition Accuracy" Claims
Many facial recognition vendors claim 99%+ accuracy, even citing government testing. But our testing and the experience of real-world video surveillance users shows accuracy is routinely far lower. How can that be?
In this note, based on IPVM's own testing (e.g., Facial Recognition Rankings 2023 - Avigilon, Briefcam, Hikvision, IronYun, Oosto, Rhombus, Turing, Verkada) and our analysis of US NIST testing, we explain this discrepancy, how testing works and what you should consider for real-world success.
Executive *******
***** **** ******* *** ** ****** up ***********, *** **** ***** **** testing, **, *.*., ***** ** ********** NIST *******, *** ******* *** ******** assumptions ** ***** ******* **** ***** what ******* ** ****-***** *****.
****** *********** ******** ****** *** ******** by *** *********:
- ******* ****** ** *******. **** ** ask *******, **** ***** **** ** supporting ******** ****** **** **** ****** themselves **** ** *****-***** **********.
- **** *****-***** ********** (**** ** ***/****) is *******, ** ** ********* **** on **** ******* ** **** ********* / ******* ****-***** ************.
- **** ** **** **** ** * problem **** ***** ************ ****** *********** than ****** *******, ***** *** *** harder ********** **** ***** ************ ******** under.
Unsubstantiated ****** *********** ******
****** *********** ******* *** ********** **** claims ******* ********** ******** ****** ****-*******. One ** *** ****** ****** ** "99% ****** *********** ********," ******* *********** any *****-***** ******* ***, ** **** cases, ******* *** *******.
*** ********* **** ***** **% ****** recognition ******** **** ****-******* ******* ***** on ****** ********. *** ********* ** not ******* ** *** ********** **** trained ************ ** ***** ********, ***** methodology *** *******, ** ******* ***** results ********.
Limitations ** *****-***** *******
********* ****** *********** ********** ********** (****, formerly ****)** *** ** *** **** ************* third-party ****** *********** *******, ** ** limited ** ******* ********** *** **** not ******* *** ****-***** ***********. ***** March ****, **** *** ******* ******* facial *********** ************ **** *** "****" dataset **** ******** ******* ** ****-***** images, *** *** ****** ** ******* to *********** ****** *********** ******* *******.
**** ****** ******* *** ******** ****** the *** *********** ********* (**** *** mugshot) *** *** ************* ******** (****), facial *********** ******** ********* *************. ***** vendors *** ******** ******** **** **** FRVT, ******* * ******* ******** ** facial *********** ******** **** **** *********** faces *** *********.
NIST/DHS ******* *********
*** **** **** **** ****/*** *** four ******* ****** **** ********** ********** levels ** ***** ****-***** *********. *** first ***** ** ****** *********** ******* is ****'* ****** *********** ******* **** (FRVT), ***** ***** *** ******** **********. The ****** ***** ** ***'* *** lab *******. *** ***** ***** ** the ********* ***** (******: **% ** ****** *********** ****** Caused ** *******), *** *** ****** ***** ** operational ***** *** ******** *** ************ and ***********.
**** ******* *** *********, ** ***** are ****** *********. *******, **** **** includes ******** ********** *** *** **** part ** ******** ******** **** ******** to * ******** **** *****. **** we **** *** **** [*****], ** actually *****'* **** * *** **** our *** ******* *** ******* ******* there. ** ** ***** *** ******** test ******* **** *** ***** *******... There ***** ** ******* *********** ** particular **************, ** **** ***** ** a *** ******** *** ******** *** cases. ** **** ********* ********* *** to ********* ****, ** *****'* **** quite ** **** ** ****'* **** it **.... *********, **'* ** ****** planning, ************ ** ********* ***** ******* the ******* *** ******* ** ********, or ******* **** **** **** ******* before ** ***** ** ******** **********.
***** **** *** *** ********** ****** facial *********** ******* ** "******" *********, they *** ********* *** ****** *******-**** scenes, *** ***** *********** *****'* ********* to ***** ************. ****'******* *********** ******** ********** **** ****** *********** *********** ** reduced ** ******* ****-**** ******, ** scenes **** ***-******* ********, *** **** accessories (****, *****, *******, ***.)
Based ** ******* **********
****'* **** *** ***** *********** ******* are **** **** ***** ******* *******, and ******* **** * ****** **** in *** *****. ***** *** ********* Rally (******: **% ** ****** *********** ****** Caused ** *******) *** ****** ** ****** ** the *****, *** ***** ** ****-***, and *** ************ *** ** *********** or ****-*********** *********, ***** ********' ***** are *******.
Not ***** ** **** ** ************ *******
***** *** ******* *********** *********** ** NIST *** *** *******, ** ** not ***** ** *** ********** *** available ** *********. **** ********** *** deployed ** ******** *******, *** ****** recognition *********** ***** ** ***********, *** resulting ** ******* *********** ** ****** results.
*** **** ********: