This is an IPVM report available to you via this special link only until May 22, 2024 04:00 AM

Masks Cause Major Facial Recognition Problems

Rob Kilpatrick
Published Feb 24, 2020 06:03 AM

Coronavirus is spurring an increase in the use of medical masks, which new IPVM test results show cause major problems for facial recognition systems.

IPVM Image

IPVM tested four facial recognition systems to see how they performed with masks:

The 1-minute video below overviews our findings:

Masks Drastically Reduce Recognition Performance

In our tests, facial recognition confidence dropped dramatically (over 50 points confidence) when wearing anti-viral face masks, with subjects not recognized at all in most cases, and no faces even detected much of the time.

Only Avigilon's Appearance Alerts was able to consistently recognize subjects, but this required confidence to be decreased from high to medium or low. While recognition was possible, it is certain that this will result in false identifications in large populations with hundreds or thousands of subjects.

Further Training Possible

It is possible that these manufacturers could improve masked face performance with further training of their algorithms. However, fundamentally, analytics will have much fewer points to base recognition on as so much of the face is covered, making recognition difficult, even with training.

Drastically Reduced Confidence

Shown below, when compared to the same subject not wearing a mask, confidence dropped by over 60 points when wearing a medical mask, from nearly 100% confidence to just over 30% in the Hikvision dual lens face rec camera.

IPVM Image

In order to recognize a face at these levels, minimum confidence needed to be reduced to ~25%. However, recognition remained inconsistent at this setting, with many subjects detected, but not recognized. This was especially common when hair covered the subject's forehead, making it more difficult to detect the face.

IPVM Image

IronYun performed similarly, with drastically reduced confidence and frequent total misses, even when using low similarity score requirements. For example, the face below matched with only 30-40 similarity, instead of 60-70 when the same subject was not wearing a mask.

IPVM Image

Avigilon Face Recognition Most Accurate But Only Medium/Low Confidence

In our tests, Avigilon's facial recognition had the best recognition rates, recognizing all subjects walking through the scene when using medium or low confidence settings (Avigilon does not show exact confidence scores).

IPVM Image

However, note that while recognition worked in our tests using a small population, users should not assume it would function similarly in larger installations where populations are more diverse. It is certain that false recognitions and missed recognitions will increase as the number of subjects increases because with more people this is more opportunity for people to look or appear to the system as looking similarily.

No Faces Detected In Masks: Verkada

In our tests, Verkada facial recognition completely missed faces of people walking through the scene when wearing masks. These subjects's faces were not detected at all and not searchable from Verkada's face list.

IPVM Image

All Systems Had Miss Detections

Our testing showed many miss facial detections - meaning that the system did not even recognize a face was present, which is necessary to even try to categorize what person's face it is:

  • Avigilon rarely missed faces walking through the scene.
  • Hikvision had the second best recognition rate but at the cost of a lower confidence threshold of 25. The default higher threshold had significant misses.
  • Ironyun missed people frequently and barely detected faces in the scene.
  • Verkada did not recognize anyone in the scene wearing a medical mask.

Improvements In Facial Detection With Masks Certain

Detecting a face with a mask is a solvable problem. The issue is most systems likely did not train their systems to recognize faces with masks. This can be solved by adding new training data of faces with masks on. As such, we would expect future firmware upgrades from many vendors to include that.

Harder Challenge To Recognize Faces With Masks

While detecting performance should go up, recognizing performance faces fundamental limits. Masks hide the chin, nose, mouth, lips, and jaw, essentially constraining systems to match on eyes and forehead.

IPVM Image

Trying to recognize faces can still try to be done and even sometimes or often be correct but fundamentally the accuracy, especially with larger scale, will be far reduced.

The best-case scenario is, structurally, building access control where limited populations, well-controlled lighting, and people presenting their faces increases the odds of accuracy. For example, China's Hanwang Technology is already claiming to do so, as their marketing video below demonstrates:

How well it actually works and how many mistakes are made is not explained. However, access control can also be tuned to allow more matches / access if the assumption is made that it is better to let people in quickly than block valid users from entering.

By contrast, facial recognition in video surveillance, e.g., the oft-cited smart city examples are going to face far harder examples - outdoors, poor weather, wide areas, etc., that combined with face masks are likely to make facial recognition mistakes the norm, even with optimizations.

Comments are shown for subscribers only. Login or Join