Dahua Face Recognition Camera Tested

By Rob Kilpatrick, Published on Oct 15, 2018

Dahua has been one of the industry's most vocal proponents of the value that AI creates:

As part of this, Dahua has released a facial recognition camera, the DH-IPC-HF8242FN-FR, claiming:

Artificial Intelligence at the edge – performing complex real-time facial recognition and facial feature comparison

But can this new model really deliver face recognition in real-world scenes? We bought and tested the HF8242 to find out, answering these questions:

dahua face recognition ipvm

  • How many false negatives or positives to expect?
  • How well does the camera handle subjects looking down or away from the camera?
  • How does recognition perform in low light?
  • How does recognition perform in <1 lux with IR?
  • How does recognition perform in WDR scenes?
  • What imaging and environmental issues impact recognition performance?
  • How well did the camera capture demographics such as age, gender, facial hair, and others?

Good / *****

*** **** ******** ****** of *****'* ****** *********** camera ** ****, *** small ******* *** *** traffic *****, *** ****** generated *** ***** ********* minimizing *** ************** ****** in **** ********** ****** recognition *********. **** ** did ********* * ****, overwhelmingly ** ********** ** accurately, *** *** *** recognize ********* ******* ** faces ** **** ***** "AI" ******** ****, *.*.,********* ********** *********** ******* as **** ******.

Bad / ******

*******, *** ***** **-***-********-** had ******** ****** ****** it **** ********* ** recommend *** ******/**** ********** use.

  • *** *********** ****:*** ****** ****** ** recognize ~**%+ ** ******** whose ***** **** ********, due ** ******** ******, with ***** ** *********, subjects ******* ****/****, *** low ***** ******* *** majority ** ********.
  • *********** ********** ********** **** looking ****/****:** *** *****, ****** confidence ********* ** ** least ~*-**% **** ******** looked **** ** **** from *** ******, ****** it *** ******* ***** in ******** *******.
  • *********** ***/*** ***** ********** reductions:************, ********** ******* ** least *-*% ***** **** difficult ******** **********, ********* simple *** ******, *-* lux *** *****, ** dark ****** **** ** on.
  • ******* ** ** *********** under * *** (******* IR):***** * ***, *********** became ******, **** *** majority ** ******** ******. At ~*.* *** *** below, ********* *** *** work ** ***.
  • ******** ********* ** ******** recognition:*** ****** ******* *** person *** ******* * times **** *** ***** of ******* ** ******** scenarios, ***** * ***** database ** ***** ** people ** * ***** office. ** ****** ****** installations, ******** ** ****** to ********, ** ********** subjects ******** *** ********** that *** ****** **** look *****.
  • ******** ********* *********** ***********:*********** *********** ** ******** was ********** *****, **** male/female, *******/** *******, *****/** beard, *** ***** ******* incorrect **** ** *** time.

** ****** *********** ****** in **** *****:

*******

*** ***** **-***-********-** ***** for ~$*,*** *** ******, not ********* ****.

Adding ****** ** *** ********/*********

********' ***** *** ***** to *** ******'* ******** through *** *** *********. Users ****** ** ***** for **** ****** ** be **********, ***** **** and ***** ***** ***********, and **** ****** ** to *** ******.

**** **** ******** ****** must ** ********. ***** is ** ***** ** simply *** ****** ** tagging **** ** *****, for *******.

****** *********** ****** *** be ******** *** *** NVR, *** *** ******** on *** ****** ******, since ******** *** ****** on *****'* ****. ***** recommends * ******* ****** of ***, *** **** series (*** ******), ***** allows ***** ** ****** for ******** ********' ****** images. ******** **** ***** searching *** *********** ***********, but *** ******** ******.

demographic information search via nvr

Missed *********** ******

****** *** ******* *** camera ****** ** ********* about ~**% ** ********** subjects **** ****** ******* the *****, **** ***** Dahua's ******* ********** ****** of **%.

*** **** ******** ***** of ****** *********** *** subject ****** *****. *** example, **** ******** ****** down (** **** ** when ******* ********), ********* was ****** **** **** 50% ** *** ****.

frequent missed recognition due to head tilt

************ ** *** ******** face **** ****** **** missed ************, **** ** indoor, *** ******, *** when ***** *****'* **** exposure ******* ***** ******** to ********** ******** ** faces ** *** *****. For *******, ** *** image *****, *** *******'* facial ******** *** *********** by ****** ****** ******** in ***** ** *** with **** ****** *** in *** ****** ****** (though **** ******* ***** still *******).

https://user-images.githubusercontent.com/11630256/46947956-a23f3500-d0a6-11e8-824a-01a3d7015b21.png

~90% *********** ********** ** ***** **********

****** *********** ********** ******** about **% ** ***** conditions, **** *** ******* looking *******, *** ***** evenly ***, *** ****** mounted ** *** ***** recommendations (~*' ****, ~**° downtilt).

Head *******/******* ~*-**% ********** ********

****** *******, *** ********-** consistently *** ~*-**% ***** confidence ****** **** *** subjects **** *** ****** down ** ** *** side, ******* * ******* for ****** ******* ******* the ***** ******* **** at ***** ***** ** feet ** ******* ***** head **** **** *** camera.

head titled

Recognition ********** ***** ** *** *****/****/*** ******

*******, ******* ********** ******* confidence ** **** *******. For *******, *** ***** scenes (~*-* ***) ********* confidence ** ~*-*%.

low light

** *** ****** ****** (<1 ***) ***** ******** integrated **. ** **** scene ********** ******* **** ~91-93% **** ****** ** and **** ** **-**% with ** **.

external ir illumination

*** *** ******* *******, dropping ** ~*% ** a *** ***** **** a ********** *** ** ~3000 **. *** ******, shown *****.

wdr

Frequently ********** ************

*** ***** **** *** camera ******** ** ******** various *********** *********** **** faces *** ********, *********:

  • ***: * ******** *** is ********, *** ** searchable **** ** ***** categories (******, *****, *****, middle ***, ***)
  • ******/****
  • ****** **********: *********, *****, happy, ***.
  • ******* ****: ***/**
  • *** * *****: ***/**
  • ******* * ****: ***/**

** *** *****, ***** identifiers **** ********** ********* on *** ******, **** performance ******* ****** *** mounting ********* ******, ** various ******, ***** ******, WDR ******, *** *****, etc.

*** *******, *** ******* below *** ********** ***** as **** ** ***** old (****** ***: **), with *** ******* *******, and **** *** ******* a *****.

age beard

glasess

**** ******** **** **** frequently ********** **** ******** as ****** *** **** versa:

female

Multiple ***************

**** *** ****** ** ~2 ***** ** *******, the ****** ********** *** registered ****** *** ******* four *****, ***** * database ** ~** ****** in * ***** ******.

** ****** ******* **** more ********** *****, ***** issues *** ****** ** increase ** *** ******* two ****** ****** ******* increases **** ****** **********.

Full ****** ************* **** ********* ** **** ***

** ***** ** ****** all ********** ******** ** the ******, *** ***** IVSS *** ** ******** (~$4,500 ****** *** *** channel/1TB). **** ******** ********* for *** ********* ** a ******** ********** ******, specific ********** *******, *** more. ********* *** ** done ** * ***** NVR **** *** ********* tab, *** ** ******* to *********** *********** ****, with ** ******** **** search.

***** **** **** **** capability **** ** ******** in * *** ********, not *** ********* ** North *******.

Typical *** ****** **** ******

*** ********-** ** ******* to ***** ***** *** models, ****** ******** ****** and ******* ** *********** additional ********** ** *****.

similar construction to other box models

Versions ****

*** ********* ******** **** used ****** *******:

  • ***** **-***-********-**: *.***.*******.*.*, ***** Date: ****-**-**

Comments (21)

I may have missed this in other analytics testing, but how about facial recognition of people of color?  Perhaps NA and Europe may detect them better, but in Asia, there are way less people of color and perhaps less testing of them.

Also, people of color in low light......I assume have white complexion is much easier to "see."

We did not test this specifically, but it's a fair issue to consider. It's something that's come up in the past, as well.

Crimes are not only committed by white people ya know.....:)

Curious to whoever voted unhelpful, why is that?

This probably also related heavily to cameras having a natural bias in capturing darker skinned people. There's been plenty of photography discussions concerning this prerequisite issue.

Awesome reporting. I'm curious to know if you were to take photos at different angles and then upload those into the database if you would get more matches say when people are looking to the right or left as they walked down the hall. 

More importantly, where do I get an IPVM shirt?

Having multiple reference images with a person turning their head in various directions does help when compared to a single reference. Having the database structure handles multiple images may be an issue.

I'm curious to know if you were to take photos at different angles and then upload those into the database if you would get more matches say when people are looking to the right or left as they walked down the hall.

I tested this using a picture of my head tilted down and a picture of me looking at the camera and it didn't make a difference what reference image I used, it would still show a ~90% similarity rating if my face was clearly seen during the walk.

head tilt vs straight 1

Even when walking past the camera with my head tilted down, similarity rating dropped similar to what we show in the report but it was the same for both images.

head tilt vs straight 2

I dunno, is it just me or is the aspect ratio widely divergent in these pictures?

John, you may wish to look at the Facepro system by Panasonic.  The exercise that you mentioned is not necessary.  Facepro uses deep learning AI instead of machine learning.  Hence, it is able to handle more variations of face angles.

It is disappointing you can not select a detected person and add them to the lists.  You need a JPEG image to add them to the lists.

 

 

You can with VSS!

I'm curious how this stacks up against other companies doing this (the Hikvision report was definitely enlightening), such as Anyvision or Gorilla.  Even the Axis Demographic, any thoughts on a competitive shoot out?

With all the press Amazon facial recognition got, I'm curious to see what passes for "average" in the industry.  I've dealt with customers that are looking for 50% accuracy just to get security guard's attention, and other customers where under 90% is useless.

Good job IPVM team. This technology comes off as a fairly simplistic 2D approach with all the inherent drawbacks of uncontrolled pose and lighting. For biometric surveillance applications, I no longer consider simple 2D to be viable. Now with controlled and repeatable enrollment, followed by database search of same controlled images? Sure. But I am sceptical that even "Deep Learning" and "AI" can deliver a sub 1% EER in an unconstrained surveillance engagement. Better sensors (e.g., RGB-D) are required to achieve the kind of results that will be acceptable and practical. 

Good report! I would like to see a comparison between embedded facial recognition solutions, like Dahua, Hikvision and Axis (with Ayonix). It would be great!
Ayonix claims to do 3D FR embedded in any Artpec6 Axis camera.

 

Great information...thank you.  Is it generally felt then that facial recognition technology is in its infancy and not yet ready for primetime?  Is Dahua's technology a good first step in the right direction?  Most innovative technology follows the "technology gap" theory, and is this the first stage typified by early adaptors? 

Does this seem like this is the future of camera technology development?   When analytics started to come of age, it became more reliable and easier to deploy, yet it still does not appear to create great confidence among most end users. 

If analytics, coupled with facial recognition, reach high confidence levels, it's not hard to believe that every system will require these features.

Interesting report - and highlights that facial recognition (applied appropriately) is steadily coming into the mainstream if you can now get a workable if limited capability on a $1k camera from a vendor like Dahua.

Some of the issues here (such as lower confidence for angled faces) are true for most of current generation of lower-end facial recognition using fairly standard cameras - and that's where it comes down to your application, ie:

- do you have cooperative/ compliant subjects in an overt capture mode (who can be directed to look at camera in a controlled way, eg. for allowing access to whitelist), or non-cooperative / unaware subjects (covert capture, for catching criminals and for finding blacklist)?

- is the intended purpose more for information / data collection / general security (where false negatives aren't a big deal, and you can play with confidence levels - or is it for a biometric authentication / identification where false positives are intolerable)?

On the first point, even the best facial recognition system can be defeated in an uncontrolled scene if people know the camera is there or really want to avoid them. There are some novel approaches for drawing the attention of the subject to look at the camera (the most notable being the Dubai immigration aquarium!) that can address this to some degree.

On the second point, you'd never use a camera like this - or most standoff CCTV-based facial recognition systems - in isolation for a critical biometric authentication application (I'd hope!).

Hi, I did same test with same model in those days. Firstly my version is a bit later firmware than I see in Rob Kilpatrick test.
Firstly results are correct. What is not correct is usage. The test show where real users will do mistakes when will use such biometric systems, exactly same as IPVM did. In fact my test was done under controlled condition to understand why real results so different from claimed. Main reason is that users believe this recognition works under any conditions. Unfortunately not, it has quite strictly limited range for resolution/shutter/angle/time in scene/threshold.

In this I fully agree with Skip Cusack comment - biometric such kind need at least minimal controlled environment.
To Skip Cusak: I have to say that I was very surprised how system is able be precious. So far my test is I have to believe that under controlled environment such AI really can reach 1% EER. However really under controlled. F.e. idea of multiple loading same face under a bit different conditions does make sense and improves a lot result.


My results (trying to shorten as possible):
Positive recognition (all people in database):
From dataset of 736 frontal faces 98,9%. Faces up to horizontal angle 22,5deg still 99%.
However angles 45 deg or worse going down the hill, say to compare faces turned into angle 75 deg is nonsense.
(note 90 deg means that you are looking left or right, 0 deg is straight into camera).

Negative recognition (NONE of recognized people in database), 452 faces: 96% (mean 18 faces was wrongly detected as person in database).

Both positive and negative depends on correctly set Similarity Threshold. Yes, Dahua has in default set 82, but for reall application you MUST set your own. It is VERY sensitive number to get reasonable results. Results above was used with 85. After test was clear that ideal value to separate negative and positive recognition will be 87.
Here are results which you will get if Similarity result setup too high or too low:
Negative recognition (mean no one from face on display was in database):
71: 20% - mean 80% was wrongly detected as are in databes fro this threshold.
80: 55%
82: 83,7%
85: 96,7%
87: 100% - however this you cannot use for positiove recognition cause threshold for people in DB cannot be so hig.

Yes, to add same person multiple time allowing resolution rate, but it must be joined with increasing Similarity Threshold to reach higher precious rate.

So why the real test is just about 90% ??

This is cause I found that has very significant result those things: resolution, shutter, time in scene.
Resolution of captured face really cannot be bellow recommended 150x150. Do not forget that camera is just 2Mpix so when recording on longer distance to get much smaller part of faces is not hard. In fact it results that lenses must be used f>10mm. This also results that scene cannot be for this camera FOV about 2,9-3,3m.

Shutter - evaluated was influence of movement blur. Found that quite small blur have horribly high influence on results. In fact results shows that using 1/100 and shorter is necessary! So it will not work under IR far walking persons. However not for color reason, but for blur reason cause in night working with much longer shutter.

In fact those two things practically disallowing to use camera for more than 10deg from movement axis, cause higher difference between walking direction and camera angles makes blur much worse. Results when camera is installed at ceil will be logically low. This is point why f.e. usage of current cameras as source for FR will be problematic. Yes, for such cases you can make datasets of people from top.

BTW to make such database is not so complicated as in comment people crying (even Dahua can make more comfort there, I agree). There are stored all snapshot in your computer as far as you have opened GUI and then you can take them, sort, label and upload into database. Simple script solving it. So if you know all YOUR people will be recorder from roof you have to use in database all them looking down.

Also time in scene plays role. Even camera is able to catch at one time 16 faces, each face need to stay in picture minimally 0,7s. F.e. it means that this face must be during that time within DOF (yeah again...shutter/iris/light)! If not you have blurred image (if any). Tested extensively, smaller time destroying results.


Anyway even to application of FR cameras is not as easy as to use IPC with motion detection, is still unbelievably useful, definitely for Police. I have to note that mentioned IVSS and IVS series really have interesting capacities. Also I have to note especially for security job - software not only provide result YES or NO. It provide result also on question - "Show me person similar to this one on my photo" and operator can in seconds change similarity threshold and limit number of results. So then is not necessary if was persons recognized on 80% or 85%, cause he can additionally limit area, time etc. and will get limited set of persons really similar searched one. So then check instead of thousand hours of video just few tens snapshots is easy!

BTW there is question about face color or race - can make you sure it does not play any role. It is matter of training dataset for neural network and current datasets are big mix of all races. Yes, author can make wrong a lot of things but this is not case.

How I tested: photo datasets used among CV/FR community, are available. Results above mentioned for classical FERET database. Datasets played on monitor, camera watching to them.

Version I have under test was:
Device Type IPC-HF8242F-FR
System Version 2.622.0000000.9.R, Build Date: 2018-08-23
WEB Version V3.2.1.621486
ONVIF Version 16.12(V2.4.3.574996)
Algorithm Version S(1.7.2(V1.5.0))-K(31391)
Safe Baseline Version V1.3

Richard, thanks for the detailed feedback. You mention:

The test show where real users will do mistakes when will use such biometric systems

Main reason is that users believe this recognition works under any conditions. Unfortunately not, it has quite strictly limited range for resolution/shutter/angle/time in scene/threshold.

While I agree that enforcing 'strictly limited range' will improve performance, overwhelmingly 'real users' are not going to accept that. Confining to the 'strictly limited range' increases complexity and cost in setup and restrict how and where it can be used. Your thoughts?

Hi John,

already price of equipment is limiting who is interested in. So I guess 'real users' are those for which such system make real reason for such money. My opinions is to tell others that technology is in fact great, but has application limits. There is no reason to inflate AI bubbles but on other hand is no reason to say the thing is crap. Simply real installers, as for every IVS systems, should in first think about installation position for it. And I hope it can save some of them away of application disaster.

It will be same for all future system called "AI" - marketing dep. are usualy not able to say details of applications cause do not understand them and documentations is very weak, sometime intentionaly not to frighten customer away...

F.e. it is pitty that I did not get into hands that Hikvision AI system which you have tested as unsucesfull. With understanding how the designer made constraints and trained neural networks I am convinced that it is possible to make it work also ;-)) Yes, you will answer that system should be more user friendly - unfortunately it means in CV more GPU/VPU power, which means higher cost. So we are where we are.

My toughts are that engineers around security systems should find a way how to applicate in reasonable way, simply to get some "feeling" for what the segment started called 'AI'.

And above all to stop marketing dep. to inflate capabilities of such systems by unacceptable "short cuts".

And above all to stop marketing dep. to inflate capabilities of such systems by unacceptable "short cuts".

Richard, I do agree with that.

I also agree with this statement from you:

sometime intentionaly not to frighten customer away...

But I do disagree here:

My opinions is to tell others that technology is in fact great, but has application limits. 

If the technology was 'in fact great', it would not have these 'application limits'. And with these limits, the market is limited to very few customers that are willing and able to conform to this.

Read this IPVM report for free.

This article is part of IPVM's 6,602 reports, 890 tests and is only available to members. To get a one-time preview of our work, enter your work email to access the full article.

Already a member? Login here | Join now

Related Reports

Hanwha AI Analytics Camera Tested on Aug 11, 2020
Hanwha has released their Wisenet P AI camera, adding person and vehicle...
Augmented Reality (AR) Cameras From Hikvision and Dahua Examined on Oct 19, 2020
Hikvision, Dahua, and other China companies are marketing augmented reality...
Hanwha AI Object Detection Tested on Sep 28, 2020
Hanwha has added detection and classification of people, cars, clothing...
Face Detection Shootout - Dahua, Hanwha, Hikvision, Uniview, Vivotek on Jul 30, 2020
Face detection analytics are available from a number of manufactures...
Uniview Deep Learning Camera Tested on Jul 14, 2020
Uniview's intrusion analytics have performed poorly in our shootouts. Now,...
Hanwha 8K / 33MP Camera Tested on Sep 14, 2020
Hanwha Techwin has released an 8K / 33MP resolution camera, the TNB-9000 with...
Exacq "Facial Matching" Facial Recognition Tested on Sep 03, 2020
Exacq is claiming "Accurate Recall of Every Noteworthy Person of Interest"...
Quantum Dots Potential for Surveillance Cameras Explained on Sep 08, 2020
Quantum dots are starting to be used in TVs for better images, but how will...
Taiwan Lilin NDAA Compliant Cameras Tested on Aug 13, 2020
Taiwan-based manufacturer Lilin is taking direct aim at Dahua and Hikvision...
Verkada Video Analytics Tested 2020 on Oct 08, 2020
In 2019, Verkada released people and face analytics but our testing then...
Ear Recognition Examined on Oct 12, 2020
Will ear recognition become a viable biometric? Facial, fingerprint, and iris...
Single Frame Gait Recognition From Michigan State and Osaka University Examined on Oct 01, 2020
Gait recognition has the potential for accurate identification at a distance,...
FaceFirst Problems And Layoff on Oct 01, 2020
FaceFirst, a US company and one of the oldest ongoing facial recognition...
Herta Facial Recognition Plus Masks Tested on Aug 19, 2020
Masks increase face recognition errors, but facial recognition developer...
Face Masks Increase Face Recognition Errors Says NIST on Aug 04, 2020
COVID-19 has led to widespread facemask use, which as IPVM testing has shown...

Recent Reports

Recruiters Online Show LIVE Today! on Oct 29, 2020
IPVM's 7th online show resumes today with 12 recruiters presenting themselves...
Hikvision AcuSense G2 Camera Test on Oct 29, 2020
Hikvision has released their next generation of AcuSense analytic cameras...
Biggest Problems Selling Access Control 2020 on Oct 29, 2020
Access control can cause integrators big headaches. What practical issues do...
Taiwan Geovision AI Analytics and NDAA Examined on Oct 29, 2020
Taiwan manufacturer Geovision's revenue has been falling for years. However,...
Bedside Cough and Sneeze Detector (Sound Intelligence and CLB) on Oct 28, 2020
Coronavirus has increased interest in detecting symptoms such as fever and...
Fever Tablet Thermal Sensors Examined (Melexis) on Oct 28, 2020
Fever tablet suppliers heavily rely on the accuracy and specs of...
Verkada Fires 3 on Oct 28, 2020
Verkada has fired three employees over an incident where female colleagues...
Eagle Eye Networks Raises $40 Million on Oct 27, 2020
Eagle Eye has raised $40 million aiming to "reinvent video...
Hikvision Q3 2020 Global Revenue Rises, US Revenue Falls on Oct 27, 2020
While Hikvision's global revenue rises driven by domestic recovery, its US...
VICE Investigates Verkada's Harassing "RawVerkadawgz" on Oct 26, 2020
This month, IPVM investigated Verkada's sexism, discrimination, and cultural...
Six Flags' FDA Violating Outdoor Dahua Fever Cameras on Oct 26, 2020
As Six Flags scrambled to reopen parks amid plummeting revenues caused by the...
ISC Brasil Digital Experience 2020 Report on Oct 23, 2020
ISC Brasil 2020 rebranded itself to ISC Digital Experience and, like its...
Top Video Surveillance Service Call Problems 2020 on Oct 23, 2020
3 primary and 4 secondary issues stood out as causing the most problems when...
GDPR Impact On Temperature / Fever Screening Explained on Oct 22, 2020
What impact does GDPR have on temperature screening? Do you risk a GDPR fine...
Security And Safety Things (S&ST) Tested on Oct 22, 2020
S&amp;ST, a Bosch spinout, is spending tens of millions of dollars aiming to...