Dahua Thermal Temperature Monitoring System TestedBy: Ethan Ace and John Honovich, Published on May 11, 2020
As such, it certainly works, at some level, but how well does it work to detect fevers (or as they market it 'elevated skin temperature detection')? And what does Dahua do to estimate human temperatures?
We bought and tested the Dahua system. Inside this report, we explain:
- Dahua's 'confidential' 'compensation algorithm' and the practical impact of this controversial technique
- How Dahua delivers such 'accurate' readings even under bad conditions
- How Dahua misses 'elevated skin temperature' detections
- How software setup impacts performance and measurements
- How Dahua's engineering recommendations conflict with Dahua's sales and marketing
Our testing of Dahua’s system found that it produced remarkably “normal” human body temperature readings, even with hard to read conditions like occlusions (hats, masks, hands, hairs, while walking, and even partial headshots). This creates a feeling of comfort and safety that was impressive. Moreover, Dahua could not be fooled by hot water bottles on foreheads at 110°F, for example.
On the other hand, our testing found Dahua regularly missed elevated skin temperatures of 100 - 102, raising a serious risk about the system missing many people with fevers, especially when subjects were moving, wearing hats, or moving next to others.
Moreover, our testing revealed that the remarkably normal temperature readings Dahua produces are artificially enhanced by what Dahua described to us as a ‘compensating algorithm’ that we found dynamically increases the offset to transform low / bad reads into more normal temperature readings.
Our testing found that Dahua regularly missed subjects with elevated skin temperature in scenarios shown in their own marketing, simply walking and even more so when wearing hats and sunglasses, ski masks, multiple subjects, etc.
To simulate fever, our subjects elevated the temperature of the forehead using a hot water bottle in the low 100s° F. The subject was then measured by the Dahua camera, and finally by a handheld IR thermometer (Extech IR200).
While health and safety measures prevented measuring actual sick or coronavirus infected people, this approach is similar to what manufacturers like Dahua use to prove their systems can alarm. Our approach differed by controlling the heated temperatures to realistic fever levels and verifying those.
Our methodology is conservative in that (1) we heated the entire forehead and (2) we had Dahua measure the subject before we use the IR gun and in the intervening few seconds between Dahua and the IR gun's measurement, the subject will, minimally, cool slightly.
In the video below, we show examples of subjects with fevers being missed, due to hats, ski masks, hair, and simply low readings by the Dahua camera.
In the first example, after heating his forehead with a hot water bottle, the Dahua camera measures the subject at 99.6°F, while he is measured via thermometer at 100.6°F, a full degree difference.
Directly following two others through the scene, the subject is measured at 99.9°F by Dahua, but again 100.6°F using an IR thermometer, over the camera's alarm threshold.
Wearing a hat and sunglasses, as shown in Dahua marketing, the subject is measured by Dahua at a max of 100°F (under the camera's alarm threshold), but a much higher 102°F using the IR thermometer.
Similarly, wearing a knit hat/beanie, the subject was measured at 98.5°F, but 101.5°F when removed.
Dahua also claims to detect elevated temperatures when wearing ski masks/balaclavas. However, the subject below was measured by the Dahua camera at 98.4°F while covering his forehead, which increases to 101.5°F when removed.
Finally, simply having hair covering the forehead prevented proper measurement of elevated temperatures. As the subject enters the scene below, Dahua measures his temperature at 99.2°F, but 102°F when his hair is pushed aside.
Response From Dahua on Misses
Below is Dahua's response, in full, on the misses:
> Big temperature difference while object is moving
After days of test, we cannot seems to replicate this issue, need to identify what’s the cause, attached is two video taken in both R&D and field.
> Regular face with heat applied, not detecting elevated temperature
We really cannot replicate this issue, temperature difference might be exist, but the video show more than 1 degree difference, need to identify what’s the cause
> Baseball hat sun glass, beanie not detecting elevated temperature.
Understand and agree with your point, north America will update our marketing material to reflect on detection on conditions where the eye canthus is showing.
We are confident is the person with beanie is really on an elevated temperature, we can detect it since his eye canthus will show elevated temperature instead of just forehead.
"Compensating Algorithm" Investigated
Dahua acknowledged they used a "compensating algorithm" in response to our questions about why the temperature offset varied between the actual surface temperature of objects and the reported human body temperature when Dahua detected a face. Dahua explained:
We have shared a lot of information about how the technology is, how our solution works, how to make necessary setup to show best test result in the past couple of emails.
If you are asking specifically about the algorithm, this is a highly confidential intellectual property of our company, I hope you understand why we cannot share any further details.
You can think like this, a thermal sensor can work out of the box, anybody can buy a thermal sensor and get read outs from it, what makes that thermal sensor to a body temperature solution is the compensating algorithm. That’s why not every thermal company has human body temperature solution.
While the algorithm is confidential, we were able to determine how it acts under varying temperatures. The lower the temperature reading Dahua has of a face, the more it increases the offset to make the reading look closer to 'normal' or statistically average human temperature.
We were able to uncover this because Dahua does not have liveness detection and thinks printouts of faces are actual human faces. Using that, we placed controlled heat sources behind the printout face (e.g., a blackbody, that controlled the heat level behind the printed face) to determine how Dahua would report specific surface temperatures to as 'human' temperatures, as we demonstrate in this video below:
Though this is certainly very clever in that it increases user confidence that the system is working well, this is deceptive because it transforms low / 'bad' reads into reads that look good. And because literally 98% of humans have temperatures between 96°F - 99° F, to a casual observer it looks like it is 'working'.
Examples Bad Reads Become More Normal Temperature
While thermography systems tend to generate low readings on bad angles or positioning (e.g., 91°F, 92°F, 94°F, etc.) Dahua never did so. This is consistent with our research showing that Dahua boosts bad reads into nearly normal measurements.
For example, even when the subject covered the top half of their face, obscuring the eyes and forehead (the warmest parts of his face), Dahua displayed a 96.8°F measurement.
Similarly, in an even less realistic scenario, the subject glides past on a skateboard, detected and measured by Dahua at 97°F.
Finally, when simply looking down or to the side, even at harsh angles, the camera continues to produce nearly normal measurements. In the example below, the subject is measured at 97.9°F when looking down, increasing to 98.8° when looking up, just a 1°F difference.
Subject Temperature Similarities
In our tests, Dahua tended to measure subjects in a very small temperature range, while IR gun thermometers varied much more. We tested this using 12 subjects of varying gender and age (ranging from infant to middle age). When walking, 8 of 12 subjects were measured within the range of 97.3-97.7°F. However, when measured using an IR thermometer, those same subjects varied by 1.2°, from 98°F to 99.2°F, a much greater difference.
Moving Vs. Standing Measurement Differences
During testing, subjects were frequently measured significantly lower when moving than standing, with temperatures increasing slowly the longer subjects remained in the scene. For example, when walking normally, the subject below was measured at 97.6°F. However, when remaining still for 3-4 seconds, his temperature is measured at 98.2°F.
These effects were seen on all subjects, with some warming faster than others when remaining in the scene.
Camera Configuration Impact On Accuracy
Several settings in the Dahua thermal camera are important for measurement accuracy, including blackbody setup, human temperature compensation, and ambient temperature. These settings should also be regularly reviewed to ensure accuracy as human temperature and ambient temperature change throughout the day and blackbodies may be bumped or knocked out of alignment.
We review these settings in this video:
Engineering Recommendations Conflict With Sales and Marketing Claims
Our testing also revealed a conflict between what Dahua engineering recommends and what Dahua sells and markets publicly, particularly as it comes to people moving and people's heads being covered, two of the most distinctive claims compared to conventional thermal providers.
While Dahua's marketing shows various examples of people's heads being covered, Dahua engineering objected to us using them. For example, from Dahua's website [UPDATE Dahua removed the ski mask, sunglasses and hat and helmet images after IPVM called it out]:
Dahua's engineering team understandably responded, "since the heated area(forehead) is covered the camera physically couldn’t get a proper reading". We covered the forehead to the same level as Dahua's marketing images. So while we agree with Dahua engineering that these coverings negatively impact accuracy, Dahua marketing had prominently claimed they can deliver accurate readings with such coverings.
Likewise, Dahua has publicly repeatedly touted its ability to do readings with people walking in a straight line one after the other, most notably in their marketing video that has received 150,000+ views:
However, Dahua's installation instructions and recent 'whitepaper', confirmed by Dahua engineering to IPVM, contradicts the straight line, one after the other, approach. Instead, Dahua calls for
Moreover, the instructions emphasize to use it only indoors and even indoors not aimed at entrances and to use a queuing area, as shown above and below:
However, in Dahua's case study video, they show an outdoors setup, excerpted below:
Consistent with Dahua's installation instructions and contrary to the case study, we generated numerous false alerts, as the example below shows:
The Dahua system had read the temperature of the pavement directly behind the person's head which was ~100°F on a ~70°F day.
Dahua's "Thermal Temperature Monitoring Solution" sells online for ~$12,000-13,000 USD, total, similar to body temperature measurement systems from Hikvision and Sunell OEMs (~$12,000-15,000 USD).
Individual prices for components (online):
- DH-TPC-BF5421-T camera: ~$8,300 USD
- JQ-D70Z blackbody calibrator: ~$3,600
- DHI-NVR5216-16P-I 16-channel NVR: ~$1,000
The DH-TPC-BF5421-T camera is physically similar to other Dahua bullets, with the addition of the second thermal lens/imager. The camera contains a white light illuminator and speaker, which trigger on over-temperature alarms by default.
The JQ-D70Z blackbody is similar to others in our tests, such as Hikvision and Sunell, about 4" square by 7" long, and adjustable from 30°C to 50°C (86° - 122°F), though defaults to 35° (95°F).
We review the construction of the camera and blackbody in this video:
Live Monitoring Temperature Events
The simplest means to view temperature events from the Dahua camera is using the local interface of their NVR (DHI-NVR5216-16P-I), which displays events along with statistics as temperature measurements occur (demonstrated below). The camera/NVR web interface or Dahua's Smart PSS client do not display these events, with the face bounding box turning red the only indication of an over-temperature event. This makes monitoring more difficult as temperatures frequently only shifted to alarm levels for one or two frames in our tests, which could be easily missed if viewing only a live stream.
These events are integrated to Dahua's DSS Express VMS, but this requires an additional license, unlike their Smart PSS CMS client.
In addition to monitoring methods above, the DH-TPC-BF5421-T is ONVIF Profile S and T conformant and visible and thermal streams integrated to Avigilon, Exacq, Genetec, Milestone, and Network Optix VMSes in our tests with no issues. However, only Milestone integrated temperature events, via an add-on plugin.
Dahua's remarkably normal temperature readings do a great job of giving people peace of mind but our testing shows that it comes with a significant risk of missing real fevers. For those who want the convenience of allowing people to wear whatever they want and move as quick as they want, without prioritizing fever detection, Dahua excels.