Nature Scientific Study Of Fever Cameras Confirms Accuracy Problems And 'Correction Algorithm'

A new study in Nature's Scientific Reports tested 3 fever camera systems on human subjects, found one used a "correction algorithm" and all had accuracy issues, independently validating IPVM's Journal of Biomedical Optics study that discovered fever camera temperature manipulation.

Inside this note, we explain what the Nature study found, the risk of fever camera users, and feedback from the USA FDA to IPVM.

Study Overview

The Australian researchers, Scott Adams, Tracey Bucknall & Abbas Kouzani from Deakin University, had 71 unique participants using an oral plus three thermal temperature / fever systems (1 measured the face, 2 others the inner canthus) comparing the results. Each subject was given at least 5 minutes to acclimate to the indoor climate which was tightly controlled, as the excerpt below shows:

The test used ideal measurement conditions with the person standing for a relatively long time and with no angle to the camera:

Each participant was then asked to spend 5 s standing in front of each IRT system facing directly into each camera at a distance of 1.5 m (indicated by a mark on the floor).

Manufacturers Not Named

The researchers would not disclose the names, brands nor models of the temperature / fever systems they tested, citing that they were loaned from the companies and did not wish to disparage the companies.

Correction Algorithm Found

One device, regardless of oral temperature, reported ~36°C to ~37°C for subjects, causing the researchers to conclude it was "employing a correction algorithm on the measurement results, which may shift the measurements into a more “acceptable” range". In the graph below, you can see it stays ~0.9°C below the fever limit, regardless of oral temperature.

The other two devices did increase slightly as oral temperature increased, but not to the level one would expect for an accurate body temperature measurement device. They also started far lower, so both read well below the febrile level even as oral temperature approached it. These findings with human subjects are similar to what IPVM found using a Blackbody as a subject.

Risk Allowing Febrile Individuals To Pass

The researchers concluded that the devices were less accurate than advertised and cautioned users from relying on them. They found that using them for screening is a potential risk:

Our results show that using the systems as a front-line intervention for fever-screening poses a potential risk, where the lack of measurement repeatability could negatively impact sensitivity and specificity, possibly allowing febrile individuals to pass through undetected. [Emphasis added]

They add that the systems were only weakly correlated with oral temperature, signaling inaccuracy, or an adjustment algorithm.

Larger Error than Advertised

The researchers found that the difference in two measurements of the same subject by the devices was worse than the 0.3 °C claimed by the manufacturers of two of the devices they tested:

In our experiment we had individuals take a measurement on each system twice within a very short time (approximately one minute). We found that those two measurements could be more than 0.3°C apart when comparing the first and second measurement.

IPVM analysis of their data found that the 95% confidence interval for bias of the devices was between ~0.3°C and 0.5°C, but it is unclear if the author's used the same method Bland-Altman analysis, as IPVM did to calculate the precision.

Researchers Warn of Real World Problems

While the study was done in tightly controlled conditions, the researchers warn of even more significant problems in real world usage that lacks acclimatization (e.g., screening people immediately as they open an exterior door to walk inside):

the acclimatisation is very important. If you come in immediately from an outside environment, particularly if there is an extreme of cold or hot outside you will measure the outside temperature more than the core body temperature.

FDA Responds

The FDA responded to IPVM, declining to comment on this study but emphasizing the importance of following IEC standards:

FDA does not comment on specific studies, and this would not be the format to evaluate a “correction algorithm.” The FDA expects thermographic temperature screening devices to conform to the FDA-recognized voluntary consensus standard IEC 80601-2-59:2017, Particular requirements for the basic safety and essential performance of screening thermographs for human febrile temperature screening, which describes factors for evaluating thermal imaging systems intended to measure human body temperature.

However, the FDA has tolerated mass violations of this IEC standard. For example, many of them (e.g., Hikvision, Sunell, most tablet providers, etc.) do not even attempt to scan the inner canthus, contrary to the IEC. Most flagrantly violate the IEC standards directive to have the face unobstructed by hats and hair as well doing screening in areas that are not temperature controlled, such as Eagle Eye's FDA-Violating fever camera school deployments.

This Nature study provides further evidence to health leaders that temperature screening is endangering the public.