Corsight AI's Aggressive Claims Examined

By Robert Wren Gordon and Sean Patton, Published Nov 01, 2021, 10:04am EDT (Info+)

Corsight AI aggressively claims that its facial recognition software works under extremely difficult conditions like below with a person in a crowd wearing a mask and with his head titled:

IPVM Image

IPVM spoke with Corsight AI's CEO Robert Watts to better understand the company's claims.

Background

IPVM Image

Corsight AI is a Tel Aviv, Israel-based facial recognition company with operations in the UK and the US. Earlier this year, Corsight AI hired Tony Porter, the former Surveillance Camera Commissioner for England and Wales, as its Chief Privacy Officer.

An interview wherein Corsight AI CEO Robert Watts made numerous aggressive claims, including being able to recognize faces behind ski masks, was featured in an article entitled "What Role Will Facial Recognition Tech Play in Post-Pandemic Travel?," published recently by New Jersey-based TravelPulse.

IPVM reached out to Corsight AI regarding Mr. Watts's claims in the TravelPulse piece, resulting in a call with Mr. Watts, Mr. Porter, and Corsight AI operations head Adam Ashby-Clarke during which the company demoed its facial recognition software.

Facial Recognition Demo Overview

Corsight AI's demonstration to IPVM showed:

  • Quick matches (1 second or less) on large datasets/watchlists (e.g., 600,000) with a single NVIDIA RTX2060 graphics card.
  • High angle of incidence (up to 90°) for recognition.
  • 45 pixels "between the ears" for recognition or ~90PPF.

Our concerns from the demonstration include:

  • Images demonstrated on watchlists were not surveillance photos but rather celebrity-style headshots.
  • While Corsight AI reported an expected decrease in accuracy due to masks, hats, angles of incidence, lighting; Corsight could not disclose general or specific values. However, the company said that it shares this information with partners.
  • Default match confidence during demonstrations was a relatively low 55 which risks increased false matches in real-world conditions.

Update 11/11/2021, Corsight responded saying " Watchlist images should always, where possible, be as high-quality as they can be (for example, passport or "mugshot-style"). Corsight AI’s demonstration databases are representative of this and what are used in the real world by customers." IPVM's response is that often this is not possible and surveillance photos are used.

Corsight also responded saying "Corsight: 55 is not low for Corsight’s system. In fact, the recommended threshold for Fortify for typical use cases is 45, a threshold that results in a very low rate of false positives, even when using a DB of 10,000s suspects."

Percentage Accuracy Claims

Corsight offered the following data, with the last 2 rows representing harder scenes and lower accuracy:

IPVM Image

Corsight declined to share any examples of images from their "challenging scenarios" datasets so we cannot tell how "challenging" / hard they are. The company told IPVM:

The data we use is designed to include a large variability of quality degrading effects, pose, occlusions, and other conditions met in the customer's site. In order to have the set qualify as challenging, each dataset is assessed by quality indicators, such as the distribution of pitch and yaw angles, image sharpness and other factors that can help to characterize the set. For example, a challenging set would have a significant ratio of large angles and occluded faces. At this stage we cannot share specific information regarding these sets.

Corsight AI CEO: "We Don't Make False Claims"

IPVM Image

Corsight AI's chief executive defended his organization to IPVM, stating that there was "no need to be critical" of his business:

We don't make false claims in the market. There's no need to be critical of our organization or our business. I was very, very clear when I took over as Chief Exec, that there are organizations in, I suppose, this market whose marketing engines are far, far in advance of their delivery engines. But that will never, that will never be Corsight. [emphasis added]

Ski Masks, Hats, and Glasses Claim

Corsight AI's CEO is quoted, at the beginning of the TravelPulse story, saying that his company's software can identify people through ski masks:

We can identify someone with 90-degree head turn. We can identify someone with a camera at 70-degree rake, we can identify someone with a face mask—not just with a face mask, but with a ski mask...with hat, glasses, all of that[.] [emphasis added]

Update 11/11/2021: Corsight responded, saying "Corsight AI recognises faces with 45° angle elevation instead of 70° (outlined above). The article contained the wrong information, which was flagged to IPVM in a previous email."

In Corsight AI's CEO's comments to TravelPulse, he is quoted saying that "we can identify someone with a face mask—not just with a face mask, but with a ski mask...with hat, glasses, all of that," with the addition of "all of that" implying that Corsight AI is capable of identifying someone wearing a ski mask, hat, and glasses simultaneously.

However, Corsight AI walked back this claim, "absolutely" agreeing that a person wearing all three items would result in a significant confidence decrease and/or missed identifications. Nonetheless, Corsight AI's operations lead defended his company's ability to identify people wearing ski masks or hats or glasses without other accessories:

Right now I'm monitoring just over 600,000 people live, in real-time, on my system. So, there are 600,000 people in my watchlist right now that we are monitoring, I can put a hat on, I can put glasses on, the system, if I cover my face completely, of course, now I'm covering way too much of the face, I'm not going be able to do recognition. But if I take the hat and the glasses off, it can do it.

In observing Corsight AI's demo, IPVM noticed that Corsight AI delivered confidence scores in the fifties and low sixties on images of individuals wearing ski masks, in line with the photo below from an earlier IPVM report on Corsight, displaying a confidence score for the image of the masked man of only 65%:

IPVM Image

IPVM Image

Additionally, it is worth noting that Corsight AI considers the item displayed in the image on the right to be a 'ski mask,' with Mr. Watts declaring:

[That] is what we would call in Europe a ski mask, that is what I would wear when I go skiing.

In the United States, this garment is more commonly known as a balaclava, neck gaiter, or, more specifically, a bandana balaclava.

Matches on Dated Photos

Corsight AI also marketed its ability to generate matches and identify people based on decades-old photos. In the TravelPulse piece, Corisght AI's CEO is quoted claiming:

We can identify someone who is now age 50 from a photograph of them at age 17.

This echoes Corsight AI marketing claims, provided in a written response to IPVM:

Corsight AI supports images dated back 15-30+ years (while the competition doesn’t go over 10 years).

Mr. Watts stood by these claims in his company's call with IPVM while alluding to Ofer Ronen, the company's head of business development:

I know of our EVP of business development, he has images of himself at 16. And he's now in his 50s, late 40s/early 50s. And that just matches against him. I mean, it comes up at, I'm going to say 45% in terms of same time? So, it's a lowish threshold, but it's still giving a match, it's still giving a strong match.

IPVM Image

However, when Corsight AI demoed matches against dated photos, the company was unable to produce the example with Mr. Ronen and instead used a glamor shot of British actress Helen Mirren when she was aged 16–24, showing that Corsight AI software was able to match that decades-old photo to contemporary images of the 76-year-old entertainer.

The confidence score on the Helen Mirren demo match was only in the forties. While Corsight describes this as "giving a strong match", such a relatively low percentage would certainly increase the number of false matches, especially in alerts.

Moreover, the glamor and paparazzi shots are far clearer than surveillance-quality photos, as stated by Corsight AI to IPVM in a written response:

Faces from captured video frames usually are of significantly lower quality and resolution, and include artifacts, difficult angles and occlusions.

Data "Deleted Within 0.4 of a Second"

Corsight AI's CEO also claimed that his company's software deletes personal data within tenths of a second in the TravelPulse interview:

So, as soon as my image or your image is seen in a surveillance database—certainly within the Corsight software—yes, you’re looked at, you’re seen, but your data is deleted within 0.4 of a second. I can guarantee that to you. [emphasis added]

IPVM pushed back against this claim, pointing out that in the case of a false match, data would not be deleted at all, a fact acknowledged by Corsight AI:

You're absolutely right. If a false match occurs, then, of course, that image will be retained against that match.

Corsight AI said that the onus would fall on the human operator, or the "human in the loop," to remove falsely identified matches, citing that process can be performed "extremely quickly, extremely simply." However, false matches and false accusations from facial recognition systems are common (e.g, 1, 2).

Corsight AI Disavows COVID-19 Detection Claim

The screenshot below shows Corsight AI's LinkedIn post sharing the TravelPulse article:

IPVM Image

Zooming into the photo reveals the text "COVID-19: detected" and "COVID-19: negative" immediately below the temperature readings of the man and woman, respectively.

IPVM Image

Although Corsight AI has not removed the post featuring this misleading image from its LinkedIn page, the company disavowed the graphic, claiming that TravelPulse published it as the lead image for the Corsight AI interview without consulting the Israeli company:

[I]t is important to flag that Corsight AI did not supply and does not own the rights of this image – it was selected as a generic image by the journalist. Let me reiterate that Corsight AI’s technology cannot and does not diagnose/detect COVID-19. [emphasis added]

This was restated during Corsight AI's call with IPVM:

The person that published used some spurious image from somewhere, don't know where, not related to our system. We made representation to the journalists, we said that image is not from our system. We only put images out from our system. That is not from our system, please remove it. [emphasis added]

While TravelPulse has updated its lead image to a generic photo of US airport security screening, Corsight AI's PR team attempted to justify why the company's LinkedIn post continues to display the misleading COVID-19 detection photo:

But the image on the social media, I don't know how much you guys are aware of previews on social media posts, but that won't change simply because the image of the article is changed.

UPDATE: CEO Interview w/ TravelPulse LinkedIn Post Removed

Corsight AI removed its LinkedIn post sharing the TravelPulse interview with Rob Watts following the publication of this post, with the link now redirecting to an error page:

IPVM Image

2 reports cite this report:

ISC West 2022 Show Report on Mar 25, 2022
Not since 2019 has ISC West felt "normal". 2020 was canceled, 2021 was held...
Facial Recognition's Controversial Rise in Brazil on Nov 08, 2021
Brazilian media, academics, and NGOs are voicing alarm at facial...

Comments (11)

Only IPVM Subscribers may comment. Login or Join.

I can say I've been in talks with Corsight and started testing their software. They have been very down to earth and realistic with their abilities to us. They have not oversold their system or claimed it's perfect. We're still testing and can't speak to how it compares against other software analytics, but so far, it's performing exceptionally well. For now, I'm willing to give them the benefit of the doubt on the LinkedIn marketing.

Agree: 1
Disagree
Informative: 4
Unhelpful
Funny

UPDATE: CEO Interview w/ TravelPulse LinkedIn Post Removed

Corsight AI removed its LinkedIn post sharing the TravelPulse interview with Rob Watts following the publication of this post, with the link now redirecting to an error page:

IPVM Image

Agree
Disagree
Informative: 3
Unhelpful
Funny

When it comes to Video Analytics, as an integrator, we have to manage the unrealistic expectations of the end-user along with the unrealistic performance promised by the manufacturer.

Agree: 2
Disagree
Informative: 1
Unhelpful
Funny

There's no need to be critical of our organization or our business.

even if he believes this to be true, saying those words out loud is a mistake, imo.

instead of building confidence in his solution with skeptics, that kind of statement tends to have the opposite effect.

Agree: 1
Disagree
Informative
Unhelpful
Funny

From seeing their marketing and their conversation with our team, my concern is that a lot of these things are not well thought out, e.g. the defense of average people being added to their system, as they said in the interview:

As it's used for surveillance purposes, facial recognition software scans individuals’ biometrics as they pass through ports or airports, running them through a database to flag anyone who appears on a particular watchlist—missing persons, people traffickers, terrorists, etc.

But, for the average person who isn’t known to the system, “we’re not in the database.” Watts explained. “So, as soon as my image or your image is seen in a surveillance database—certainly within the Corsight software—yes, you’re looked at, you’re seen, but your data is deleted within 0.4 of a second. I can guarantee that to you.”

But as they subsequently acknowledged to us:

If a false match occurs, then, of course, that image will be retained against that match.

This is what many worry about. That they are just walking down the block and they are falsely picked up. Or I suppose even worse, someone adds you to the watchlist (and you are not a terrorist or people trafficker, etc.) and these systems are running a non-stop search for you.

To be clear, I don't think what Corsight is doing with its facial recognition is worse than other facial recognition suppliers but using a defense of "your data is deleted within 0.4 of a second. I can guarantee that to you" is at best naive and does not address the privacy risks here.

Agree: 2
Disagree
Informative
Unhelpful
Funny

Corsight has responded, copied below:

Thank you for providing a link to the article.

We would like to express our disappointment on the tone of the piece, which in our view does not provide a true reflection of Corsight. With reference to the image within the Tech Pulse article, we have on numerous occasions stressed that Corsight did not supply or does not own the rights to the image, yet this is still documented within the piece.

See attached a response from Tony Porter, Chief Privacy Officer at Corsight AI, which addresses some of the concerns raised by Mr Honovitch.

Listed below are claims within the article that Corsight feel are inaccurate. Corsight would appreciate it if IPVM could rectify these claims as soon as possible.

Let me know if you have any questions.

Regards,

Holly

IPVM: “Our concerns from the demonstration include:

Images demonstrated on watchlists were not surveillance photos but rather celebrity-style headshots.”

Corsight: This line does not make sense. Watchlist images should always, where possible, be as high-quality as they can be (for example, passport or "mugshot-style"). Corsight AI’s demonstration databases are representative of this and what are used in the real world by customers.

IPVM: “While Corsight AI reported an expected decrease in accuracy due to masks, hats, angles of incidence, lighting; Corsight could not disclose general or specific values. However, the company said that it shares this information with partners.”

Corsight AI shared the data detailing their performance on lower quality of images compared to higher.

IPVM: “Default match confidence during demonstrations was a relatively low 55 which risks increased false matches in real-world conditions.”

Corsight: 55 is not low for Corsight’s system. In fact, the recommended threshold for Fortify for typical use cases is 45, a threshold that results in a very low rate of false positives, even when using a DB of 10,000s suspects.

IPVM: “Corsight AI's CEO is quoted, at the beginning of the TravelPulse story, saying that his company's software can identify people through ski masks:

"We can identify someone with 90-degree head turn. We can identify someone with a camera at 70-degree rake, we can identify someone with a face mask—not just with a face mask, but with a ski mask...with hat, glasses, all of that[.] [emphasis added]”

Corsight AI recognises faces with 45° angle elevation instead of 70° (outlined above). The article contained the wrong information, which was flagged to IPVM in a previous email.

IPVM: “The confidence score on the Helen Mirren demo match was only in the forties. While Corsight describes this as "giving a strong match", such a relatively low percentage would certainly increase the number of false matches, especially in alerts.”

Corsight: This was demonstrated on a forensic, non-live scenario where Corsight took a near 50 year old image and showed it could identify the person within the database, where human operators then evaluate all the top candidates - not just the top 1 where they do in "live" - therefore the base threshold was actually set to 0 for this purpose to return the top 50 - alerts were not mentioned.

It is worth noting that thresholds in the range of 40+ in fact reflect high certainty in case of the specific example. This threshold exceeds the thresholds tested by NIST FRVT, which resulted in a FPR of 1/1,000,000.

I've added some of these responses to the body of the report, also noting our response to them, e.g. on Corsight's response that watchlist images should be high quality, IPVM's response is that often this is not possible and surveillance photos are used.

As for Corsight's complaint about the fever detection image that "we have on numerous occasions stressed that Corsight did not supply or does not own the rights to the image, yet this is still documented within the piece." We noted that in the report but we also accurately noted that they used that image in marketing for weeks even after I warned them publicly even tagging Tony Porter in the LinkedIn post, who ignored it.

Agree
Disagree
Informative
Unhelpful
Funny

From Corsight's Merry Christmas LinkedIn post, notice, Shay on the left, hat, sunglasses, ski mask:

IPVM Image

I don't see how any technology could reliably identify a person in such conditions at any reasonable production scale. Does anyone care to disagree or debate this?

Agree
Disagree
Informative
Unhelpful
Funny

Test it and find out.

Agree
Disagree
Informative
Unhelpful
Funny

More / new:

IPVM Image

Agree
Disagree
Informative
Unhelpful
Funny: 1

More / new - from a Corsight ISC West 2022 promo email:

when everyone else fails we will recognize your suspect

IPVM Image

Agree
Disagree
Informative
Unhelpful
Funny

Arguably Corsight's most aggressive examples yet:

IPVM Image

Agree
Disagree
Informative
Unhelpful
Funny: 1
Loading Related Reports