Evolv Supports Missing Guns To Reduce False Alarms

Published Mar 05, 2024 16:00 PM
PUBLIC - This article does not require an IPVM subscription. Feel free to share.

While Evolv marketed "weapons-free zones" with "touchless," "frictionless" screening, claiming unprecedented convenience and security that, privately, Evolv knew it could not deliver.

Watch the video below, excerpted from Evolv's own presentation, showing how Evolv deliberately misses guns to reduce its significant false alert problems:

Deleted by Evolv

Evolv deleted its video immediately after IPVM inquired, refusing to comment despite us providing them five days to respond.

Chart Showing Weapons Missed By Evolv

The chart below, based on Evolv's manager's statement, demonstrates what types of guns and weapons Evolv knowingly can miss at each sensitivity setting Evolv supports:

IPVM Image

Evolv has even fought to hide public knowledge of Evolv's supported sensitivity settings, alleging disclosing it is "irresponsible and makes the public less safe" rather than Evolv deceiving the public into believing they can deliver far more than they can.

Evolv has never publicly admitted to missing guns, despite reported incidents (1, 2, 3). This newly uncovered evidence shows that Evolv not only knew it would miss guns but actually built settings that miss many of the most widely used models, such as Glock G19s, to reduce false alarms.

But just last month, Evolv's CEO reiterated Evolv's stance that "unless you have a weapon, you can walk right in," and people "know when they get in, there's no weapon there."

The company knew that accomplishing "frictionless" screening without the significant false alarms on everyday objects of their much-maligned competitors meant Evolv would have to - on purpose - support missing guns, sacrificing security for convenience.

Evolv Presentation Explains Missing Guns to Minimize Alarm Rates

During a January 15, 2024, presentation for its reseller Integrated Protection Services, Evolv's Midwest sales manager Kevin Rabinowitz, who reports selling 400+ Evolv systems (contract value of ~$50 million), explained what guns Evolv expects to miss at each of its seven sensitivity settings, how they impact false alarms, and even what settings were used at the Super Bowl.

At settings A and B, Evolv's sales manager said, "you're looking at large guns...[and] you're looking at minimal alarm rates." For example, "where we've used a B setting is at the perimeter of the Super Bowl," trying to detect "a press trigger bomb or an AR-15," but "maybe they had a 38," presumably referring to a .38 caliber pistol.

IPVM ImageSetting C detects "the Glock 19 only, and like, bigger firearms." A Glock G19, pictured below, is a compact size 9mm handgun that is among the top-selling guns in the US. A standard-size magazine contains 15 rounds; the Federal Assault Weapons Ban defines a "high-capacity magazine" as >10 rounds.

For sensitivity "D, you get into the compact, subcompact [pistols]," such as the subcompact the Sig Sauer P365, while at sensitivity E, "you're getting into your micros [micro-compact pistols], your .38 specials, your 380 Ruger LCPs."

Sensitivity F is for "novel items like the one-shot Derringers" and "smaller knives." He said F is rarely used, and "the higher sensitivity, the more false alarms." Most customers, he said, use sensitivities D or E.

While the sales manager's comments provide important new insight into what Evolv expects to detect at various sensitivity settings, they should not be relied upon as definitive statements of what Evolv will detect at specified settings.

For instance, at sensitivity D he says, "you get into the [detection of] compact, subcompact [pistols]...E you're getting into your micros, your 0.38 specials, your 380 Ruger LCPs." [emphasis added]

Evolv's salesperson did not clarify whether "getting into" detection means sometimes detecting these guns or consistently detecting them. (In fact, Evolv detected only 92% of micro-compact pistols during its NCS4 testing at setting E, a more sensitive setting than D.)

Evolv's Sensitivity Slider Examined

While Evolv marketed its Cortex AI "threat AI classifier" as a core innovation, Evolv fought hard to keep from the public that they support a simplistic sensitivity slider, much like the competitors they call "stupid," "dumb analog devices" that "go off every time." This simplistic slider, with levels A, B, C, D, E, F, G, rather than AI, controls what weapons Evolv misses to meet their marketing claims of "touchless" and "frictionless" screening.

For more background on Evolv sensitivity settings, see Evolv's Secret Metallic Sensitivity Setting And The Risk Of Missing Weapons and Evolv Becoming More Of A Metal Detector With New "G" Sensitivity Setting.

Moreover, while Evolv denigrates its competitors, Evolv's sensitivity settings are similar to those of metal detectors. For example, CEIA and Garrett's manuals (widely used metal detector manufacturers) describe similar settings:

IPVM Image

Evolv's Approach Vs. Metal Detectors Approach

Evolv's approach, which prioritizes reducing false alarms at the risk of missing weapons, is the direct opposite of that of conventional metal detectors. Metal detectors are usually set to highly sensitive settings as their goal is to find weapons rather than to optimize visitor experience.

Evolv and 'Security Theater'

Despite bashing "stupid metal detectors" for being "theater security" (more commonly known as "security theater"), screening people with devices that do not detect dangerous firearms while assuring them otherwise is security theater.

Endangering The Public

Organizations can choose not to use security screening or to use security screening that knowingly missing guns.

What is deceptive and endangers public safety is Evolv repeatedly assuring the public that it will detect guns and then privately telling its resellers that they can not detect guns for the sake of convenience.

Evolv, under investigation by the FTC and now SEC, has deceived the public and investors into believing it does far more than it knows it can, with missing guns being critical.

Comments (24)
UM
Undisclosed Manufacturer #1
Mar 05, 2024

All I can say is what is wrong with this company...

(2)
(1)
(1)
Avatar
Brian Karas
Mar 05, 2024
Pelican Zero

This is the challenge of any kind of detector, particularly with AI.

You never get perfect performance.

There is a decision you have to make between "No missed events" and "No false alarms". Different companies approach this concept in different ways, but in the end there is always a tradeoff.

Evolv has bitten off a large challenge in this.

(3)
JH
John Honovich
Mar 05, 2024
IPVM

The K-12 Shooting Database founder David Riedman shared this article on LinkedIn, noting that "+76% of shootings at schools are committed with handguns."

(2)
JW
Jerry Wilkins
Mar 05, 2024

Interesting the chart says, "Can Detect" not "Will Detect." I can win this week's Powerball; I just need to buy a ticket, then have the correct numbers to win it, then not lose or destroy my ticket, then make to the PB headquarters to pick-up the check.

(1)
(2)
Avatar
Brian Karas
Mar 05, 2024
Pelican Zero

In fairness to Evolv that is the correct wording. Analytics is ultimately "can", not "will".

(1)
JH
John Honovich
Mar 05, 2024
IPVM

We chose the words "can" and "may" as Evolv did not respond to our request for comment, and, to Brian's point, we felt that was a fair description given the impreciseness of the technology.

(2)
DL
Daniel Lewkovitz
Mar 05, 2024

Traditional metal detectors have always had adjustable sensitivity to balance between false positives and false negatives or throughput versus thoroughness. If I remember correctly the Garrett walk through systems even had one-touch settings for it e.g. "Airport" which which might tolerate a belt buckle versus "Prison" which wouldn't tolerate a paperclip secreted in your tuches.

This seems to be exactly the same thing in every way. So I fail to see how there is anything unique introduced by the technology in that regard and it seems crazy applying marketing hype to a product where a failure could be potentially catastrophic. This is why Qantas will never ever brag about their safety record despite having never had a fatal jet aircraft accident.

You're only as good as your next fuckup.

(2)
(2)
(1)
JH
John Honovich
Mar 05, 2024
IPVM

This is why Qantas will never ever brag about their safety record despite having never had a fatal jet aircraft accident.

You're only as good as your next fuckup.

An interesting comparison. Security companies are generally fairly restrained in their marketing, understanding that things can go wrong and that they aim to persist for decades.

To that end, another comparison is the relationship between Motorola and Evolv. Since Motorola entered into security ~6 years ago, we have seen them be relatively cautious in marketing, but their partnership with Evolv is putting this to the test.

One thing I do not fully understand is why Peter George, Evolv's CEO, keeps on saying these things. We can asses their veracity and accuracy, but it's hard to see why it's helping him or Evolv.

DL
Daniel Lewkovitz
Mar 06, 2024

CEO's gonna CEO? 🤷🏼

(2)
JH
John Honovich
Mar 06, 2024
IPVM

We interview / interact / track dozens of CEOs on an ongoing basis. I can hardly recall a CEO as careless in his statements as Peter George. While a fair number of CEOs are deceptive, usually they are sophisticated in doing so, not Peter George.

U
Undisclosed #2
Mar 06, 2024

Security companies are generally fairly restrained in their marketing,...

not the new VC-funded security companies.

...understanding that things can go wrong and that they aim to persist for decades.

VC groups are generally interested in a successful exit - not persisting for decades before they move on to their next opportunity to make money.

(3)
(1)
DL
Daniel Lewkovitz
Mar 06, 2024

I agree with this.

We have had a number of people enquire whether our products or service can do XYZ like a competing product they saw online. In many cases what they're expecting is completely impossible with current technology.

False advertising is nothing new but there's a world of difference between a kitchen gadget and a life-safety or mission critical product, the innovation cycle of which does not operate at the speed of Silicon Valley for a reason. See also medical innovation.

(1)
(2)
JH
John Honovich
Mar 06, 2024
IPVM

Related, new poll:

IPVM Image

I tried to frame this neutral to positive for this approach (since we are overall critical), but responses are still quite negative.

Avatar
Brian Karas
Mar 06, 2024
Pelican Zero

It's kind of a weird question. Without clear understanding of the tradeoffs and options why would anyone be inclined to vote anything other than No on that?

I'm not sure if you used the G19 as an example related to something that Evolv has said, but I wouldn't call that a good example of a compact gun, it's more average, IMO. The G19 is a 9mm 10+1 format with a 4" barrel. It's slightly bigger than my USP 45C, where the C is for "compact", but this is a fairly large pistol overall:IPVM ImageA "pocket pistol", like the .380LCP below would probably be a better example for this scenario:IPVM Image

(1)
(1)
JH
John Honovich
Mar 06, 2024
IPVM

Without clear understanding of the tradeoffs and options why would anyone be inclined to vote anything other than No on that?

It's a poll so it's hard to list all the tradeoffs and options.

That said, happy to run a new poll with different wording. Please suggest.

Avatar
Brian Karas
Mar 06, 2024
Pelican Zero

I think that people taking the poll need more context. When you say "truthfully disclosed", to who? Is there a sign on the unit that says "this system may not detect Glock 19's"?

For what you are trying to get at, maybe something along the lines of:"Who should weapons detection companies disclose accuracy limitations to?"

A) General public/any person relying on the system for their own security

B) Buyers/users of the system (eg: schools and stadiums), but not the general public

C) Authorized law enforcement personnel or government agencies only

D) Nobody, this information should remain secret.

(1)
JH
John Honovich
Mar 06, 2024
IPVM

"Who should weapons detection companies disclose accuracy limitations to?"

The Evolv debate is not simply about disclosure. It's about whether a company can tell the public the opposite of what they say to their customers.

If Evolv took the position, "We are a highly sensitive security company and we tell the public nothing one way or the other about how we work or what we could do", that would be completely different and far less objectionable to us than publicly saying, as Evolv's CEO literally said weeks ago "people walk right in without breaking stride and know when they get in, there's no weapon there" at the same time his sale team is telling people that they can miss various guns for the sake of convenience. That creates a dangerous false sense of security.

(2)
UM
Undisclosed Manufacturer #3
Mar 06, 2024

I can attest that I've gone thru their detectors carrying a Glock 19 and nothing happened, at different places even.

(4)
JH
John Honovich
Mar 06, 2024
IPVM

One thing to keep in mind is that, as the Evolv sales manager explains, it is a factor of sensitivity level. Maybe one day you go through an Evolv Express unit set a F and it alerts on the Glock 19 but the next day you go through an Evolv Express unit set at C or B, etc., and it might not.

Evolv markets to the public that it's Evolv "AI" but that risks a real false sense of security depending on what sensitivity setting you use.

And the final irony is that if Evolv units are set to be sensitive to smaller weapons (like with G), the facility and people are now roughly back where they started with high false alarm rates but at 10x the price.

(2)
(2)
U
Undisclosed #4
Mar 06, 2024

I've seen this recently with a monitored CCTV client who had lots of false alarms. they kept turning down the sensitivity of their camera motion/analytics detection until there were no more false positives.

I pointed out the risk of false negatives. They were too busy patting themselves on the back and will probably learn the hard way. I'm sure it will become our fault when they have a loss and we failed to respond to a signal we never received...

(1)
JH
John Honovich
Mar 06, 2024
IPVM

CCTV client who had lots of false alarms. they kept turning down the sensitivity of their camera motion/analytics detection until there were no more false positives.

An interesting comparison! Video analytics (that is detecting people and vehicles) have genuinely become much better in the last few years (generally) as our testing has shown.

Evolv's best-case optimistic scenario is that it delivered similar advances on the covert object side. But as this example shows, it's unfortunate for the public being driven by turning down sensitivity.

Related, we don't link to this in the original post but we have other research where we tested and were able to replicate Evolv-like performance by similar tuning of metal detectors. I've embedded the video for those who have not yet seen it:

JW
Jerry Wilkins
Mar 06, 2024

My final comment on this subject... If it does detect who is providing a tactically sound approach to what to do next. When I was in Haiti, they screen everyone up close and personal. If the detector went off, you were going to the ground until it got sorted out.

JH
John Honovich
Mar 06, 2024
IPVM

they screen everyone up close and person

But what buyers are most responding to Evolv is the holy grail pitch that you get the weapons but nobody else is inconvenienced. That's the Catch-22.

(1)
Avatar
Steve Beck
Mar 07, 2024

Columbus Zoo to install new security system at entrance | 10tv.com

The story mentions it will detect guns but not knives….anyway can’t wait to take my niece through some Evolv detectors to go visit some flamingos.

(2)