If you don't know about the weaknesses of a product, you can't take steps to minimize the impacts of the weakness being exploited. At the end of the day, once the product is deployed in the field someone will come across it's weaknesses - disclosing them ahead of time just gives the end user/integrator the chance to do something about it.
Evolv Objects To IPVM Reporting on Its Weapon Detection Weaknesses
JH
PUBLIC - This article does not require an IPVM subscription. Feel free to share.
Evolv has strenuously objected to IPVM reporting on weaknesses of the company's weapon detection systems. Evolv contends that its private disclosure to its end users is sufficient and that IPVM's reporting would endanger public safety, risking deaths.
In this report, we examine Evolv and IPVM's positions and review three months of IPVM efforts to discuss the weaknesses with Evolv, the publicly-traded company that aims to increase revenue from $4 million last year to $595 million in 4 years.
Evolv Statement
Evolv sent us 2 detailed statements (see the diffs here). The first discussed weaknesses and vulnerabilities but when we responded asking about "weaknesses you can't fix, why did you choose to sell this product?", they removed that language. Below is the first statement copied in full, with the portions in bold that Evolv subsequently deleted:
We take our mission of keeping people safe seriously. That requires continuous improvements to our system, rigorous testing, and evaluation by third parties, the security professionals that are considering our technology and employed by our customers, and our own advisors. They relentlessly test our system because they have a personal and professional stake in the outcome. If they find an important weakness in our product, and they perceive we are dragging our feet to address it or help them work around it, they will have no problem telling their colleagues at other facilities. They do not compete, and they share information freely because they have a common interest in avoiding incidents. They serve the public, not vendors. We could not possibly be more motivated to make our products as effective as we know how.
Just as keeping people safe requires our customers and qualified prospects to be fully informed about Evolv Express’s capabilities, it also requires keeping potential threat actors in the dark about security measures. Fear of the unknown can be an incredibly effective deterrent as is lack of knowledge of the defensive landscape.
Because the general public may include potential threat actors, it is important to avoid disclosing information publicly that could help threat actors mount a successful attack. Unlike the software world, where nearly every vulnerability can be patched once known, physical security systems (inclusive of people, process, and technology) are subject to the unyielding laws of physics and operational constraints that make it impossible to fix every vulnerability or correct every lapse in human judgment. [emphasis added for sections removed in second statement]
Below is the second statement Evolv provided us, with the portions added highlighted in bold:
We take our mission of keeping people safe seriously. That requires continuous improvements to our system, rigorous testing, and evaluation by qualified third parties, the security professionals that are considering our technology and employed by our customers, and our own advisors. They relentlessly test our system because they have a personal and professional stake in the outcome. They serve the public, not vendors. We could not possibly be more motivated to make our products as effective as we know how.
Just as keeping people safe requires our customers and qualified prospects to be fully informed about Evolv Express’s capabilities, it also requires keeping potential threat actors in the dark about security measures.The unknown can be an incredibly effective deterrent as is lack of knowledge of the defensive landscape.
The increased frequency and carnage of active shooter events demands a new solution. Metal detectors have been around for decades, yet are not deployed at many venues that are concerned about firearmsand other threats because of the high nuisance alarm rates, the need for all visitors to divest their personal items, and the security threat created by the lines of people waiting to go through. As a practical matter, it just doesn’t work for most venues that are not required to use them. It’s critical that any potential solution strike a significantly better balance between threat detection and visitor experience and operator demands. We designed Evolv Express to detect firearms and other threats without requiring the operator to sacrifice the entrance experience of fans, visitors, students or employees. [emphasis added for sections added to second statement]
Evolv declined to comment on what third parties evaluated their technology beyond adding the word 'qualified' in the second statement.
Why Evolv Believes This Is Unlikely To Be Disclosed By Others
On two separate calls with IPVM, Evolv emphasized its belief that if IPVM did not report on these weaknesses that others would not be likely to do so, emphasizing the use of NDAs and other methods to keep problems in confidence.
"Security Through Obscurity"
Evolv and IPVM disagreed on whether their approach is a measure of "security through obscurity".
Our position is that disclosure of weaknesses incentivizes companies to be transparent about what they can and cannot do, giving the public a clearer understanding of a product's capabilities. Furthermore, transparency incentivizes accountability to the public instead of a select group of company insiders, who may weigh the tradeoffs of disclosure against financial performance.
As Evolv mentioned in its first statement, though:
Unlike the software world, where nearly every vulnerability can be patched once known, physical security systems (inclusive of people, process, and technology) are subject to the unyielding laws of physics and operational constraints that make it impossible to fix every vulnerability
How People Can (And Will) Find Out About Evolv's Weaknesses
The sheer size and scale of Evolv's ambitions (they expect $595 million in revenue in four years) means the company anticipates thousands of systems deployed across the United States and the world. Accordingly, vast numbers of security professionals and end-users will develop inside knowledge of the system's weaknesses.
Furthermore, scrutiny of Evolv's technology will increase as the public becomes more aware of these systems and how they work. It is unlikely IPVM will be the only publication to research and inquire about the technology. As a publicly-traded company, Evolv should expect increased scrutiny from investors seeking to evaluate risks and opportunities. As a company that aims to secure public schools, sports stadiums, etc., it should anticipate that far more than IPVM will be interested in their product's performance and ability to secure the public (or not).
How Hiding These Weaknesses Clouds End-users Understanding of Risks
By not having a public disclosure of risks, end-users are left to evaluate Evolv through the information that only Evolv controls. This imbalance of information favors Evolv, a publicly-traded company with ambitious growth targets and sales reps heavily incentivized to close deals with end-users.
Three Months of Exchanges With Evolv
We have engaged with Evolv for more than 3 months, so despite Evolv repeatedly proactively criticizing us for 'gotcha' journalism, we have tried hard to understand their position and talk with them.
Evolv's CMO, Dana Loof, first responded on September 14th agreeing to speak:
Over one month passed, with Evolv missing a scheduled meeting and other delays before company representatives agreed to let IPVM see a demo on November 5th, but only 'off the record':
However, the demo was then canceled on November 9th, with Evolv instead suggesting we speak to end users:
This email was then followed by a "meet and greet" offer before anything was said "on the record." IPVM declined this offer and made it clear a "meet and greet" is not part of our research and testing process:
Finally, on December 2nd, Evolv agreed to a call where Michael Ellenbogen stated it is not Evolv's philosophy to disclose Evolv performance information and that the company never intended to allow us to test the system.
Evolv Sales Rep's School Board Presentation Opens Door
IPVM found a one-hour presentation by Evolv at a public school board meeting (on the public Internet) which detailed various limitations and weaknesses associated with the Evolv Express system, running contrary to Evolv's stated policy of not publicly disclosing such information.
IPVM flagged this information for Evolv and asked them to address these weaknesses on the record. In response, IPVM finally obtained an on-the-record statement regarding Evolv's weaknesses and associated philosophy.
Subsequently, IPVM found various other weaknesses buried in Evolv's own public documentation.
IPVM Position on Evolv
As we explained to Evolv, we are not fundamentally negative on Evolv, the company, or its products. Since we have not been allowed to test or even to see it in real-life use, we cannot speak definitively about our assessment of its performance but we certainly believe the end-users we have spoken to where the systems are helping to detect weapons.
The product, at least, has some material weaknesses that the company privately acknowledges but strives hard to block the public from knowing. How much worse (or not worse) these problems are, by definition unknown to us, given Evolv's position.
We believe scrutiny makes the public safer and ensures that companies do not take advantage of hiding material information from the public. We do plan to report further on Evolv's weaknesses and are interested in the community's feedback on this issue.
Vote / Poll
UPDATE:
Evolv has since formalized this position into what they call a "Transparency of Sensitive Information" policy.
Comments (60)
UE
Undisclosed End User #1
Dec 15, 2021JH
John Honovich
Dec 15, 2021disclosing them ahead of time just gives the end user/integrator the chance to do something about it.
Evolv's point is that they are transparent with the user and any partners involved. With that noted, does that impact your thoughts?
UM
Undisclosed Manufacturer #4
Dec 15, 2021I'd be interested to see how transparent they really are. I'm sure it happens occasionally, but I doubt the general sales pitch goes something like this ...
We're great, buy our stuff. Well, unless you are concerned about X, Y or Z. Then you shouldn't buy it.
If they do present their product that way - then kudos to them. However, given their aggressive targets I think they're trying to get every sale they can, not talk their potential customers out of a sale.
MA
Matt Alvey
Jan 05, 2022So, just a 2-cents here:
I came to IPVM to research this product specifically. I had seen several demos and spoken to various sales reps (not named here). After reading all of IPVM's reporting, I have actually been reassured that I should go forward in recommending Evolv to my larger clientele. Why? Because zero of it was a surprise at all.
In the very first demo I saw, the sales rep talked about missing some knives. Looking back, I could sense he was a little nervous about saying it; perhaps bracing for objections. But it made sense to me, and I had no objections. The rep also spoke about laptops possibly setting them off. At the time, all I could think about was those stupid metal detectors; taking my belt off, emptying my pockets, taking off shoes, and still somehow I set them off every time!
So I can attest that they were above-and-beyond forthright with myself and my team. This is MUCH better than what I experienced back when Megapixel cameras first started coming out. Remember how they'd lead people to believe you could read a tattoo on the back of a fly from the end of a football field? I could name names, but I won't.
Every sales rep's demo I have seen, including with end-users, has included clearly stating these reported 'weaknesses,' and not in a rushed, under-their-breath sort of way.
JH
John Honovich
Jan 05, 2022Remember how they'd lead people to believe you could read a tattoo on the back of a fly from the end of a football field? I could name names, but I won't.
Recall 2012: Arecont Lies, Now Threatens Lawsuit. For people new to the industry and who somehow think IPVM is simply against their company (Evolv, Verkada, whoever), this should give some context.
James Miller III
Dec 15, 2021To me this is reminds me of how bug disclosures acts within the cybersecurity community. Personally I'm glad these are being caught now versus having them out there in the wild being involved in a highly publicized security incident.
UI
Undisclosed Integrator #2
Dec 15, 2021Take a field trip to Disneyworld! I watched both sides of the system in action standing next to the security officer. Also, the new fireworks show is spectacular along with the Castle illumination.
UM
Undisclosed Manufacturer #16
Feb 18, 2022Got a text message from a good friend of mine yesterday. He's in the theme park industry, and he frequently visits Disneyworld to see what they're up to (and because he just straight up loves all things Disney). He was impressed with the Evolv scanners, and is thinking of putting them into his own parks.
I'll let you guys know if it ever gets beyond the idea stage, and how the demo goes.
UM
Undisclosed Manufacturer #3
Dec 15, 2021Just to clarify as it was not clear, IPVM has no way of procuring the system to test it themselves?
JH
John Honovich
Dec 15, 2021Our general approach is to try to communicate with the manufacturer first. This is not something that is sold online and and even if they would sell us one, they are sold as multi year contracts for tens of thousands of dollars.
There are other ways for us to go about this but we are proceeding step by step.
UM
Undisclosed Manufacturer #3
Dec 15, 2021Oh so you couldn't afford it. Totally kidding. Seriously, what does the multi-year contract cover? You may have already covered it in another article so I apologize if I missed it.
JH
John Honovich
Dec 15, 2021Two options are roughly pay $50k up front and then $1,500 per month for at least 4 years or pay $2,500 per month for 4 years. You need an active subscription for the hardware to work. It's a $100,000+ commitment for a system.
UE
Undisclosed End User #11
Dec 16, 2021So they are running a live service subscription model while taking the stance that they won't/can't address issues/bugs?
We were interested in the product and have a demo on the 6th with them. But this is eye opening and is giving me second thoughts.
JH
John Honovich
Dec 16, 2021that they won't/can't address issues/bugs?
They are blaming it on physics:
subject to the unyielding laws of physics and operational constraints that make it impossible to fix every vulnerability
On that point, I disagree. This is not like asking them to build a perpetual motion machine. They have limitations in how they can detect weapons, but it's their limitations, not Einstein et al.
However, on the positive side, I think they would argue that, whatever limitations they have, they are still far better than a conventional metal detector, which is fairly dumb. I think the more important questions include:
How much better are they really than a metal detector? They make some strong claims:
And how does one configure the Evolv system such that it balances the labor savings they tout while minimizing the risk of Evolv missing weapons?
MM
Michael Miller
Dec 16, 2021Not once when I had the Evolv unit in-house with the Evolv team did they say they get ALL weapons. Everyone was very clear about what weapons the unit can detect and we tested multiple types of weapons. Larger Knives, guns, and pipe bombs all set off the unit. Even my 4-inch leatherman was detected though that can change depending on the level of sensitivity you have in the unit.
If you need to make sure no weapons of any kind get in your building then you need to search every person and Evolv is not for you. If you want to detect weapons of mass destruction and do it quickly then Evolv is something to look at.
JH
John Honovich
Dec 16, 2021Not once when I had the Evolv unit in-house with the Evolv team did they say they get ALL weapons. Everyone was very clear about what weapons the unit can detect
And that's good. The question for Evolv is why are they trying to stop the public from knowing this? And why does their website emphasize weapons detection without any disclaimer or clarification about what it can do? Is this enhancing public safety or giving the public a false sense of safety?
UM
Undisclosed Manufacturer #6
Dec 16, 2021And that's good. The question for Evolv is why are they trying to stop the public from knowing this? And why does their website emphasize weapons detection without any disclaimer or clarification about what it can do?
Another way to look at it might be how much of a right do the taxpayers have to know about the limitations, since they are footing the bill in public schools?
But on the flip side, do the taxpayers really need to know the details of what it can't detect, or do they really just need to know that it is for example only 80% effective across all weapons types. That even can be misleading. Because even if it can detect 80% across all weapon types, what matters are the types that it can't detect near 100% of the time.
Years ago they tried to ban the The Anarchists Cookbook. Even before the Internet, there were kids passing around Xeroxed copies they got from friends and friends of friends. Now with the Internet, any vulnerability known is almost sure to found with minimal search effort.
Brian Karas
Dec 16, 2021without any disclaimer or clarification about what it can do?
I didn't peruse every corner of the Evolv website, but from a basic review it does not appear to make excessive claims, and does not present the product as a catch all.
Eg, this section of text has a fair bit of hedging. "Highest degree of weapons detection accuracy" (does not make a specific claim like 99.9%), "reduce security risk" (not eliminate security risk), "reduce false alarms" (not eliminate false alarms".
Generally speaking, companies do not focus on product limitations or weaknesses in public marketing, I would not expect the website to say "doesn't work for scenario X" or things like that.
JH
John Honovich
Dec 16, 2021companies do not focus on product limitations or weaknesses in public marketing
Yes, but they can also more narrowly make claims. Instead of saying "weapons" generally, they could say "weapons causing mass casualties" or name the specific category or type of weapons, i.e. "guns" or "assault weapons", etc.
So even without writing down a long list of things they don't do by more fairly and accurately describing what they can do, a reasonable reader can deduce what they do, thoughts?
Brian Karas
Dec 16, 2021Yes, but they can also more narrowly make claims.
Sure, they could do it all kinds of ways. I don't have any vested interest in evolv, but when I looked through their website I didn't come away with the impression that they were trying to be deceptive in their presentation of the product.
I also think that given the application for the product, the price category, and the target buyer, they are marketing to an audience that is assumed to come in with some understanding of the common limitations. As an example, I don't think their target audience would assume it's going to reliably detect something small like a single razor blade, a CIA letter opener, small caliber composite frame pistol broken down into multiple components and smuggled in via multiple people, etc.
JH
John Honovich
Dec 16, 2021they could do it all kinds of ways
But I am not suggesting their website should be in pink or light blue or burgundy or some other aesthetic issue that is inherently subjective.
They are a publicly traded company, they are providing security to major public venues, and marketing claims are generally regulated by the FTC, etc.
they are marketing to an audience that is assumed to come in with some understanding of the common limitations
They sell to schools, the same types of schools that bought Hikua fever cameras by the pallet...
And not only is the concern about the school decision makers but the students and the parents who expect their children to be protected against weapons by this system.
Brian Karas
Dec 16, 2021the same types of schools that bought Hikua fever cameras by the pallet.
That's not a fair example, IMO.
Fever cams were an absolute scam. Anybody who had both a basic understanding of thermal cameras and a moderate amount of integrity knew that those systems could never provide the intended results.
From what I can tell, the evolv system is based on a solid premise, and can/will perform to a fairly repeatable level of performance, detecting weapons that fall within a range of sizes and physical properties.
Saying that they sell to schools in some ways supports their marketing approach. I do not think the average school user has a threat level that requires them to detect a large degree of edge cases, they are more likely going to want to detect weapons that are very "average", all things considered.
and marketing claims are generally regulated by the FTC
Yes, and their "claims" all seem to be representative of the performance reported by people who have used or tested it in person (I have not). They detect a lot of different kinds of weapons and do so with a minimal amount of false alarms or other delays. Their claims do not, to me, imply that this system is going to give you the detection levels of more advanced "airport" style systems.
Compare evolv's website to something like Leidos, which targets airport passenger screening, arguably an application that needs a higher level of performance. I'm sure the Leidos system does not detect EVERYTHING that could be classified as a weapon, but I don't see any specific mentions on their website of the items that their system is likely to miss. Overall, i would say the two companies websites and claims are more or less similar.
I'm just not seeing anything devious or malicious on the evolv site at this time.
JH
John Honovich
Dec 16, 2021That's not a fair example, IMO.
Fever cams were an absolute scam.
My comparison was about the sophistication of the school end users, not the products. To elaborate, my point is that many school end users have no idea when they are getting scammed, getting something great, etc. This was in direct response to you saying, " I don't think their target audience would assume it's going to reliably detect". I do think many in their target audience (e.g., schools) will think it detects all sorts of weapons, just like a year ago they thought a Hikua camera detects a fever in a kid wearing a ski mask with the doors open to the outside.
Brian Karas
Dec 16, 2021I do think many in their target audience (e.g., schools) will think it detects all sorts of weapons
Maybe, but I would say it's far easier for a school system to test that than it is for them to test fever camera performance, all things considered.
Fever camera sellers posted materials that were completely unrealistic (eg: scanning multiple people at once, immediately after coming in from outdoors, etc.). In the evolv materials I have reviewed so far I didn't see them showing examples of defections that would be impractical to repeat in real life scenarios. Do you have examples of evolv showing levels of performance that set false expectations? (honest question, you've clearly spent more time reviewing their materials than I have).
JH
John Honovich
Dec 16, 2021I would say it's far easier for a school system to test that than it is for them to test fever camera performance, all things considered.
I agree with this. But it's even easier for Evolv to just publicly say what they privately, see Miller's comment in this thread about what specifically they do claim to detect, which is distinctly narrower than the general weapons detection claim on the website.
As for specific false issues or weaknesses, we are waiting. Related, at the Imperial Capital event yesterday (which you were, lol, also at), Evolv's CEO agreed to let us test the system so we are aiming to refrain from getting into further details if we can first go deeper into the tech before reporting more.
JH
John Honovich
Dec 24, 2021Evolv's CEO agreed to let us test the system
Don followed up with Evolv about this but we have received no response nor communication from Evolv's management in the past week.
SC
Scott Clingan
Dec 16, 2021I can echo Michal's comments. That has also been my experience with Evolv. I have been screened by various Evolv systems at venues and as a potential integrator working directly with end users interested in the system. There are limitations with the system and I think they have been upfront about what those are. The product is another layer of security, not a complete solution.
As to why Evolv is less forthcoming in public forums I think that is pretty clear, no one markets their product with what it can't do. Specific to public safety products there is certainly some relevance to obscuring what bad actors learn to prevent active planning to defeat such systems. There are some inherent limitations that I am aware of that I do not think need to be publicly proclaimed. But likely in the end this information will certainly get out in some fashion. There does seem to be a parallel here to the way bug disclosures are handled in the cybersecurity community as someone commented above.
Evolv should be more forthcoming and more transparent with IPVM. I think then IPVM would be less concerned about some sort of bad acting (think fever cams and Verkada) with the product. I think IPVM would make appropriate journalistic decisions as to what needs to be published and what does not.
UE
Undisclosed End User #15
Dec 30, 2021JH
John Honovich
Dec 30, 2021It is not HaaS because they give you an option not to buy it and just effectively lease / rent the hardware as part of the monthly fee.
U
Undisclosed #5
Dec 15, 2021from the March 2021 report on Evolve going public:
Evolv also did not speak about how it expects revenue to increase 5x this year, to reach ~$20 million by the end of 2021.
it's now the end of 2021... did they make it (or even come close)?
JH
John Honovich
Dec 15, 2021Yes, they will make the $20 million this year, from their Q3 announcement:
Total revenue for the first nine months of 2021 was $16.8 million, an increase of 513% compared to $2.8 million for the first nine months of 2020
That's partially boosted by more people choosing the 'purchase subscription' option as we discussed in our Q2 analysis:
Evolv offers 2 purchasing options to customers: "subscription" and "purchase subscription." Under the subscription model, customers pay $2,000-3,000 USD a month for Evolv's hardware and software over 4 years. Under "purchase subscription," customers pay ~$50,000 upfront for the hardware but still must sign a 4-year contract. They then pay a lower monthly subscription fee (~$1,500) to use Evolv's software.
They are definitively growing revenue strongly though stock price has been trending down:
UM
Undisclosed Manufacturer #6
Dec 15, 2021JH
John Honovich
Dec 15, 2021I think financially they may end up fine, at the very least too soon to definitely say.
U
Undisclosed #7
Dec 15, 2021At the most recent GSX Expo (Orlando) I had a chance to get a personal demo of how their product performs and it sounds pretty promising. As expected they didn't outright disclose all weaknesses at just an initial demo - although they did mention that it's a work in progress - but it's definitely something they should do once serious project planning/negotiations are in progress, as well as constantly updating existing users of such.
On the same note they do have a point in not putting all their weaknesses on the front page and neither will any manufacturer, both from a marketing and a security aspect. No technology is perfect but as long as they don't make any false marketing campaigns and are commited to actively improve / eliminate weak points IMHO they're acting responsibly enough.
Another factor to consider here is that the majority of the client base they're appealing to would be coming from virtually zero or very weak security hence this with even only let's say an 80% success rate would be a major security improvement, as it's self-understood that no facility with Metal Detector and X-Ray style security would even consider switching to this.
Since lots of security professionals pay for their IPVM membership specifically for this reason to get unbiased and reliable info when researching such systems for their clients you do have a responsibility to provide true and unbiased reports on this technology just like with any other other. I do believe though that their should be some level of constraint when reporting on weaknesses i.e. providing overall idea of the weakpoint vs. providing highly specific detailed instructions how this technology was defeated.
U
Undisclosed #5
Dec 15, 2021Another factor to consider here is that the majority of the client base they're appealing to would be coming from virtually zero or very weak security hence this with even only let's say an 80% success rate would be a major security improvement, as it's self-understood that no facility with Metal Detector and X-Ray style security would even consider switching to this.
wait. what?
I agree that adding something to nothing would increase a success rate... but they are marketing heavily to facilities that use metal detectors and X-ray style security. one of their main marketing value propositions is 'increased efficiency' vs traditional screening.
i.e. they are certainly pitching this to facilities that already screen.
U
Undisclosed #7
Dec 15, 2021That wasn't my understanding but if that's the case I stand corrected.
This technology is NOT ready to replace traditional thorough screening methods like X-ray and metal detectors.
JH
John Honovich
Dec 15, 2021not putting all their weaknesses on the front page and neither will any manufacturer, both from a marketing and a security aspect
For sure, most manufacturers prefer their weaknesses not to be publicly reported on. Evolv went further than any other manufacturer we have ever interacted with, especially with the whole IPVM puts American lives at risk tactic, which is one of the reasons we chose to report on this issue first before going into any technical details.
as they don't make any false marketing campaigns
We do have some concerns with their marketing being deceptive but we are talking with them about this.
providing highly specific detailed instructions how this technology was defeated.
As an example, without getting into specifics right now, there are certain types of objects, that are obviously not weapons, that Evolv detects as weapons. Do we simply never report on that?
U
Undisclosed #7
Dec 15, 2021As an example, without getting into specifics right now, there are certain types of objects, that are obviously not weapons, that Evolv detects as weapons. Do we simply never report on that?
False positives report on to your heart's delight. It's the false negatives I was encouraging the reporting restraint on...
UI
Undisclosed Integrator #8
Dec 15, 2021It could have something to do with it exposing that Evolv sells direct.
JH
John Honovich
Dec 15, 2021They have been open about that and their aim to shift it, see Evolv: "We're A Channel First Company"
UI
Undisclosed Integrator #9
Dec 16, 2021Hey we're about to sell a bunch of cars, some of them the brakes don't work, please don't report on that.
U
Undisclosed #5
Dec 16, 2021Braking when it's not needed, report on to your heart's delight.
It's the brakes not working when needed I encourage the reporting restraint on...
UI
Undisclosed Integrator #10
Dec 16, 2021I had a personal demo, have a customer with one, and several customer sites with full-body ionizing radiation and Millimeter-wave scanners in use.
Evolve rep was open to shortcomings with us in the demo. Specifics No, but answered the questioning.
Once you see how it works as an operator, it's fairly easy to see what it's not going to catch very well or at all. Many other detection systems also have their own flaws.
I do believe that reporting on product comparisons and shortcomings between like products in terms of who does what better is key to innovation and overall makes perfect sense, I think it may be a little premature to report on the shortcomings of these and any other not-so-similar detection systems. It's pretty much the reason why this market sector is very tight on distribution.
By all means report on its capabilities, highlight that "like all detections systems" they can't catch everything and may have x miss rate over x device but, specifics I would stay away from.
I don't come here to see the best scents to hide items from drug sniffing dogs, how to down surveillance drones, mask facial recognition, hide from LPR scanners, bypass lidar, best lasers to blind cameras, access codes to devices, etc., etc.
As others have mentioned, those that are in the know will not purchase these to replace or take place of a full-body scanner at most of the traditional screening checkpoints.
I have no ties to Evolve, It is true there is a need for a more affordable commercially available, less intrusive way to screen. I'm not going to like it everywhere when they hit their $595mil target.
U
Undisclosed #5
Dec 16, 2021I think it may be a little premature to report on the shortcomings of these and any other not-so-similar detection systems.
this is where I completely disagree with you.
if someone is selling any kind of 'solution', the expectation is that it actually IS a solution.
if it aint (yet) then what is premature is not the reporting on the deficiencies of the 'solution', but instead, it's the 'solution' itself.
UI
Undisclosed Integrator #10
Dec 16, 2021if someone is selling any kind of 'solution', the expectation is that it actually IS a solution.
It does serve a purpose, and what I'm saying is they made that purpose clear when it was pitched. It's a real object and you can drive it. No doubt somewhere it will end up in the news that it didn't catch this or that.
U
Undisclosed #5
Dec 16, 2021It does serve a purpose, and what I'm saying is they made that purpose clear when it was pitched.
I am not discounting the purpose... the purpose is clear.
instead I am just saying that if you are going to sell me something for a particular purpose, you should be able to show me that your thing can do the thing that I need done.
if I buy a stickup LED light for my pantry and after I install it no light occurs when I smack it, I should be able to ask why you are selling me a stickup LED light that doesn't work when I need light in my pantry.
No doubt somewhere it will end up in the news that it didn't catch this or that.
no doubt.
UI
Undisclosed Integrator #10
Dec 16, 2021after I install it no light occurs when I smack it
but does it need to be stated that the stickup light is not going to light up that soup can behind the pretzels that are behind the Cheetos, on the bottom extra deep shelf? Should it be the responsibility of the stick-up light to package it together with an, even more, portable magnetic flashlight?
U
Undisclosed #5
Dec 16, 2021but does it need to be stated that the stickup light is not going to light up that soup can behind the pretzels that are behind the Cheetos
not at all. and you are seemingly purposefully ignoring my point and introducing a diversion instead of a defense.
I am not anti technology and I am glad that there are those taking risks to bring new solutions to the market.
but when I mash the stickup LED light I want to see light.
that's it.
U
Undisclosed #12
Dec 16, 2021Perhaps if they didn't lead with the fear sale the weakness would be less glaring. When you tell me mass murders and threat actors are the scary problem your system solves, it had better well solve it.
UI
Undisclosed Integrator #13
Dec 16, 2021I have recently been working with a few companies that provide "gun detection" systems. Although I think that any system or procedures that can be put in place to detect a potential active shooter incident faster is great I believe the money spent on a gun detection system would yield greater benefits being utilized elsewhere. The world does not operate in a vacuum and it is only a matter of time until this type of system is public knowledge. Potential shooters will know to keep the weapon hidden until they begin to start shooting. It is my opinion that the time between when a gun detection system is activated and a "gunshot" detection system is activated will be about the same in the future. As of now a gunshot detection system is going to have a much higher probability of activating then a gun detection system picking up a person carrying a gun. I would advise my customers to spend the money on physical enhancements to the building, metal detectors, gunshot detection system, etc before using their capital on a gun detection system. As professionals, this is one area that we must put the safety of personnel before profit. Also, although I have zero law experience this seems like an area ripe for a company being sued if the system does not work in an active shooter incident.
JH
John Honovich
Dec 16, 2021I would advise my customers to spend the money on physical enhancements to the building, metal detectors, gunshot detection system, etc before using their capital on a gun detection system.
Structurally speaking, my general approach would be to prioritize gun detection systems (like Evolv) over gunshot detection systems (SDS, whoever) because I'd rather (try to) get the gun as they enter the building, rather than after the first shot, thoughts?
UI
Undisclosed Integrator #13
Dec 16, 2021I totally agree. I made a mistake in thinking that the Evolv system was a video analytic system that would detect a gun. My point was that shooters would just hide the weapon until they started to shoot which would negate the advanced warning over a gunshot detection system. If there was a system that detected weapons as a person entered a facility that would be far superior. I will have to conduct some more research to see how the Evolve system operates and why it would be better than just standard metal detectors. I will see about deleting my original post as it appears it does not apply to this discussion.
JH
John Honovich
Dec 16, 2021#13, you're good. For you and others, here is their marketing video embedded below that overviews the basic pitch/concept:
UM
Undisclosed Manufacturer #14
Dec 16, 2021Unlike the software world, where nearly every vulnerability can be patched once known, physical security systems (inclusive of people, process, and technology) are subject to the unyielding laws of physics and operational constraints that make it impossible to fix every vulnerability or correct every lapse in human judgment.
A weak argument.
Physical security manufacturers, while often taking longer to improve upon a security weakness, generally act with a greater sense of urgency to correct the weakness once they--or their customers--are made aware of it.
Take locks for example. Many a manufacturer is more apt to address a weakness in their padlock, high security lock, combination lock, etc... when the locksport community demonstrates that weakness for all to see.
Related--if you can't improve upon a weakness, then you likely are not marketing your solution with transparency--for what it "can" do.
Donald Maye
Dec 30, 2021UPDATE: Evolv's CEO, Peter George, agreed to allow IPVM to test their system while at the Imperial Capital Investors conference on December 15th. On December 16th, IPVM emailed Evolv to set up a time and place to perform the testing.
After receiving no response, we reached out twice more on December 20th and 27th. On December 28th Evolv responded with the following:
Over the past few weeks, it’s become evident that Evolv and IPVM have different perspectives regarding the best way to share information to improve public safety. We have concluded that it is not in the best interest of our customers, the security profession, the public, or Evolv to support IPVM’s efforts at this time. As we move into 2022, we remain committed to our founding mission: to make places where people work, live, and play safer.
To be clear, IPVM never asked for Evolv's support and its not clear what Evolv meant by that suggestion.
We followed up twice to clarify if Evolv plans not to respond to us going forward when we have questions on reporting. We will provide an update if Evolv responds.
UI
Undisclosed Integrator #9
Dec 31, 2021That "support" means no longer talking to you, until you stop reporting what our life saving solutions CANNOT DO. Mind boggling, and will always speak about this when going against evolv.
JH
John Honovich
Jan 24, 2022Was looking at an Evolv video from a few months ago and noticed an interesting contrast. While Evolv does not want the public to know about its issues, it has its branding and name all over its units that it promotes in its own marketing, e.g.:
By emphasizing who makes the "weapons detection" process it makes it far easier for real adversaries to research and understand its weaknesses. If Evolv really wants to be so super security-sensitive, they should hide their branding from their public deployments to make it harder for adversaries to figure out what technology is being used.
To be clear, I am not objecting to companies branding their products but if your position is that information about your product could be used to defeat it, it would be far more consistent to not give out such information by literally putting the name bolded on each product.
JH
John Honovich
Apr 05, 2022Update: Evolv has a "Transparency of Sensitive Information" document and Transparency in Physical Security: Q&A with John Pistole and Mike Ellenbogen blog post.
Oddly, Evolv justifies this by comparing it to cybersecurity where full disclosure is the norm:
Similar to cybersecurity and counterterrorism, protecting the methods and means, and sharing them only with trusted security partners is critical to preventing the exposure of potential vulnerabilities and compromising a security plan.
Evolv is feeling pressure from our reporting as well as criticism following Evolv preventing the NCS4 from publicly releasing its full report.
Donald Maye
May 23, 2022UPDATE: While Evolv objects still to IPVM's reporting on Evov's weaknesses, they are now regularly responding to our requests for comment. Previously, Evolv refused to speak with IPVM, see: Notice: Evolv Refuses To Speak With IPVM.
JH
John Honovich
Jun 07, 2023On two separate calls with IPVM, Evolv emphasized its belief that if IPVM did not report on these weaknesses that others would not be likely to do so, emphasizing the use of NDAs and other methods to keep problems in confidence.
Just looking back over this, now 18 months later. The weaknesses have since become very public news in various publications including: BBC Exposes Evolv with IPVM Research, Report: Atlanta Schools Not Told Of Evolv's Problems Missing Knives, Evolv Faces Scrutiny From Colorado, North Carolina, And Ohio, etc.