Critiquing Carnegie's AI Surveillance Paper

By: John Honovich and Charles Rollet, Published on Sep 25, 2019

The Carnegie Endowment has issued an ambitious paper on the Global Expansion of AI Surveillance. While its aim is applaudable, the paper has significant problems in actually understanding what 'AI surveillance' is and what is being used where.

Inside our post, we examine the paper and key issues including:

  • AI video surveillance has far more widespread availability than the report concludes as it misses substantial, even public, evidence of such sales.
  • However, the report conflates conventional non-AI video surveillance with AI video surveillance, mischaracterizing substantial company and country usage.
  • In particular and related, the report incorrectly judges and contrasts country AI surveillance usage, in particular, the far higher usage for the PRC vs western liberal democracies.
  • Finally, the 'key companies' column per country suffers from significant fundamental errors, including a basic misunderstanding of what companies are 'key' in which countries.

On the positive side, we have engaged with the researcher Steven Feldstein and he has updated some portions based on our feedback and we aim to continue to provide input into their research and other global public policy efforts.

Paper And Sources

To review this work, see 3 main sources:

Broader Availability

AI video surveillance is far more widely available than the report indicates, underestimating Hikvison's position and broader Chinese manufacturer availability. For example, the report says "at least seventy-five out of 176 countries globally are actively using AI technologies" and that "Huawei is the leading vendor of advanced surveillance systems worldwide by a huge factor".

By contrast, Hikvision alone is selling AI video surveillance is virtually every country in the world, as Hikvision has made AI a standard part of their commercial offering. For example, the same facial recognition camera model Hikvision deploys in Xinjiang can be bought online in the USA from '123SecurityProducts'. And up until recently, that model was available for sale on Aliexpress. It is not just the US and China for Hikvision. Last year, Hikvision did a 71 city / 20 country European AI roadshow plus a 10-city South Africa AI roadshow, a Dubai AI roadshow stop, etc.

Hikvision and Huawei are far from alone in this. Literally dozens of Chinese manufacturers are offering AI video surveillance, we covered various ones at two recent trade shows (ISC West and Secutech). Moreover, low-cost facial recognition cameras are easily found on mainstream marketplaces like Amazon and Aliexpress; one Chinese-American firm (DeepCam) recently touted facial recognition cameras for $59 apiece at a trade show.

Get Notified of Video Surveillance Breaking News
Get Notified of Video Surveillance Breaking News

While it certainly makes sense to prominently feature Huawei, better understanding Hikvision's role, as well as the emerging array of smaller China manufacturers that want to sell facial recognition as a low-cost mass-market offering, are key to understanding usage and risks globally.

Feldstein responded explaining:

I want to reiterate that the purpose of the research (and what’s of greatest interest to me) was to document: 1) government usage of AI surveillance, 2) usage at scale (several standalone cameras purchased by a police department doesn’t tell much; 1000 facial recognition cameras positioned in a capital city and linked to a security operations center is a much different story), 3) usage of AI surveillance that includes but is not limited to video surveillance. The research extends beyond AI video surveillance. It also includes a range of other technologies -- such as smart policing algorithms used to identify suspects and discern patterns of criminal behavior. Thus, while AI video surveillance is an important pillar of the research, there are other technologies and considerations that I incorporated as well.

We shared with Feldstein that, on the 'government usage' side, e.g. Canada, a quick check of the report's sources show that many of them have nothing to do with AI surveillance, e.g. (1) the City of Markham says it is about "detect leaks in water supply pipes, storm flooding, energy usage in city buildings as well as environmental conditions such as temperature and humidity"; (2) this CBC article sourced only cites malls testing out face recognition, not governments and (3) this Microsoft document talks about Canada developing AI (surely true) but nothing about surveillance or Canadian government entities using surveillance.

Feldman responded with clarifications and updates to the report:

About my focus on government usage of surveillance technology, you're right that I included a wide range of systems, both surveillance non-surveillance related, under the tag of "smart city." I intended for the smart city tag to simply note where such systems were being deployed. An article tagged as a "smart city" was not included in the index unless it also had a "safe city" tag as well. To make sure this is clearer, I've now changed the tag so that it reads: "smart city (non-surveillance)." Any item tagged with this label alone is not counted in the index. For example, while I decided to include the Markham example in the Zotero database because it illustrates the presence of smart city technology, it did not count towards AI surveillance in Canada. If you look at the full table index, you'll see that I did not include a checkmark under the smart city/safe city column for Canada.

As for the mall example you cited, you're correct. It's operated by a private developer and should not be included. I've corrected the issue and taken it out of the database.

Not "AI" Surveillance, Just Surveillance

Many of the sources used to justify 'AI surveillance' are not actually "AI surveillance". For example, different Axis sources included are clearly not using 'AI', certainly not from Axis:

A significant distinction exists between the systems described in these case studies and emerging AI-driven surveillance deployments.

Moreover, Pelco is inaccurately cited as a 'key company' in the US for AI surveillance demonstrating problems in how companies and countries were assessed. The report has just 2 Pelco sources tagged, shown below:

The first source is an article that only mentions Pelco issuing a press release about partnering with IBM. Pelco is not an AI provider and their partner IBM has since shut down its video surveillance AI. The second one is from a dubious market report provider that simply mentions Pelco by name, without any evidence. That same dubious report used as a source mentions Pelco and Infinova together.

The Carnegie report also lists Infinova as a 'key company' in the US for AI surveillance but the only source provided is from a Los Angeles Metro Infinova case study that clearly is just conventional surveillance and has nothing to do with AI. While undated, the technology promoted (up-the-coax) infers it is many years old. The repeated conflation of AI and non-AI obscures what the real usage and risks are.

Feldman responded:

Regarding the critique that the research conflates non-AI video surveillance with AI video surveillance -- thank you for pointing out source material that was improperly included, such as inclusion of Axis products. I have updated the index to reflect those changes; future public versions will also include these modifications. Overall, I did my utmost to distinguish AI vs. non-AI technology and discounted many sources as a result.

Key Countries Issues

The report created an AI Global Surveillance (AIGS) Index, which is "a country-by-country snapshot of AI tech surveillance" that is based on publicly-available documentation i.e. reliable media reports, government announcements, or other solid sources related to AI deployments.

The index lists countries by ten factors, including each country's "key companies":

Because of this methodology focusing purely on decontextualized publicly-available documents, the "key company" section includes numerous errors. The PRC (China) and USA categories are particularly wrong:

There is no mention of China's massive facial recognition startups, like Megvii, which recently filed a detailed IPO and makes 95% of its business in China. Rather, Axis is listed, which has a negligible presence in China. There is no mention of the fact that China explicitly discriminates against foreign firms for its AI surveillance projects.

And on the US side, there is no mention of Briefcam, by far the most common AI service used by US police departments, or Axis, the most popular enterprise camera provider.

The "key companies" category has many other issues, including but not limited to:

Ranking Companies Issues

The report ranks companies by the number of countries they have been found in:

image

This is a risky method of ranking companies' global presence. It says nothing about the type of AI surveillance: IBM canceled its entire Intelligent Video Analytics program in Spring 2019, so they do not currently provide AI solutions comparable to the major PRC firms. Additionally, the graph does not reflect the fact that Hikvision and Dahua sell AI products in most of the countries in the entire world, yet Dahua is not even mentioned and Hikvision is just one country ahead of NEC.

This graph is largely a product of the fact that Huawei and IBM have better documented their safe/smart city projects. It is not a reflection of the reality on the ground.

Feldman responded about challenges with key companies noting:

I don't want to completely omit reference to companies, but I also want to be accurate about who is responsible for supplying what equipment.

We can empathize with him. While it is valuable to provide such information, it is risky and hard (even for us) to precisely say who are the 'key companies' in 70+ countries around the world.

Background

The report was released on September 17, 2019 by the Carnegie Endowment for International Peace, the first US 'think tank' which was founded in 1910 with the goal of "advanc[ing] peace" worldwide.

Global Media Coverage

The report received significant media coverage:

Overall Good For Researchers To Pursue

Overall, it is good for researchers to better cover AI and surveillance, especially as the technology becomes more powerful and potentially more destructive.

And given Feldstein's constraints ("one full-time researcher plus volunteer research assistance"), it is impressive that he came as far as he did. Plus, he has already shown a willingness and capacity to improve his research so we look forward to further developments from him and the Carnegie Endowment generally.

Comments (3) : PRO Members only. Login. or Join.

Related Reports

Hikvision DS 2nd Gen Intercom Tested on Dec 12, 2019
With its newest IP intercom, Hikvision proclaims users can 'get full control over an entrance' regardless of where it is installed, home or office...
Acquisitions - Winners and Losers on Dec 10, 2019
Most major manufacturers have been acquired over the last decade. But which have been good deals or not? In this report, we analyze the...
IP Camera Installability Shootout 2019 - Avigilon, Axis, Bosch, Dahua, Hanwha, Hikvision, Uniview, Vivotek on Dec 09, 2019
What are the best and worst cameras to install? Which manufacturers make it the hardest or easiest to install their cameras? We tested 35 total...
Security Sales Course 2020 on Dec 05, 2019
This sales course is customized for the current needs and challenges specific to professionals selling video surveillance and access control...
Ireland National Children's Hospital Chooses Hikvision End-to-End With Facial Recognition on Dec 05, 2019
The world's most expensive hospital project ever, the New Children's Hospital in Ireland, has chosen an all-Hikvision surveillance system including...
Hikvision Global News Reports Directory on Dec 02, 2019
Hikvision has received the most global news reporting of any video surveillance company, ever, ranging from the WSJ, the Financial Times, Reuters,...
Freebie Face-Off: Rise of ~$30 Giveaways Examined on Dec 02, 2019
The freebies are growing. Verkada started this (see The Fastest Growing Video Surveillance Sales Organization Ever - Verkada) and now has given...
China Uyghur Analytic Projects Require Intel And NVIDIA, Intel Condemns, NVIDIA Silent on Dec 02, 2019
At least 8 PRC China police projects require NVIDIA and Intel chips to power their Uyghur-detecting analytics, according to procurement documents...
Directory of Access Reader Manufacturers on Nov 27, 2019
Credential Readers are one of the most visible and noticeable parts of access systems, but installers often stick with only the brand they always...
Vunetrix Health Monitoring Company Profile on Nov 26, 2019
Vunetrix boasts that they make the integrators the 'HERO' by using Vunetrix's monitoring. We spoke to Vunetrix to better understand their...

Most Recent Industry Reports

Video Surveillance 101 Course Opened on Dec 12, 2019
IPVM is adding a Video Surveillance 101 course, designed to help those new to the industry to quickly understand the most important terms,...
Verkada Notification Outage on Dec 12, 2019
Verkada is suffering an event notification outage and analytic search failures. Inside, we examine what the issues are, what Verkada told IPVM...
Hikvision DS 2nd Gen Intercom Tested on Dec 12, 2019
With its newest IP intercom, Hikvision proclaims users can 'get full control over an entrance' regardless of where it is installed, home or office...
Honeywell 30 Series Cameras Tested Vs Dahua and Hikvision on Dec 11, 2019
Honeywell has infamously OEMed Dahua and Hikvision for years, but now they have introduced an NDAA-compliant line, the 30 Series, claiming "lower...
"Good Market, Bad Business Models" - Residential Security on Dec 11, 2019
Industry banker John Mack, at his company's annual event, took aim squarely at the problems in the residential security...
IP Camera Browser Support: Who's Broken / Who Works on Dec 10, 2019
For many years, IP cameras depended on ActiveX control, whose security flaws have been known for more than a decade. The good news is that this is...
Acquisitions - Winners and Losers on Dec 10, 2019
Most major manufacturers have been acquired over the last decade. But which have been good deals or not? In this report, we analyze the...
IP Camera Installability Shootout 2019 - Avigilon, Axis, Bosch, Dahua, Hanwha, Hikvision, Uniview, Vivotek on Dec 09, 2019
What are the best and worst cameras to install? Which manufacturers make it the hardest or easiest to install their cameras? We tested 35 total...
Viisights Raises $10 Million, Behavior Analytics Company Profile on Dec 09, 2019
Viisights, an Israeli AI analytics startup marketing "Behavioral Understanding Systems", announced $10 million Series A funding. We spoke to...
Disruptor Wyze Releases Undisruptive Smartlock on Dec 06, 2019
While Wyze has disrupted the consumer IP camera market with ~$20 cameras, its entrance into smart locks is entirely undisruptive. We have...