Critiquing Carnegie's AI Surveillance Paper

By: John Honovich and Charles Rollet, Published on Sep 25, 2019

The Carnegie Endowment has issued an ambitious paper on the Global Expansion of AI Surveillance. While its aim is applaudable, the paper has significant problems in actually understanding what 'AI surveillance' is and what is being used where.

Inside our post, we examine the paper and key issues including:

  • AI video surveillance has far more widespread availability than the report concludes as it misses substantial, even public, evidence of such sales.
  • However, the report conflates conventional non-AI video surveillance with AI video surveillance, mischaracterizing substantial company and country usage.
  • In particular and related, the report incorrectly judges and contrasts country AI surveillance usage, in particular, the far higher usage for the PRC vs western liberal democracies.
  • Finally, the 'key companies' column per country suffers from significant fundamental errors, including a basic misunderstanding of what companies are 'key' in which countries.

On the positive side, we have engaged with the researcher Steven Feldstein and he has updated some portions based on our feedback and we aim to continue to provide input into their research and other global public policy efforts.

Paper And Sources

To review this work, see 3 main sources:

Broader Availability

AI video surveillance is far more widely available than the report indicates, underestimating Hikvison's position and broader Chinese manufacturer availability. For example, the report says "at least seventy-five out of 176 countries globally are actively using AI technologies" and that "Huawei is the leading vendor of advanced surveillance systems worldwide by a huge factor".

By contrast, Hikvision alone is selling AI video surveillance is virtually every country in the world, as Hikvision has made AI a standard part of their commercial offering. For example, the same facial recognition camera model Hikvision deploys in Xinjiang can be bought online in the USA from '123SecurityProducts'. And up until recently, that model was available for sale on Aliexpress. It is not just the US and China for Hikvision. Last year, Hikvision did a 71 city / 20 country European AI roadshow plus a 10-city South Africa AI roadshow, a Dubai AI roadshow stop, etc.

Hikvision and Huawei are far from alone in this. Literally dozens of Chinese manufacturers are offering AI video surveillance, we covered various ones at two recent trade shows (ISC West and Secutech). Moreover, low-cost facial recognition cameras are easily found on mainstream marketplaces like Amazon and Aliexpress; one Chinese-American firm (DeepCam) recently touted facial recognition cameras for $59 apiece at a trade show.

Get Notified of Video Surveillance Breaking News
Get Notified of Video Surveillance Breaking News

While it certainly makes sense to prominently feature Huawei, better understanding Hikvision's role, as well as the emerging array of smaller China manufacturers that want to sell facial recognition as a low-cost mass-market offering, are key to understanding usage and risks globally.

Feldstein responded explaining:

I want to reiterate that the purpose of the research (and what’s of greatest interest to me) was to document: 1) government usage of AI surveillance, 2) usage at scale (several standalone cameras purchased by a police department doesn’t tell much; 1000 facial recognition cameras positioned in a capital city and linked to a security operations center is a much different story), 3) usage of AI surveillance that includes but is not limited to video surveillance. The research extends beyond AI video surveillance. It also includes a range of other technologies -- such as smart policing algorithms used to identify suspects and discern patterns of criminal behavior. Thus, while AI video surveillance is an important pillar of the research, there are other technologies and considerations that I incorporated as well.

We shared with Feldstein that, on the 'government usage' side, e.g. Canada, a quick check of the report's sources show that many of them have nothing to do with AI surveillance, e.g. (1) the City of Markham says it is about "detect leaks in water supply pipes, storm flooding, energy usage in city buildings as well as environmental conditions such as temperature and humidity"; (2) this CBC article sourced only cites malls testing out face recognition, not governments and (3) this Microsoft document talks about Canada developing AI (surely true) but nothing about surveillance or Canadian government entities using surveillance.

Feldman responded with clarifications and updates to the report:

About my focus on government usage of surveillance technology, you're right that I included a wide range of systems, both surveillance non-surveillance related, under the tag of "smart city." I intended for the smart city tag to simply note where such systems were being deployed. An article tagged as a "smart city" was not included in the index unless it also had a "safe city" tag as well. To make sure this is clearer, I've now changed the tag so that it reads: "smart city (non-surveillance)." Any item tagged with this label alone is not counted in the index. For example, while I decided to include the Markham example in the Zotero database because it illustrates the presence of smart city technology, it did not count towards AI surveillance in Canada. If you look at the full table index, you'll see that I did not include a checkmark under the smart city/safe city column for Canada.

As for the mall example you cited, you're correct. It's operated by a private developer and should not be included. I've corrected the issue and taken it out of the database.

Not "AI" Surveillance, Just Surveillance

Many of the sources used to justify 'AI surveillance' are not actually "AI surveillance". For example, different Axis sources included are clearly not using 'AI', certainly not from Axis:

A significant distinction exists between the systems described in these case studies and emerging AI-driven surveillance deployments.

Moreover, Pelco is inaccurately cited as a 'key company' in the US for AI surveillance demonstrating problems in how companies and countries were assessed. The report has just 2 Pelco sources tagged, shown below:

The first source is an article that only mentions Pelco issuing a press release about partnering with IBM. Pelco is not an AI provider and their partner IBM has since shut down its video surveillance AI. The second one is from a dubious market report provider that simply mentions Pelco by name, without any evidence. That same dubious report used as a source mentions Pelco and Infinova together.

The Carnegie report also lists Infinova as a 'key company' in the US for AI surveillance but the only source provided is from a Los Angeles Metro Infinova case study that clearly is just conventional surveillance and has nothing to do with AI. While undated, the technology promoted (up-the-coax) infers it is many years old. The repeated conflation of AI and non-AI obscures what the real usage and risks are.

Feldman responded:

Regarding the critique that the research conflates non-AI video surveillance with AI video surveillance -- thank you for pointing out source material that was improperly included, such as inclusion of Axis products. I have updated the index to reflect those changes; future public versions will also include these modifications. Overall, I did my utmost to distinguish AI vs. non-AI technology and discounted many sources as a result.

Key Countries Issues

The report created an AI Global Surveillance (AIGS) Index, which is "a country-by-country snapshot of AI tech surveillance" that is based on publicly-available documentation i.e. reliable media reports, government announcements, or other solid sources related to AI deployments.

The index lists countries by ten factors, including each country's "key companies":

Because of this methodology focusing purely on decontextualized publicly-available documents, the "key company" section includes numerous errors. The PRC (China) and USA categories are particularly wrong:

There is no mention of China's massive facial recognition startups, like Megvii, which recently filed a detailed IPO and makes 95% of its business in China. Rather, Axis is listed, which has a negligible presence in China. There is no mention of the fact that China explicitly discriminates against foreign firms for its AI surveillance projects.

And on the US side, there is no mention of Briefcam, by far the most common AI service used by US police departments, or Axis, the most popular enterprise camera provider.

The "key companies" category has many other issues, including but not limited to:

Ranking Companies Issues

The report ranks companies by the number of countries they have been found in:

image

This is a risky method of ranking companies' global presence. It says nothing about the type of AI surveillance: IBM canceled its entire Intelligent Video Analytics program in Spring 2019, so they do not currently provide AI solutions comparable to the major PRC firms. Additionally, the graph does not reflect the fact that Hikvision and Dahua sell AI products in most of the countries in the entire world, yet Dahua is not even mentioned and Hikvision is just one country ahead of NEC.

This graph is largely a product of the fact that Huawei and IBM have better documented their safe/smart city projects. It is not a reflection of the reality on the ground.

Feldman responded about challenges with key companies noting:

I don't want to completely omit reference to companies, but I also want to be accurate about who is responsible for supplying what equipment.

We can empathize with him. While it is valuable to provide such information, it is risky and hard (even for us) to precisely say who are the 'key companies' in 70+ countries around the world.

Background

The report was released on September 17, 2019 by the Carnegie Endowment for International Peace, the first US 'think tank' which was founded in 1910 with the goal of "advanc[ing] peace" worldwide.

Global Media Coverage

The report received significant media coverage:

Overall Good For Researchers To Pursue

Overall, it is good for researchers to better cover AI and surveillance, especially as the technology becomes more powerful and potentially more destructive.

And given Feldstein's constraints ("one full-time researcher plus volunteer research assistance"), it is impressive that he came as far as he did. Plus, he has already shown a willingness and capacity to improve his research so we look forward to further developments from him and the Carnegie Endowment generally.

Comments (3) : PRO Members only. Login. or Join.

Related Reports

Hikvision Global News Reports Directory on Oct 17, 2019
Hikvision has received the most global news reporting of any video surveillance company, ever, ranging from the WSJ, the Financial Times, Reuters,...
US DoD Comments on Huawei, Hikvision, Dahua Cyber Security Concerns on Oct 16, 2019
A senior DoD official said the US is "concerned" with the cybersecurity of Hikvision, Dahua, and Huawei due to "CCP" (China Communist Party)...
Hikvision Dissolves North American Business Unit, Splits Canada and USA on Oct 15, 2019
Hikvision has dissolved its North American Business Unit, splitting up US and Canada operations as the PRC-government owned manufacturer faces...
Dahua Celebrates PRC 70th Wearing Communist Party Hammer and Sickle on Oct 11, 2019
Dahua celebrated the PRC's 70th anniversary with a video of various Dahua employees wearing China Communist Party hammer and sickle pins as shown...
Network Optix NxWitness 4.0 Tested on Oct 10, 2019
Network Optix released Nx Witness 4.0, proclaiming new features like a deep learning analytics metadata SDK, increased H.265 support, and UX...
ONVIF Suspends Dahua and Hikvision on Oct 09, 2019
Dahua and Hikvision have been 'suspended', and effectively expelled, from ONVIF, immediately following US sanctions being placed on the 2 mega...
Hikvision And Dahua Sanctioned For Human Rights Abuses on Oct 07, 2019
In a groundbreaking move that will have drastic consequences across the video surveillance market, Dahua and Hikvision have been sanctioned by the...
Crisis At China's Largest VMS Provider, Netposa, Now State-Controlled on Oct 07, 2019
NetPosa, which bills itself as the PRC's largest VMS provider, is in a crisis. The firm is pursuing huge unpaid bills from clients, and its...
Knightscope Sells Just 1 Net New Robot In 6 Months on Oct 04, 2019
For the first half of 2019, US government records show that Knightscope has sold just 1 net new robots ('machines-in-network'), inching up from 52...
US - China Review Commission Cites IPVM on Foreign Provider Threat on Oct 01, 2019
A bipartisan congressional commission cited IPVM twice in its analysis of how the PRC government protects its surveillance firms from foreign...

Most Recent Industry Reports

Alarm Veteran "Demands A Criminal Investigation" Of UL on Oct 18, 2019
The Interceptor's Project pressure against UL continues to rise. Following Keith Jentoft's allegation that "UL Has Blood On Their Hands", Jentoft...
Camect "Worlds Smartest Camera Hub" Tested on Oct 18, 2019
Camect is a Silicon Valley startup that claims the "Smartest AI Object Detection On The Market", detecting not only people and vehicles, but...
Hikvision Global News Reports Directory on Oct 17, 2019
Hikvision has received the most global news reporting of any video surveillance company, ever, ranging from the WSJ, the Financial Times, Reuters,...
Camera Calculator V3.1 Release Improves User Experience on Oct 17, 2019
IPVM has released a new version of our Camera Calculator, V3.1, with significant user experience improvements, a new development plan, and an...
Securing Access Control Installations Tutorial on Oct 17, 2019
The physical security of access control components is critical to ensuring that a facility is truly secure. Otherwise, the entire system can be...
Access Control Course Fall 2019 - Last Chance on Oct 17, 2019
Register Now - Fall 2019 Access Control Course. Thursday, October 17th is the last day to register. IPVM offers the most comprehensive access...
US DoD Comments on Huawei, Hikvision, Dahua Cyber Security Concerns on Oct 16, 2019
A senior DoD official said the US is "concerned" with the cybersecurity of Hikvision, Dahua, and Huawei due to "CCP" (China Communist Party)...
Pelco Sarix Pro3 Camera Tested on Oct 16, 2019
Pelco has released their Sarix Professional Series 3 cameras, claiming "more security detail in challenging scenes with excellent low light and...
IPVM Camera Calculator User Manual / Guide on Oct 16, 2019
Learn how to use the IPVM Camera Calculator. The guide below includes instructions, images, gifs, and videos demonstrating and explaining the...
Altronix Claims Tango 'Eliminates Electricians' on Oct 15, 2019
Power supply provider Altronix claims its new Tango power supply 'eliminates the need for an electrician, dedicated conduit and wire runs'. In...