Critiquing Carnegie's AI Surveillance Paper
The Carnegie Endowment has issued an ambitious paper on the Global Expansion of AI Surveillance. While its aim is applaudable, the paper has significant problems in actually understanding what 'AI surveillance' is and what is being used where.
Inside our post, we examine the paper and key issues including:
- AI video surveillance has far more widespread availability than the report concludes as it misses substantial, even public, evidence of such sales.
- However, the report conflates conventional non-AI video surveillance with AI video surveillance, mischaracterizing substantial company and country usage.
- In particular and related, the report incorrectly judges and contrasts country AI surveillance usage, in particular, the far higher usage for the PRC vs western liberal democracies.
- Finally, the 'key companies' column per country suffers from significant fundamental errors, including a basic misunderstanding of what companies are 'key' in which countries.
On the positive side, we have engaged with the researcher Steven Feldstein and he has updated some portions based on our feedback and we aim to continue to provide input into their research and other global public policy efforts.
Paper And Sources
To review this work, see 3 main sources:
- The Global Expansion of AI Surveillance paper (the main text)
- The AI Global Surveillance (AIGS) Index covering 70+ countries including 'key companies' providing AI surveillance
- The report's sources indexed in Zotero including 5 IPVM articles
Broader Availability
AI video surveillance is far more widely available than the report indicates, underestimating Hikvison's position and broader Chinese manufacturer availability. For example, the report says "at least seventy-five out of 176 countries globally are actively using AI technologies" and that "Huawei is the leading vendor of advanced surveillance systems worldwide by a huge factor".
By contrast, Hikvision alone is selling AI video surveillance is virtually every country in the world, as Hikvision has made AI a standard part of their commercial offering. For example, the same facial recognition camera model Hikvision deploys in Xinjiang can be bought online in the USA from '123SecurityProducts'. And up until recently, that model was available for sale on Aliexpress. It is not just the US and China for Hikvision. Last year, Hikvision did a 71 city / 20 country European AI roadshow plus a 10-city South Africa AI roadshow, a Dubai AI roadshow stop, etc.
Hikvision and Huawei are far from alone in this. Literally dozens of Chinese manufacturers are offering AI video surveillance, we covered various ones at two recent trade shows (ISC West and Secutech). Moreover, low-cost facial recognition cameras are easily found on mainstream marketplaces like Amazon and Aliexpress;
While it certainly makes sense to prominently feature Huawei, better understanding Hikvision's role, as well as the emerging array of smaller China manufacturers that want to sell facial recognition as a low-cost mass-market offering, are key to understanding usage and risks globally.
Feldstein responded explaining:
I want to reiterate that the purpose of the research (and what’s of greatest interest to me) was to document: 1) government usage of AI surveillance, 2) usage at scale (several standalone cameras purchased by a police department doesn’t tell much; 1000 facial recognition cameras positioned in a capital city and linked to a security operations center is a much different story), 3) usage of AI surveillance that includes but is not limited to video surveillance. The research extends beyond AI video surveillance. It also includes a range of other technologies -- such as smart policing algorithms used to identify suspects and discern patterns of criminal behavior. Thus, while AI video surveillance is an important pillar of the research, there are other technologies and considerations that I incorporated as well.
We shared with Feldstein that, on the 'government usage' side, e.g. Canada, a quick check of the report's sources show that many of them have nothing to do with AI surveillance, e.g. (1) the City of Markham says it is about "detect leaks in water supply pipes, storm flooding, energy usage in city buildings as well as environmental conditions such as temperature and humidity"; (2) this CBC article sourced only cites malls testing out face recognition, not governments and (3) this Microsoft document talks about Canada developing AI (surely true) but nothing about surveillance or Canadian government entities using surveillance.
Feldman responded with clarifications and updates to the report:
About my focus on government usage of surveillance technology, you're right that I included a wide range of systems, both surveillance non-surveillance related, under the tag of "smart city." I intended for the smart city tag to simply note where such systems were being deployed. An article tagged as a "smart city" was not included in the index unless it also had a "safe city" tag as well. To make sure this is clearer, I've now changed the tag so that it reads: "smart city (non-surveillance)." Any item tagged with this label alone is not counted in the index. For example, while I decided to include the Markham example in the Zotero database because it illustrates the presence of smart city technology, it did not count towards AI surveillance in Canada. If you look at the full table index, you'll see that I did not include a checkmark under the smart city/safe city column for Canada.
As for the mall example you cited, you're correct. It's operated by a private developer and should not be included. I've corrected the issue and taken it out of the database.
Not "AI" Surveillance, Just Surveillance
Many of the sources used to justify 'AI surveillance' are not actually "AI surveillance". For example, different Axis sources included are clearly not using 'AI', certainly not from Axis:
- An Axis China case study from 2009 - no AI, very old and simple IP cameras (Axis models 207 and 221)
- An Axis South Korea case study from 2009 - no AI, very old and simple IP cameras (Axis model 233D)
- An Axis Argentina case study from 2013 - Axis only supplied basic IP cameras (M1113), another company provided LPR, no facial recognition or other AI
A significant distinction exists between the systems described in these case studies and emerging AI-driven surveillance deployments.
Moreover, Pelco is inaccurately cited as a 'key company' in the US for AI surveillance demonstrating problems in how companies and countries were assessed. The report has just 2 Pelco sources tagged, shown below:
The first source is an article that only mentions Pelco issuing a press release about partnering with IBM. Pelco is not an AI provider and their partner IBM has since shut down its video surveillance AI. The second one is from a dubious market report provider that simply mentions Pelco by name, without any evidence. That same dubious report used as a source mentions Pelco and Infinova together.
The Carnegie report also lists Infinova as a 'key company' in the US for AI surveillance but the only source provided is from a Los Angeles Metro Infinova case study that clearly is just conventional surveillance and has nothing to do with AI. While undated, the technology promoted (up-the-coax) infers it is many years old. The repeated conflation of AI and non-AI obscures what the real usage and risks are.
Feldman responded:
Regarding the critique that the research conflates non-AI video surveillance with AI video surveillance -- thank you for pointing out source material that was improperly included, such as inclusion of Axis products. I have updated the index to reflect those changes; future public versions will also include these modifications. Overall, I did my utmost to distinguish AI vs. non-AI technology and discounted many sources as a result.
Key Countries Issues
The report created an AI Global Surveillance (AIGS) Index, which is "a country-by-country snapshot of AI tech surveillance" that is based on publicly-available documentation i.e. reliable media reports, government announcements, or other solid sources related to AI deployments.
The index lists countries by ten factors, including each country's "key companies":
Because of this methodology focusing purely on decontextualized publicly-available documents, the "key company" section includes numerous errors. The PRC (China) and USA categories are particularly wrong:
There is no mention of China's massive facial recognition startups, like Megvii, which recently filed a detailed IPO and makes 95% of its business in China. Rather, Axis is listed, which has a negligible presence in China. There is no mention of the fact that China explicitly discriminates against foreign firms for its AI surveillance projects.
And on the US side, there is no mention of Briefcam, by far the most common AI service used by US police departments, or Axis, the most popular enterprise camera provider.
The "key companies" category has many other issues, including but not limited to:
- Israeli facial recognition firm Anyvision is only listed as a "key company" in one country (Israel.) However, Anyvision is currently the largest non-Chinese facial recognition company in the world, with a big global presence. It is aiming for $1 billion in sales by 2022 and hiring sales managers all over the world, such as Thailand, Indochina, and Hong Kong.
- China's Sensetime facial recognition firm is only listed as a "key company" in one country: Mongolia. Yet it's the world's most highly-valued AI startup (worth $7 billion) based largely on its extensive PRC activities, where the government is a major customer. Sensetime also has a large presence in Singapore where it expects to triple staff to 300 over the next 3 years, indicating the firm has substantial Southeast Asia activities as well.
- Panasonic is not listed for Japan or anywhere else. Yet they are a main facial recognition provider in Japan, where their solution is used by authorities at the Tokyo airport. Panasonic told us previously that their face rec solution has been deployed in five major airports in Japan, where they have a 60% networked camera market share.
- There is zero mention of Axxonsoft, a significant facial recognition/intelligent VMS provider especially in Russia with an AI-related deployment for the Moscow 2018 World Cup and with large AI projects elsewhere i.e. a large city in South Korea, a safe city in Ukraine, Mexico's main airport, etc.
Ranking Companies Issues
The report ranks companies by the number of countries they have been found in:
This is a risky method of ranking companies' global presence. It says nothing about the type of AI surveillance: IBM canceled its entire Intelligent Video Analytics program in Spring 2019, so they do not currently provide AI solutions comparable to the major PRC firms. Additionally, the graph does not reflect the fact that Hikvision and Dahua sell AI products in most of the countries in the entire world, yet Dahua is not even mentioned and Hikvision is just one country ahead of NEC.
This graph is largely a product of the fact that Huawei and IBM have better documented their safe/smart city projects. It is not a reflection of the reality on the ground.
Feldman responded about challenges with key companies noting:
I don't want to completely omit reference to companies, but I also want to be accurate about who is responsible for supplying what equipment.
We can empathize with him. While it is valuable to provide such information, it is risky and hard (even for us) to precisely say who are the 'key companies' in 70+ countries around the world.
Background
The report was released on September 17, 2019 by the Carnegie Endowment for International Peace, the first US 'think tank' which was founded in 1910 with the goal of "advanc[ing] peace" worldwide.
Global Media Coverage
The report received significant media coverage:
- Wall Street Journal - World Catching Up With China on Surveillance Tech
- Associated Press - Researchers: AI surveillance is expanding worldwide
- Al Jazeera - Researchers warn of increased China-style AI surveillance
Overall Good For Researchers To Pursue
Overall, it is good for researchers to better cover AI and surveillance, especially as the technology becomes more powerful and potentially more destructive.
And given Feldstein's constraints ("one full-time researcher plus volunteer research assistance"), it is impressive that he came as far as he did. Plus, he has already shown a willingness and capacity to improve his research so we look forward to further developments from him and the Carnegie Endowment generally.