Underestimated: Stanford Study On Surveillance Cameras In Large Cities Worldwide

By IPVM Team, Published Jun 28, 2021, 11:37am EDT

IPVM analysis finds Stanford researchers have significantly underestimated the number of outdoor surveillances in 10 US cities plus Bangkok, London, Paris, Seoul, Singapore, and Tokyo.

IPVM Image

IPVM analyzes the methodology and summarizes its findings including talking with the team behind it and the Amnesty International team behind a similar study.

Study Shows Camera Count Small and Declining

The study estimates that outdoor surveillance camera usage is quite small in most cities and actually has declined over the decade period they tracked usage.

IPVM Image

Decline Highly Unlikely

While the researchers found that camera count was overall flat and declined in many cities, such as NY, Baltimore, and Washington, DC., we believe this is highly unlikely. Camera sales increased significantly over the last decade, by dollar value, easily over 50%, and by units (most relevant to this), by 100%+ (due to the race to the bottom). Combined, this means, camera counts should be not decreasing.

Smaller Cameras Could Lead to More Missed Detections

The average size of cameras has shrunk which may have caused more missed detections in the newer 2016-2020 images than older 2011-2015 images causing them to estimate camera count is flat. For example, in the early 2010s, PTZs were far more common and while later in the decade the market increasingly shifted to minidomes, which are harder for Stanford's algorithm to detect.

Join IPVM Newsletter?

IPVM is the authority on physical security technology including video surveillance, access control, weapons detection and more. Refusing to accept advertising or sponsorships, over 15,000 subscribers globally trust and pay for IPVM's independent reporting and research.

Enter your email to get notified of new shootouts, tests, investigations, online shows and more.

IPVM Image

New York City Estimate Low

Examining New York City, which the study claims have just over 10,000 cameras, IPVM found a number of factors that point to the Study significantly underestimating total camera count:

  • There were 33,000 retail businesses in New York City, according to New York State. If no other building outside of retailers had outdoor cameras, the study implies that only 1 of out 3 retailers would have a single camera, which by itself is low.
  • Moreover, there are 12,500 intersections with traffic lights in New York City according to the City. These often have cameras on them and signal busier areas where cameras are more common.
  • There are over 1 million buildings in New York City according to New York City. The study is effectively estimating that only 1 out of 100 buildings have a single camera. Even if every building was residential this would be a low estimate.
  • IPVM walked around residential areas of Brooklyn and observed cameras on many houses. IPVM estimates that no less than 1/50 houses had cameras and those that did often had multiple, as seen below:

IPVM Image

Study Overview

Stanford researchers Hao Sheng, Keniel Yao, and Sharad Goel used deep learning on Google Street View images to estimate the number of outdoor surveillance cameras in 10 US cities (Boston, New York City, Chicago, Washington D.C., Los Angeles, Seattle, Milwaukee, San Francisco, Baltimore, and Philadelphia) and 6 cities outside the US (Tokyo, Seoul, Paris, London, Bangkok, and Singapore).

Motivation

The Stanford team says they were motivated by a lack of overall data on surveillance cameras per city, specifically:

Despite the wide-ranging implications of surveillance cameras on public safety, police enforcement, and democratic governance, relatively little is known about the precise number and placement of cameras, hampering efforts to assess their impacts

Methodology

The team created a deep learning model to locate cameras on Google Street View images and ran it on randomly selected stretches of road. Humans verified that the cameras found by the algorithm were cameras, not false detections. Next, the researchers adjusted their estimate to compensate for missed detections (effectively increasing count found by 58.7%), using the missed-detection rate they calculated by testing their algorithm on a hand-labeled dataset they created. Finally, the researchers multiplied their adjusted estimate by the proportion of roads they had analyzed to estimate the count for the whole city, not just the subsection they looked at.

Amnesty International Counts 50%+ More Cameras in NYC

Amnesty International volunteers counted 15,000+ cameras in New York using Google Street View images. Amnesty also only looked at Manhattan, Bronx, and Brooklyn (where 2/3rds of New York City residents live according to US census data), but still found 50% more than the Stanford team did in all of New York City.

Amnesty International Methodology

Amnesty used hand counting and did not find signs of overcounting, but some consumer cameras were counted. Amnesty had 3 volunteers count the same Street View image to avoid an error caused by one volunteer miscounting the image. Amnesty told IPVM they did not find signs of overcounting in a post-study analysis done by "experts" and using statistical analysis. Some consumer cameras were counted, but Amnesty said the "other" category where they fit was "rarely" used.

Study Authors says NYPD has 80%+ More Cameras than they Estimate

The study noted that the New York City Police Department has access to 18,000 cameras according to a 2018 Daily News Article, 80+% more than the 10,000 they estimated in all of New York City, suggesting that the limitations in this study led to a significant undercounting:

We note that our estimates exclude indoor cameras, as well as outdoor cameras not captured by street view images. Perhaps due to these limitations, our estimate of 10,100 cameras in New York City is lower than the 18,000 cameras that the NYPD reportedly has access to. [They cite a 2018 Daily News Article]

Claim US Camera Counts Consistent Over Time

The researchers concluded that the number of cameras in the US cities had not changed “substantially” in the two periods they analyzed, (2011– 2015) and (2016–2020) which IPVM does not beleive [they did not look at change over time in non-US cities]:

our camera estimates do not vary substantially between the two time periods we consider (2011– 2015 and 2016–2020), suggesting that the installation of surveillance cameras in these cities may have reached a plateau.

The researchers used the newest Google Street View image of a selected street between 2011 and 2015 and the newest between 2016 and 2020 to estimate the change in camera count over time.

IPVM Image

Cameras Relatively Small, Easy to Miss

Surveillance cameras are small and can be hard to see from Google Street View images making counting them challenging. The researchers said images were ~7 ppf and estimated that the cameras were 30-50 square pixels. As can be seen in this example Google Street View Image from New York City, cameras are hard to see and distinguish from pipes, lights, alarms, etc.

IPVM Image

Algorithm Trained on Partial Data

The algorithm was trained using partial data potentially leading to undercounting. The researchers used 2019 data EFF obtained from the City of San Fransisco to build their training dataset. It excludes at least 500/3,253 cameras (16%+) that the city did not share with EFF along with any cameras not recorded by the city. The researchers said that they only found 977 of the 2,753 cameras noting most appeared to be indoors or hidden. This means the algorithm was likely trained on images with unlabeled cameras, leading to missed detections.

Same Data Used to Calculate Missed Detection Rate

The same data was used when calculating the missed detection rate, meaning this was likely underestimated as well. Every unlabeled but missed camera would not be compensated for in their final estimation, causing them to underestimate the total number of cameras in each city.

Only Looked at External, Non-Consumer Cameras

The study did not look at residential cameras (like Wyze or Nest) or cameras that were not visible from the Street View images. Consumer cameras make up a substantial percentage of cameras and the Washington Post reports law enforcement can access video from these cameras in some cases. Also, many cameras may have been obscured or inside, causing undercounting. This limits the usefulness in determining how surveilled residents of the investigated cities are.

Minority Neighborhoods Face More Surveillance

The researchers found that in the US, areas with a higher percentage of minority residents had a higher density of cameras than other areas, even after controlling for land use [they only analyzed the US cities]:

also in areas with higher shares of non-white residents. This concentration of cameras in majority-minority neighborhoods persists even after adjusting for zone, pointing to the potential disparate impacts of surveillance technology on communities of color

The researchers defined "minority" as someone who was Hispanic and/or not white, using data from the 2018 American Community Survey.

IPVM Image

Industrial, Commercial Zones Have More Cameras than Public, Residential

The study found analyzed US land use and zoning information, finding that public and residential areas had lower estimated camera densities than other areas:

We find that images from mixed, industrial, and commercial zones are more likely to contain an identified camera than images from public (such as parks and other public facilities) and residential areas. For example, the identification rate in mixed zones (2.1%) is more than three times the rate in residential zones (0.6%). This pattern holds for the majority of our chosen cities.

They told IPVM that the order of density by neighborhood type was consistent city to city although there were slight variations in the exact values.

IPVM Image

Relative Density of Cameras Resilient to Undercounting, Limitations of Study

The relative density of cameras by neighborhood demographic or land use should be more resilient to undercounting and flaws in their methodology than other estimates. Both numbers compare the estimated number of cameras in one area of a city to another area of the same city, so as long as undercounting is consistent it should not impact the study.

Outlook

The study ignores a large portion of cameras and has estimates far lower than Amnesty International and the NYPD reported to the Daily News, making the results questionable. However, their findings that neighborhoods with a higher percentage of minority residents and those with industrial or commercial zoning have a higher camera density than other neighborhoods should be more resilient to flaws.

Comments (6) : Subscribers only. Login. or Join.
Loading Related Reports