Beware Amazon Go Store Hype (Tested)

By: John Honovich, Published on Nov 13, 2018

IPVM's trip to and testing of Amazon Go's San Francisco store shows a number of significant operational and economic issues that undermine the hype.

amazon go deceptive

While the press has overwhelmingly fawned at Amazon Go's store and the report that Amazon will deploy 3,000 of these stores in the next few years, our review of the operation points to significant challenges in turning this into a viable enterprise.

In this note, we examine:

  • The time delay and behind the scenes human intervention involved
  • The density and cost of sensors used
  • Amazon mischarged IPVM during testing
  • Limitations in the system's capabilities
  • Video surveillance and access control used
  • Positives for Amazon and Go customers
  • Outlook on Amazon Go stores

Executive Summary

Given the massive amount of:

  • Infrastructure involved - hundreds of large sensors for a small convenience store
  • Computing and electricity consumed - undisclosed but clearly heavy amount of back-end processing for these hundreds of sensors
  • And most notably human operators reviewing transactions

IPVM is skeptical that Amazon is anywhere near close to being able to run Go stores as a profitable business. Now, given Amazon's scale and ability to absorb losses, this may not be an issue to them but it does undermine the hype that this is economically viable for real businesses.

Density And Cost of Sensors

The most notable hardware differences for the stores are their uses of turnstiles to enter (which are easy to see) and their mass of hundreds of sensors overhead that most people ignore. This image from the entrance overviews the sensors laid out:

very high density of sensors - hundreds per small store

Get Video Surveillance News In Your Inbox
Get Video Surveillance News In Your Inbox

And this image shows how many sensors are needed just to cover the ~10' long chips section:

numerous sensors even over small chips section

The following video shows the sensors across half the back side of the store:

Sensors are square shaped, ~1' per side, with what appears to be 2 imagers:

close up of amazon go sensor

These sensors are put right above every shelf as well as throughout the aisles. Based on limited Amazon explanations, they read / detect what objects each user is picking up.

Most Important - Receipt Delay and Human Review

The most surprising and under-reported element is that receipts take 5 - 10 minutes normally to be generated. By contrast, Amazon's own marketing video shows it to be done instantaneously. And CNET who just a few weeks ago went to the same store we did, hides that as well, as the video excerpt below shows:

CNET edits it to make it look instant but the timestamps from their own video shows it was captured 25 minutes later:

Indeed, long delays for receipts are normal, according to 3 different Amazon Go employees who spoke to us. They even explained that they have people reviewing transactions though they either did not know or were not authorized to explain how often they did so.

However, 5+ minutes is an eternity for computers whose timescale is the milliseconds or less to process information. Amazon is not letting on to how much humans need to be involved but such a delay points to serious concern for errors and the need for human intervention.

Mischarge / Error

Amazon charged us for a bottle of Teriyaki sauce instead of a bottle of water. In fairness to Amazon, this was done on purpose to test the system but even though they took 3 extra hours to have humans analyze this, they still made a mistake:

Here is what happened:

  • Take water bottle from back shelf
  • Walk to front shelf, take Teriyaki sauce off shelf
  • Put down water bottle in the Teriyaki sauce column
  • Go back to back shelf, put Teriyaki sauce in water bottle column
  • Return to front shelf, grab water from Teriyaki sauce column, put in pocket, leave store

Our hypothesis is that the system thought we left with a teriyaki sauce bottle since that is the area where we picked it up last and the system could not differentiate between the two, even ostensibly after hours of human intervention.

We certainly do not want to overdo this as some sort of Achille's heel but it does point to limitations in the intelligence of the system and the cost of humans to review transactions.

Limitation - Cannot Detect Bags

One other peculiar limitation of the system is that it cannot detect bags, requiring users to manually add bags in the app as the display captured below shows:

sensors cannot detect bags

Oddity - Strong Glare Lighting

While people did not seem to mind (or at least call it out), the lighting in the store was harsh with significant glaring, as the video below shows:

Since the strong lightning was centered over aisles and next to sensors, we suspect this was done to help the sensors better detect objects that would be undermined by normal lightning levels.

Limited Cameras And Access Control In the Store

Outside of the literally massive array of sensors, conventional security cameras and access control were limited. Only a few unbranded surveillance cameras were visible in the public area of the store, this one below from the entrance area:

few conventional surveillance cameras in store

Another element was access control credentials with employees swiping RF IDeas pcProx cards:

employees use pcprox cards

Positives For Amazon Go Store And Customers

Customers were overwhelmingly happy in our trips to the store. While there was some confusion installing the Amazon Go app (separate from the Amazon app) and the delay in getting a receipt, the store had a variety of products at low cost and with generous refund terms.

Amazon Go employees cheerfully encouraged visitors to refund any product bought. Indeed, the 5+ million viewed YouTube video We Stole Tampons from the Cashier-less Amazon Go Store highlights this (spoiler: they did not steal anything, they simply pointed out Amazon's DIY refund / no returns needed policy).

Moreover, many customers remarked to the Amazon Go greeter about how low the prices were, comparing it to the CVS down the block.

Losing Money

These stores are clearly losing money. While that in and of itself is no surprise for Amazon, we believe the losses are significant and fundamental. The need for an employee anyway, the high cost of the technology installed, the ongoing processing costs (computing and electricity) and the undisclosed but ongoing human transaction monitors combine for a significantly more expensive store than conventional ones.

Outlook

While technology certain improves and Amazon has a well documented long-term outlook plus the ability to absorb losses for years, we are skeptical that cashier-less stores will be financially viable for many years to come. The need for massive numbers of sensors / cameras is unlikely to diminish as they cannot see through people and will need to be spread out all over and on top of shelves to overcome that. Add the difficulty of ensuring accurate transactions means that expert human operators (far more expensive than cashiers) will need to be involved.

It is completely sensible for Amazon to have a limited number of stores as effectively a public R&D experiment (with the bonus of massive marketing gains) but beware the viability of such stores, on real economic terms, for years to come.

Vote / Poll

Comments (34) : PRO Members only. Login. or Join.

Related Reports

Sighthound Transforms Into Enterprise AI Provider on Jun 14, 2019
Sighthound is now rapidly expanding its R&D team, building an enterprise AI service. This may come as a surprise given their origins 6 years...
Farpointe Data Conekt Mobile Access Reader Tested on Jun 13, 2019
California based Farpointe Data has been a significant OEM supplier of conventional access readers for years to companies including DMP, RS2, DSX,...
Embattled $400 Million China Funded Philippines Surveillance System Proceeds on Jun 13, 2019
An embattled 12,000 camera surveillance system project that will cost ~$400 million will proceed.  The project contract was awarded, had its...
Carnegie Mellon AI Startup Zensors Profile on Jun 11, 2019
Zensors is a startup formed by Carnegie Mellon graduates from a Carnegie Mellon research project, offering customized models per camera that they...
Dumber Techs, Bad Box Movers, Says Australian Distributor on Jun 10, 2019
Techs today are "dumber" than they used to be, despite better education and training and that makes a typical day "frustrating" for one...
Directory of 30+ VSaaS / Cloud Video Surveillance Providers on Jun 07, 2019
This directory provides a list of VSaaS / cloud video surveillance providers to help you see and research what options are available. 2019 State...
Hikvision DeepinMind 2019 Test on Jun 06, 2019
In 2018, Hikvision's DeepinMind AI NVR performed terribly, recognizing vehicles, animals, and other objects as humans, misclassifying demographics,...
Startup Rhombus Systems Says Twice the Features, Half the Price of Verkada on Jun 04, 2019
Closed cloud systems may be the fastest growing segment of video surveillance with Meraki and Verkada. Now another California company is joining...
OSDP Access Control Guide on Jun 04, 2019
Access control readers and controllers need to communicate. While Wiegand has been the de facto standard for decades, OSDP aims to solve major...
Vidsys New President Interviewed on May 31, 2019
A decade ago, PSIM was hot with projections then of a billion dollar market by now. This has not come close to happening. However, Vidsys, one of...

Most Recent Industry Reports

Sighthound Transforms Into Enterprise AI Provider on Jun 14, 2019
Sighthound is now rapidly expanding its R&D team, building an enterprise AI service. This may come as a surprise given their origins 6 years...
ADT Eliminating Acquired Brands, Unifying Under 'Commercial' Brand on Jun 14, 2019
ADT is eliminating the brands of the many integrators it has acquired over the past few years, including Red Hawk, Aronson Security Group (ASG),...
NSA Director Keynoting Dahua and Hikvision Sponsored Cybersecurity Conference on Jun 13, 2019
The technical director for the NSA’s Cybersecurity Threat Operations Center will be keynoting a physical security cybersecurity conference that is...
Farpointe Data Conekt Mobile Access Reader Tested on Jun 13, 2019
California based Farpointe Data has been a significant OEM supplier of conventional access readers for years to companies including DMP, RS2, DSX,...
Embattled $400 Million China Funded Philippines Surveillance System Proceeds on Jun 13, 2019
An embattled 12,000 camera surveillance system project that will cost ~$400 million will proceed.  The project contract was awarded, had its...
False Verkada 'Unrivaled' Low Light Performance Claim Removed on Jun 12, 2019
Verkada falsely claimed that it delivered 'UNRIVALED LOW LIGHT PERFORMANCE' until IPVM questioned. In fact, Verkada's low light performance is...
Manufacturer Favorability Guide 2019 on Jun 12, 2019
The 259 page PDF guide may be downloaded inside by all IPVM members. It includes our manufacturer favorability rankings and individual...
Camera Course Summer 2019 - Register Now on Jun 12, 2019
Register for the Summer 2019 Camera Course.  This is the only independent surveillance camera course, based on in-depth product and technology...
Favorite Wireless Manufacturers 2019 on Jun 12, 2019
Many wireless options exist for video surveillance but how are integrator's overall favorites? 170 integrators answered the question: What is...
Carnegie Mellon AI Startup Zensors Profile on Jun 11, 2019
Zensors is a startup formed by Carnegie Mellon graduates from a Carnegie Mellon research project, offering customized models per camera that they...

The world's leading video surveillance information source, IPVM provides the best reporting, testing and training for 10,000+ members globally. Dedicated to independent and objective information, we uniquely refuse any and all advertisements, sponsorship and consulting from manufacturers.

About | FAQ | Contact