Google Clips Camera Tested - Weak AI

By: Rob Kilpatrick, Published on Mar 12, 2018

The Google Clips is drawing a lot of interest, especially given its use of artificial intelligence (AI) to deliver a 'smart' camera. Indeed, Google claims:

We purchased one and tested these claims. Inside, we report our findings on:

  • Camera pricing
  • App usability
  • Inconsistent suggested clips
  • Face/pet/expression recognition issues
  • Camera hardware

Most notably, we examine how the camera was not very 'smart' at all.

Summary

However, in our tests, we saw several issues contradicting this claim:

  • Clips were generated by practically any sort of motion (people, pets, etc.) or even in still scenes (empty rooms, walls, etc.).
  • Further, while it claims to suggest clips that users will like most, these suggestions had seemingly no rhyme or reason, with nearly identical clips suggested and not suggested directly after each other.
  • Finally, while Google also claims that expression recognition is used to determine which clips to capture, we saw no difference in the number of captures regardless of expression (smiling, laughing, neutral, etc.).

Despite these issues, many users may still find the camera useful, as it may be placed in a stationary location during events or when playing with children or pets, automatically taking photos without them needing to use their phones.

Note: Claims To Improve Over Time

Google claims that Clips improve over time, with the camera learning what is important to users. We saw no difference in performance during our testing, ~5 days. However, it may improve with additional time and use.

Get Notified of Video Surveillance Breaking News
Get Notified of Video Surveillance Breaking News

Pricing

The Google Clips camera can be purchased from the Google Store for $249 USD, or as a bundle with a tripod mount case for an additional $14.99.

Physical overview

The video below provides a physical overview of the google clips camera and included case:

Clips Mobile App Overview

The Clips mobile app is essentially the only UI to the camera. The main interface of the app is a list of captured clips, which users scroll through to review, swipe to save/delete, etc., and so-called "Suggested Clips" are highlighted with a star icon, though we found these suggestions poor (discussed below). The camera may also be viewed live for positioning and switched from video to gif mode, etc.

We review these functions in this video:

Failure: Clips Generated On Anything/Nothing

According to Google, clips should be created when the camera sees human faces and pets. However, during our tests, the camera created several clips which contained none of these, including an empty room, object waved in front of the camera, people without faces visible, etc.

Because this feature worked so inconsistently, the Clips camera functions more like a camera recording on motion might, recording when it perceives changes in the scene.

For example, while we were writing this report, the camera captured six images of the side of a file cabinet:

Note that there is no way to verify what the camera "saw" to trigger a clip, nor is there any clear sensitivity adjustment.

Inconsistent Suggested Clips

Google states that the camera suggests specific clips based on recognizing familiar faces or pets, which it then highlights with a star icon in the top right corner of the clip. However, in our tests, this was incredibly inconsistent. The app suggested clips, but others immediately before and after those suggested were not tagged, despite containing the same subject(s) and scene.

For example, the image below shows two clips taken back to back, with one recommended and one not, but the content nearly identical.

The same was true of clips with pets. The clip below left was recommended, but the very similar clip on the right was not.

Failed Expression Recognition

Google claims that facial expressions are one criteria which it uses to capture clips. However, in our tests, clips were generated regardless of expression. Users smiling, laughing, frowning, etc., were just as likely to trigger a clip as those with neutral expressions.

Camera Teardown

In the video below we review components of the Clips camera. The camera's AI is powered by a Movidius MA2150 "Vision Processing Unit" (VPU), visible on the front of the board (see Intel Movidius Targets Video Surveillance Market). The only other chip with visible markings is its 16GB solid-state storage, on the rear of the board.

Note that in order to tear down the camera, it must be broken, with no way to remove and replace the lens or front cover and delicate ribbon cables connecting components. The teardown was conducted after performance testing.

Test Conditions

Note that our tests were performed in areas with few subjects present, with activity ranging from sporadic to constant. If the camera were used in an area with a crowd of people, such as a party, performance may differ (known people suggested instead of unknown, fewer or more clips shown, etc.), though given how much it struggled with simple scenes, we are skeptical of how it would perform in harder ones.

Firmware Versions Used

The following firmware versions were used during testing:

  • App version: 1.3.185005366
  • Camera Version: 1.3.5.1185005431

1 report cite this report:

Worst Products Tested In Past Year on Jan 09, 2019
IPVM has done over 100 tests in the past year. But which products performed the worst? Which ones should users be most aware of? In this report,...

Comments (5)

Only IPVM Members may comment. Login or Join.

As an owner of the Google Pixel 2 XL, I can assure you that the AI will improve noticeably over development cycles (which is pretty quick).

Promise?

Cross his heart and hope to lose market share.

Interesting, lets see whether it goes towards another flagship Google product and revolutionize entire CCTV/camera industry;  ... ... or just gets killed by Google itself. 

Related Reports

Bezos-Funded Deep Sentinel Tested on Mar 28, 2019
Backed by Jeff Bezos, the Silicon Valley startup, Deep Sentinel, has declared: No One Does Home Security Like We Do Our Surveillance Team has...
Bosch AI Camera Trainer Released And Tested on Apr 09, 2019
Bosch is releasing a highly unusual new AI feature - 'Camera Trainer'. Now, coming as a standard feature in Bosch IVA/EVA analytics, one can train...
The HIVIDEO $31 Face Detection DVR Tested on Apr 25, 2019
Face detection in a $31 DVR? That is what "HIVIDEO" (not to be confused with Hikvision, even if the company intends to do that) was promoting at...
Anyvision Facial Recognition Tested on Aug 21, 2019
Anyvision is aiming for $1 billion in revenue by 2022, backed by $74 million in funding. But does their performance live up to the hype they have...
Verkada People And Face Analytics Tested on Aug 16, 2019
This week, Verkada released "People Analytics", including face analytics that they describe is a "game-changing feature" that "pushes the...
Camect "Worlds Smartest Camera Hub" Tested on Oct 18, 2019
Camect is a Silicon Valley startup that claims the "Smartest AI Object Detection On The Market", detecting not only people and vehicles, but...
Avigilon Appearance Search Tested on Oct 30, 2019
Avigilon Appearance Search claims that it "sorts through hours of video with ease, to quickly locate a specific person or vehicle of interest...
Rhombus Cameras, VMS and Analytics Tested on Nov 06, 2019
Rhombus boasts they have created "the new standard in Enterprise, cloud-managed video security" and told IPVM in January 2019 they offer twice the...
BriefCam Video Analytics Tested on Jan 06, 2020
BriefCam, acquired by Canon in 2018, is one of the most commonly used video analytics offerings in the West. But how well does it work? We...
Bosch / Milestone Forensic Search Tested on Jan 08, 2020
Bosch's Forensic Search Milestone plugin integrates Bosch IVA and EVA analytics search in Milestone XProtect, claiming to "gain complete control on...

Most Recent Industry Reports

IronYun AI Analytics Tested on Feb 17, 2020
Taiwan startup IronYun has raised tens of millions for its "mission to be the leading Artificial Intelligence, big data video software as a service...
Access Control ADA and Disability Laws Tutorial on Feb 17, 2020
Safe access control is paramount, especially for those with disabilities. Most countries have codes to mandate safe building access for those...
ISC West 2020 Removes China Pavilion, No Plans To Cancel Or Postpone on Feb 17, 2020
ISC West plans to go on next month, amidst concerns over coronavirus. However, the Asia / China Pavilion has been removed, show organizers...
Hanwha Wisenet X Plus PTRZ Tested on Feb 14, 2020
Hanwha has released their PTRZ camera, the Wisenet X Plus XNV-6081Z, claiming the "modular design allows for easy installation". We bought and...
IPVM Conference 2020 on Feb 13, 2020
IPVM is excited to announce our 2020 conference. This is the first and only industry event that will be 100% sponsor-free. Like IPVM online, the...
Bosch Dropping Dahua on Feb 13, 2020
Bosch has confirmed to IPVM that it is in the process of dropping Dahua, over the next year, as both IP camera contract manufacturer and recorder...
BluB0X Alleges Lenel, S2, Software House Are Dinosaurs on Feb 13, 2020
BluB0X is running an ad campaign labeling Lenel, S2, Software House, Honeywell, AMAG and more as dinosaurs: In a follow-up email to IPVM,...
London Live Police Face Recognition Visited on Feb 13, 2020
London police have officially begun using live facial recognition in select areas of the UK capital, sparking significant controversy. IPVM...
Converged vs Dedicated Networks For Surveillance Tutorial on Feb 12, 2020
Use the existing network or deploy a new one? This is a critical choice in designing video surveillance systems. Though 'convergence' was a big...