Camio Natural Language Processing Tested

By Ethan Ace, Published Sep 27, 2016, 12:00pm EDT (Research)

The ex-Googler led team from Camio has advanced its video monitoring offering to include natural language processing.

Camio ingests video, automatically determines what objects are, labels those objects and then allows users to search for them with words, like Google but for surveillance video.

Here is an example of the type of labeling they claim to do:

#MachineLearning vocab lessons. @camio labeled this "deer" but also "even-toed ungulate" via @JeffBierBDTI pic.twitter.com/goPOoHzSoa
— Carter Maslan (@CarterMaslan) August 27, 2016

We tested Camio's performance connected to an Axis camera and a Nest cam using a Camio Box we bought.

Summary

Camio's search / classification capabilities are far beyond anything we have ever seen in video surveillance. This does not mean it is the 'best' (as searching can be done in many ways including PoS data, by license plate, by face matching, by person similarity, etc. which Camio does not do). However, its Google like text search input and its fairly detailed labeling of people, colors, time, vehicle types and relationships between them lets it do things not possible even in enterprise VMSes.

However, Camio's targeting of home users is unfortunate, since the average home users is unlikely to get the full benefits and less likely to pay the $19.90 per month per camera for these advanced analytics. By contrast, we could see Camio's technology be highly attractive / an important differentiator for an enterprise video surveillance provider.

Key Findings

Here are our key findings from this test:

Overall, object labeling worked well in our tests, though with some instances of mislabeling and some objects not labeled at all.
Vehicles were labeled in most detail, with specifics such as car, compact car, sport utility vehicle, truck, minivan, etc. These categories are generally accurate, though some overlap or misclassification occurs, with more specific labels less accurate.
Camio attempts to tag some objects carried by human subjects, such as bags or "weapons", but these labels were inaccurate in our tests.
Some common objects, such as lawnmowers or boxes, were not included in the labels which Camio had learned. Camio says these may be learned as more examples become available.
Some labels were not practically useful, such as many outdoor clips being labeled "grass" or "lawn", and indoor shots containing "ceiling." These objects appear in the clip, but are not likely to be useful in day to day use.

The original functions of Camio (vehicles, people, color, direction) are essentially unchanged, reviewed in our original test and can be combined with the new more advanced functionalities described above.

Pricing

Camio's Pro plan, required for advanced analytics, is $19.90 USD per month per camera. Their basic cloud plan, which does not advanced analytics/labeling, only basic events/search, is $9.90 per month.

Camio does not offer any free plans but live streaming of iOS and Android device cameras, with no history nor search, is available.

Camio Natural Language Processing Overview

Camio's analytics process video using natural language processing, which applies labels to objects in the video for alerting and search, in much more detail than typical analytics. This labeling is automated, performed completely by machine, not human operators.

For example, where many analytics recognize a vehicle entering a given area, Camio applies more detailed labels, such as "sport utility vehicle", "blue", "approaching", etc., not only on the moving object, but also to background objects in the scene, such as grass, trees, or asphalt. These labels may be searched later, or alerts on specific objects sent.

This video reviews Camio's labeling and how it may be searched/used:

People Labeling

Camio correctly labeled people in all scenes tested, day and night, indoor and outdoor. We saw no instance of non-human objects being labeled as people.

Vehicle Types Differentiated

Instead of simply the generic term "vehicles", users may search for specific types of car, such as "sport utility vehicle" or "minivan" or "sports car." More specific categories, such as "compact sport utility vehicle" or "full size car" are also possible.

In our tests, these searches were generally accurate (shown below), though Camio errs on the side of caution, and sometimes identifies an SUV as a minivan, a truck as an SUV, etc. We found that the more specific categories, such as "compact car" or "full size car" or "compact sport utility vehicle" were less accurate than the more general terms.

Object Modifiers

In addition to labeling objects, Camio allows users to search by color and direction (approaching/departing). So users may search for "cars approaching" or "people wearing blue" (below).

Note that approaching/departing apply to objects' motion in relation to the camera, not the objects motion relative to each other. So "person approaching desk" currently returns scenes in which a person approaches the camera and a desk is present, not people specifically approaching a desk.

Camio says they intend to improve how these modifiers operate in conjunction with each other to make searchers more intuitive, and expand the list of modifiers over time.

Mislabeling Smaller Objects

In our tests, Camio labeled objects carried by human subjects only a handful of times, none of which were accurate.

For example, the subject carries a notebook/folio in the image below, which was labeled as "bag" by Camio. Though similar in its size/shape and the way it is carried, not accurate.

Camio also attempted to apply the general label "weapon" to a subject carrying a water glass. Camio says that this is likely due to the size/shape of the object and the subject's posture, making it look similar to a knife.

Objects Not Labeled

In several instances, common objects were not labeled. For example, during our tests, there were clear clips of landscapers mowing grass, both on a riding mower and walk behind. Camio labeled these both as "vehicles", but with no more specific data, i.e., a "lawnmower" tag.

Some Labels Not Practical

Finally, some of the labels applied by Camio were simply not practically useful in many cases. For instance, "lawn" was one of the available labels, but searching it simply shows clips in which lawn was visible. Note that this is not a specific zone labeled lawn, but automatically tagged.

In our indoor camera, many clips were labeled "ceiling", though essentially every clip has visible ceiling in it, as the camera is in hallway/corridor mode in a relatively small area.

Camio Box

Camio previously required complex setup to connect cameras to their service, typically by uploading video clips or snapshots upon event, with no access to live video.

However, they have simplified this process with the "Camio Box", a small form factor device which scans the local network for cameras and streams video to the cloud without manual setup. There are two models of Box available: