Wyze AI Analytics Tested - Beats Axis and Hikvision, Now Improved, Now Being Terminated

By Ethan Ace and Rob Kilpatrick, Published Nov 25, 2019, 01:31pm EST

When we originally tested Wyze's free person detection deep learning analytics in July 2019, they performed well, with few false alarms, but a long 5-minute cooldown period which caused missed alerts.

[UPDATE: days after releasing this updated test, XNOR.ai has terminated their agreement with Wyze and XNOR.ai's analytics will be removed from Wyze firmware in early 2020.]

Now, they have released a new paid Complete Motion Capture plan which eliminates the cooldown.

IPVM Image

To see how Complete Motion Capture impacts performance, we tested multiple Wyze Cams examining:

  • How does this new plan improve person detection?
  • How well does person detection work? Does it miss people day or night?
  • How does walking/running speed impact detection?
  • Do nuisance sources like lights, shadows, foliage, and animals trigger person detection?
  • How difficult is analytic setup?
  • How does performance compare to Nest Cam IQ?
  • How does performance compare to Hikvision Intrusion analytics?

*******

****'* ****** ********* ********* performed **** **** *** course ** ***** ** testing, ******** **** ***** person ****** **** ******* which ******** ********* **** commercial *******, **** ** shadows, ***** *******, ******* foliage, *** *******.

**** *********** *** ** par **** *** **** more ********* **** *** Outdoor ** *** ****** than *********'* ********* *********, which ******* ** *******, shadows, *** *******.

*******, ***** **** ***** key ********* ***** ** our *******:

  • ****** ********* ******* ** 12 ******* *** *****:****** ********* ** ******* to *** ***** ** seconds ** **** ****** clip, **** **** ***** their ************ ****** ***********. **** ***** ****** in ****** ***** ****** in ***** **** **** amounts ** ******, ** people **** *** ** detected **** ****** ** of ** ***** ***** motion ***** *** ********.
  • * ****** ******** ****** without **** ****:** *******, ****'* ********* have * **** * minute ******** ****** ******* Person ********* ****** ***** could ****** ** ****** activity ** **** ******* areas. **** *** ********** a **** **** ***** eliminates **** ******** (******** Motion *******, $*.**/*****).
  • ****** ******* ********:*** **** *** ****** to ****** ******** ******* through *** *****, ********** of ******** ** ******** conditions. ** ******** ******* at ****** ** **** speeds **** ******.

** ******* ** ********* results ** **** *****:

Versus ***** ******** / *** *********: ****** ********* *** ** ******* ******

****'* ****** ********* ********* similarly ** ** ****** than ***** ********/*** ********* in *********** ** ****** ********* Shootout, ********** ******* ** Wyze's **** ******** **** completely ** *** ****, without ***** ************, ******** by ******.

****,****, *******, *** *** ***** on ******* ** *******, either, ***** *********'************ ******* ** ****** all ****** ** *** scene. **** ****, ***** other ********* **** ****** running ******** (****** ***** did ***).

*******, **** ** **** less ********* **** ***** other *******, **** ****** ranging ** ***** **** ~$110 (*****) ** $*** (Nest *** *******). ************, Wyze **** *** ******* a ************ *** ***** analytics, ***** ****, ****, and **** ** ($*-*/***** per ******), ****** **** do ******* * ***** cost $*.**/***** *** ****** for ********** ****** *******.

*** ******* ******** ** Wyze ******** ** ***** other ******* ** *** lack ** ******* ****** models, ***** ***** ************* offer. **** **** ***** outdoor ***** ** ******* for **** ****/***** ****.

************

**** ****** ********* ** available ** ******* *** ($** ***)******* *** ********** *** ****. ***** simply **** ** ******** the ****** ******** *** turn ** ****** *********.

Person ********* ***********

**** *** *** *** limitations ** **** ******** not ****** ** ***** consumer ** ********** ****** we **** ****** (****** present ** **'***** ****** * **** AI *********):

  • ****** ********* ******* ** 12 ******* *** *****
  • * ****** ******** ******* paid ****

** ******* ***** ** more ****** *****.

People ********* ******* ** ***** ** ******* ** *****

**** ****** ********* ***** only ** *** ***** 12 ******* ** **** motion ****. **** ***** that ** * ****** enters * ***** ***** motion ** ******* ***** 12 *******, **** **** not ** ********, *** will * ****** ****** trigger ** ********** ***** if *** ** ******* in *** *****.

**** ** ***** ***** this ********** ** ***** documentation *** ***** * notice ** ****** ********* clips ** ***** ****** app:

IPVM Image

**** *** ** ** issue ** ***** ***** constant ****** ** *******, such ** * ****** facing * ****** **** blowing *******, ** ****** detection *** ****** *** trigger. **, ** * subject ****** *** *** but **** *** ******* person ********* ****** ** seconds, ** ****** **** not *******, ***** ***** (green ******** *** ********* motion, *** ****** *********):

Five ****** ******** ****** ****** ****** ** *******

***** * ****** ********* event ** *********, ** default **** *** ** unconfigurable * ****** ******** period ****** ******* ****** detection ***** ** ****. This ***** ****** ***** missed ****** ********* ** high ******* ***** **** as ******* ** *******.

**** **** **** ** a **** ***** **********, not ** **/******** **********. We ********* **** ********** with **** **** *******, as ****.

IPVM Image

*** ****** ****** ******, this ******** **** *** exist, **** ****** **** closer (***** * ****** apart *****):

IPVM Image

No ******** **** **** ****

** ****** **** ********, Wyze ********** * **** plan ************** ****** ************ ********** *** * minute ******** ****** ******* events. ** *** *****, this ****** ** *******, with ********** ****** ********* events ******** ******* ***** seconds *****.

IPVM Image

**** **** ***** *** not ** ******** ******, but ****** ** **** 5 ****** ********, *.*., if ****** ***** * minutes, *** ***** *** created. *******, **** **** person ********* **** **** use *** ***** ** seconds ** **** ****, regardless ** ******.

******** ****** ******* ***** $1.50 *** *** ***** per ******.

Person ********* ***********

****'* ** ****** ********* performed **** ** *** tests, ********* ******** **** day *** ***** (**** IR **) **** ** missed ********** ** ******** walking ** ****** ** fast *****.

People-Detected-Day-And-Night-Walking-Normal_Fast-Speed

**** **** *** *** alert ** ****** ******* of ***** ****** ***** triggered **** *******, **** as ******* ** *** scene:

No False Alerts From Shadows

** ****** ******* ** and ***:

No False Alerts From Light Changes

************, ****** **** ****** are ****** ****, ** also ****** ***** ** analytics ********. ********, ****** Detection ******* ******* ******* and ******* ***** ********* other ****** ** **** tests. **** **** ******** boxes ******** ****** ****, not ****** *********.

No-False-Alerts-On-Foliage

***** *** ***** ******* were **** ******* ** Person *********.

No False Alerts On Animals

Running ******** ******

*** **** ****** ** detect ******* ******** ** our *****, ***** *****. However, ***** ***** ****** market (*********** ******), **** is **** ****** ** be * ******* **** commercial ******* ** **** residential ******* *******.

Running-Subjects-Completely-Missed

Simple *****: ****** ******** *** **** **

****** ********* ***** ** simple, ********* ***** **** to ****** ******** *** turn *** ******* ** on **** ******. ** setup *** ****, *******/******* sizes, ** ***** ************* is ********.

Simple-Setup--Switch-On-For-Each-Camera-copy

Xnor: *** ******* ******** **** **

****'* ****** ********* ********* were ********* ** *********** with****, * *******-***** *******. Xnor's ***** ** ** optimizing ****** ** *** on *** **** ******** hardware, **** ** ********** ******* ** *** **** Cam *****:

IPVM Image

**** ** *********** ****** as ****** *****-**** ***** perform **** ********, ***** historically *** *** **** capable. **** ** ** alternative ** ***** **** expensive **** ******** ***** such *******'* ******** ******.

****'* ****** *** ********** on *** ****, ******* analyzing ** *** *****. Wyze *********** **** ** Nest ** **** ******* cloud ************ (*** *************) to ****** ***** **** Aware/Arlo ***** *********. *********** **** ****'* ********* video********* ****:

**** *** ******* ** 2016 *** *** ****** nearly$*** *** ** ***. *** **** *******, see *** **** ******:******* ****.**, ********* ** YOLO, ***** ************ ***** Examined

Vs ********* ********* *** **** ******* ** ****** *********

**** ****** ********* ********* similarly ** **** ****** detection, *** ***-********** ******** in*** ******** ********* ********, **** **** ******** false ****** **** *******, lights, *******, *******, ***., though **** ****** ******* subjects.

**** ****** ********* ************ Hikvision's ********* *********, ** Hikvision ******* ** ******* such ** ******* ******* and ****** *******.

Hikvision False Alerts On Foliage Shadows

** **** ** ***** and ***** ******* ** the **********.

Hikvision False Alerts On Animals

*******, *********'* ********* **** alerted ** *** ********** of ******* ****, ***** neither *** **** *** Nest ******* ******* **.

Hikvision False Alerts On Headlights

Vs. **** *********

****'* ****** ********* *** not ******* ** **** sources ***** ****** ***** alerts ** **** ******* analytics, ********' ****,****** *****, ************ ************ *** ********* ** shadows, *******, ** ******* in *** *****.

**** ****** ** ******** object ********* ** ***** new******-* ****, ****** *********** ** the ***** ****** ***** this *** ****,*** *****-**** **** ******* ** slightly ****** **** **** tests.

Versions ****

*** ********* ******** **** used ****** *******:

  • **** *** **: *.*.*.**
  • **** ** *******: *******
  • ********* **-**********-*: **.*.** ***** 180718

Comments (40)

I have one of these with the analytics installed looking out my front window at the sidewalk coming up to my porch. For the most part it has worked fairly well considering this is very low traffic, even detecting a person when walking out from the house to the driveway. It has missed some though. Overall for the price point these are good little cameras. I have the new sensors and light bulbs as well but haven't done much with these yet as I had hoped. For some reason my daughter wanted to get married two weekends ago so my spare time has been involved in that. 😊

Agree
Disagree
Informative: 5
Unhelpful
Funny: 2

This is extremely good for the industry, as this will push incumbents to get real deep learning released.

However, it is seriously embarrassing for all the big companies that still can’t do this. For example, Hikvision and all its “big data”, “strongest R&D”, 16,000 “engineers”, claims.  Axis too, while they do not say much about AI, they really should if they want to justify their price premium.

It is wild that person detection has taken this long to become normal for video surveillance but this is a good sign we are now quite close.

Agree: 27
Disagree
Informative
Unhelpful
Funny

You saved me from typing the same message. Spot on.

Agree: 1
Disagree
Informative
Unhelpful
Funny

Add the Wyze Smart Bulb to the test. :D

Agree: 1
Disagree
Informative: 2
Unhelpful
Funny

Love love love my Wyze cameras.  I have not had any issues with the alerts looking out my front window at the sidewalk and get my notifications much faster than the Ring I have covering the same area.  

Agree: 2
Disagree
Informative: 3
Unhelpful
Funny

Note: I changed the title to "Wyze AI Analytics Tested - Beat Axis and Hikvision"

The reason is to emphasize its relationship to professional video surveillance. I can see by reads and engagements that Wyze is not of much interest (not just this report, the various reports we have done on Wyze).

However, what Wyze has done here is very significant for professional video surveillance. Wyze is going to sell a million or two million cameras this year and more 'regular' people are going to expect people detection as standard (even if they don't understand what 'deep learning' or YOLO or object classification is). And products like Axis and Hikvision are going to look foolish when Wyze is smarter than they are.

Those professional companies that catch up quickly should be fine but it could portend major pain for laggards in AI.

Agree: 9
Disagree
Informative: 1
Unhelpful
Funny

I have Wyze and it is impressive in relation to cost.  What drives me crazy is the 5 minute cool down for recording.  Perhaps I read the manual wrong, but I thought inserting an SD Card would have fixed that.

Also, the new Person Detection is good, except when a TV is in view of the camera.....

Agree
Disagree
Informative: 2
Unhelpful
Funny

UI2, when you insert the SD card you can go to advanced settings > local recording and enable continuous recording, the camera should then just continuously record to the SD card and you can search through recorded video.

in our tests motion events were recorded within 5 minutes of eachother but person detection still triggered a 5 minute cooldown.

Motion Events No Cooldown

Agree: 1
Disagree
Informative: 3
Unhelpful
Funny

Yes, I did all that.  For some reason I misread or hoped the SD card would allow you to continually record motion without 5 minute cooldown.

As for the person detection detecting TV people, I expect Wyze to allow you to do privacy masks hopefully soon.

Agree
Disagree
Informative: 1
Unhelpful
Funny

Seems that there will be an article on Anyvision's Facial recognition review soon.

 

Agree
Disagree
Informative
Unhelpful
Funny: 3

Wyze cameras are fun/cheap little things.  Not sure if it was covered here but they also have a RTSP firmware so you can pull it into about any VMS.  I have a few friends that I have convinced to go with Wyze and they are all happy.

Agree: 2
Disagree
Informative: 1
Unhelpful
Funny

We have not covered Wyze's RTSP support.

According to the Wyze Cam RTSP article, it is a separate firmware version that will be upgraded infrequently, so as a consumer / home technique, it could make sense but not that much for commercial applications.

Agree: 1
Disagree
Informative: 2
Unhelpful
Funny

Hmm... Wyze not...

Agree
Disagree
Informative
Unhelpful
Funny: 5

I have 6 of these at my house and the only thing that bugs me is the USB power cord, but that is a tiny complaint.  They work fine for general purpose stuff and making sure my dog is not eating the couch.  

Agree: 1
Disagree
Informative: 1
Unhelpful
Funny: 1

I am curious about the business model.  Does Xnor charge Wyze a per-unit royalty?  NRE?  Some sort of arrangement based on future business?  Is Wyze just eating the cost for now in order to grow interest in future ala carte, maybe subscription based, AI algorithms?

Agree
Disagree
Informative: 1
Unhelpful
Funny

XNOR declined to disclose pricing beyond confirming that it is a licensing / software based model.

My guess is that it is at least somewhat of a trade / barter since xnor.ai's wordmark is displayed now on Wyze video, which is an unconventional move. It's not like technically speaking Wyze would need to show xnor's wordmark on their video stream so that's why I guess it was negotiated. If it is, it is probably a good deal for xnor since that will get them lots of exposure for other potential customers.

Agree
Disagree
Informative: 2
Unhelpful
Funny

1 Hour webinar with Wyze and xnor answering questions about their person detection:

I have not watched yet, passing along for others interested.

Agree
Disagree
Informative: 2
Unhelpful
Funny

Guaranteed a Dahua camera would behave the same as the Hikvision.  The damn things trigger on sun shadows throughout the day.  

Agree
Disagree
Informative
Unhelpful
Funny: 1

Yes, we picked Axis and Hikvision for emphasis as they are respectively the largest Western and China manufacturers.

But Wyze would beat Dahua, Hanwha, Uniview and pretty much every other mainstream camera manufacturer right now save for Avigilon and Bosch. Related: IP Camera Analytics Shootout - Avigilon, Axis, Bosch, Dahua, Hanwha, Hikvision, Uniview

Agree
Disagree
Informative
Unhelpful
Funny

I almost have to retract my previous comment about the person Detection. My camera hasn’t detected anyone for more then a week now. I even stand outside for a few moments so the camera can recognize me. Could be a glitch and need to be turned off and back on but been to lazy to do that yet. 

Agree
Disagree
Informative: 1
Unhelpful
Funny

I experienced something similar with my cameras, it was just the notifications needed to be toggled on/off, not the camera in my situation.

Agree
Disagree
Informative: 2
Unhelpful
Funny

Actually, Ingenic (a Chinese chip manufacturer), is also a provider for Xiaomi (for Face unlocking devices) and, more interestingly, HIKVision (!) and seems to be hugely funded by the Chinese government:

https://www.burglaryalarmsystem.com/product-news/battery-powered-smart-doorbell-camera-ingenic-t10-t20-soc-solution.html

I wonder how much "stitches" and how much real innovation is inside Xnor and Wyze if the whole "AI" thing is inside the chip, as per its specification states it.

 

Agree
Disagree
Informative: 1
Unhelpful
Funny

Just ordered a couple of these.  Can't wait to try them out!  Anybody know if the AWS cloud part works in Canada?

Agree
Disagree
Informative
Unhelpful
Funny

UI#6, I checked with Wyze support, their response:

Yes! The cloud storage is available to customers whose cameras are setup and located in Canada and other countries outside of the U.S.

Agree
Disagree
Informative: 2
Unhelpful
Funny

Thanks Rob. I installed 4 Wyze cameras, a motion sensor and 2 door contacts about 3 weeks ago. I'm in sales on the manufacturing side. Very easy to install and if installation was easy for me it will be easy for all of your readers. Person detection had significant false positives initially but that has improved to the point of almost zero false positives. It definitely misses people at times though. Funny example: it's currently 4pm and I am the office. I can see my brother-in-law drinking my beer on the "backdeck camera" yet there was no person detected in the driveway or in the front of the house. (The only ways to get to my back deck.) This was not caused by the 5-minute cooldown because no people have been detected since 2:21pm. I live on a fairly quiet residential street in the city. People have definitely walked by on the sidewalk in front of my house in the past 2 hours.

Having said that; I'm pretty impressed with Wyze. For basic residential use they have decent quality at a very low price and very easy installation. If you want to have an actual security benefit you must install the micro-SD storage though. Otherwise you get 12 seconds of the burglar approaching in the distance and then a 5 minute blank. Obviously the cameras and sd can be easily stolen or destroyed. But it's better than no camera system.

The contact closures also work really well and alert promptly on my phone if those doors are left open. Wyze blogged recently about an outdoor camera; looking forward to checking that out...

Agree
Disagree
Informative
Unhelpful
Funny

Test Update / 2019-08-01:

The camera has been up and running since we completed our test, looking at our workbench and storage area. False alert performance has been very solid, with only 2 false alerts in the past week, both at night, triggered by a bug flying past the camera. The full clip:

It has also rejected alarms from light changes here, both from the overhead lights turning on and off and a nearby door opening/closing while the lights are off in this room.

Agree
Disagree
Informative: 2
Unhelpful
Funny

Update -

Wyze has added a new add-on service for 1.49/month that removes the cooldown period between motion events, we will be testing this and releasing an update to the report.

Agree
Disagree
Informative: 1
Unhelpful
Funny

Wish they support NFS...it can't be that hard to implement

Agree
Disagree
Informative
Unhelpful
Funny

Wyze update:

"Person Detecion will be temporarily removed from Wyze Cam beginning with a new firmware release pallned for mid-January 2020, due to the unexpected termination of our agreement with our AI provider."

Looks like the squeeze is on.

Agree
Disagree
Informative: 3
Unhelpful
Funny

We received that email as well and are in the process of getting more details from Wyze and Xnor. We’ll post here when we have an update.

Agree: 1
Disagree
Informative: 3
Unhelpful
Funny

Wyze just communicated that they just had a falling out with Xnor. It had to do with the agreement they had in place. Maybe they got the publicity they needed and now they want more money? It would be interesting to find out why and whether Wyze is able to create anything comparable (it's mentioned in the Wyze email blast) to replace them (Cloud is an option based on the message).

Agree
Disagree
Informative: 1
Unhelpful
Funny

Removing person Detection on cew camera sales is one thing. It sounds like this also applies to cameras already purchased! I don't see how than make that new firmware mandatory. They collected the money when they sold the cams with person detection. Surely they can't make users "return" a key feature of product they purchased...

Can't even imagine the scrapping behind the scenes on all of this!

Agree
Disagree
Informative
Unhelpful
Funny

yep

but,

What if we do not upgrade firmware?

Agree
Disagree
Informative
Unhelpful
Funny

Maybe Axis wrote a bigger check.

Agree: 1
Disagree
Informative
Unhelpful
Funny: 2

Initial reflex is Wyze deserves it if they didn't have a good contract in place.

Agree
Disagree
Informative
Unhelpful
Funny

Wyze released a timeline for upcoming products and their updated non-Xnor person detection still does not have a release date. It is currently shown as being in Alpha:

Product Progress Bars-12

They had this to say, which only offers up the vague "soon" as a release date:

We were really bummed when we had to remove person detection from Wyze Cams due to the Xnor.ai acquisition. Since their acquisition, we have been doubling down on our solution. Our in-house person detection for all Wyze Cams is in alpha testing. We are making great strides here and should have this highly requested service live again soon.

Agree
Disagree
Informative
Unhelpful
Funny

Hey Ethan. Wyze recently released their version of Person Detection. Would love to see an evaluation. Also wondering if I'm at risk by keeping the old firmware with Xnor analytics....

Agree
Disagree
Informative
Unhelpful
Funny

We will be testing it, but it will probably be in a few weeks. Rob has been in the beta test program and may be able to share some thoughts. Did they make their person detection public? I usually get newsletters from them about that sort of thing.

As far as a risk by keeping old firmware, I am not sure. Their release notes don't mention security improvements, but that doesn't mean there aren't any.

Agree
Disagree
Informative
Unhelpful
Funny

Thought they had released their version but can't find anything. Guess I was mistaken. "Early 2020" is past at this stage and Wyze seems to have a focus problem. Weigh scales, fitness bands, etc...

One cool thing they have released is firmware to convert a camera into a webcam. Has anyone tried that functionality?

Agree
Disagree
Informative
Unhelpful
Funny

Wyze recently added their own "Person Detection" analytics? Is IPVM planning on testing it? Have any others here tested it? I have 3 cams but have not been doing updated to keep the Xnor analytics.

Agree
Disagree
Informative
Unhelpful
Funny
Read this IPVM report for free.

This article is part of IPVM's 7,327 reports and 971 tests and is only available to subscribers. To get a one-time preview of our work, enter your work email to access the full article.

Already a subscriber? Login here | Join now
Loading Related Reports