Amazon Rekognition Facial Detection Tested

By: IPVM Team, Published on Dec 03, 2018

Amazon's Rekognition service has announced improvements, that if accurate, would be a significant advance over conventional facial detection.

amazon recognition

In particular, Amazon claims:

Amazon Rekognition can now detect 40 percent more faces – that would have been previously missed – in images that have some of the most challenging conditions described earlier... These aspects might include pose variations caused by head movement and/or camera movements, occlusion due to foreground or background objects (such as faces covered by hats, hair, or hands of another person in the foreground), illumination variations (such as low contrast and shadows), bright lighting that leads to washed out faces, low quality and resolution that leads to noisy and blurry faces, and distortion from cameras and lenses themselves. [emphasis added]

IPVM developed a dataset of 100+ 3-second video clips:

facial detection dataset 2

These clips are done across a number of common real-world scenarios:

  • Day light (200 lux)
  • Low light (1 lux)
  • Side tilt (from 0° to 90°)
  • Down tilt (from 0° to 30°)

We compiled various combinations (such as 200 lux, 15° side tilt, 22° down tilt or 1 lux, 75° side tilt, 30° down tilt, etc.).

We then ran tests of these 100+ clips across facial detection approaches:

  • HAAR
  • HOG
  • Openvino (specifically the face-detection-retail-0004 model on Intel i7 and Myriad X chips expanded upon from our NCS2 test)
  • Amazon Rekognition

Inside, we explain full results and key variances in performance.

******'* *********** ******* *** announced ************, **** ** accurate, ***** ** * significant ******* **** ************ facial *********.

amazon recognition

** **********,****** ******:

****** *********** *** *** detect ** ******* **** faces – **** ***** have **** ********** ****** – ** ****** **** have **** ** *** most *********** ********** ********* earlier... ***** ******* ***** includepose ********** caused by head movement and/or camera movements, ************ ** ********** ** background ******* (**** ** faces ******* ** ****, hair, ** ***** ** another ****** ** *** foreground),illumination ********** (such as low contrast and shadows), bright lighting that leads to washed out faces, low ******* and resolution that leads to noisy *** ****** *****, and distortion from cameras and lenses themselves. [emphasis added]

**** ********* * ******* of ***+ *-****** ***** clips:

facial detection dataset 2

***** ***** *** **** across * ****** ** common ****-***** *********:

  • *** ***** (*** ***)
  • *** ***** (* ***)
  • **** **** (**** *° to **°)
  • **** **** (**** *° to **°)

** ******** ******* ************ (such ** *** ***, 15° **** ****, **° down **** ** * lux, **° **** ****, 30° **** ****, ***.).

** **** *** ***** of ***** ***+ ***** across ****** ********* **********:

  • ****
  • ***
  • ******** (************ *** ****-*********-******-**** model ** ***** ** and ****** * ***** expanded ******** *** **** ****)
  • ****** ***********

******, ** ******* **** results *** *** ********* in ***********.

[***************]

Key ********

****** *********** ****** ********* accuracy *** *** ****** than *** ****** ****** including ****** ** **** of ********:

facial detection overall accuracy

*** ****** *** ********* quite '***' *** **** of *** ******** ** used **** ***** ****, including *** ********* *** bad ******.

*** *******, **** ** segment *** ****** ** key ********** ****, **** for ****** ***********, ***** differences ** *********** ******* are *******:

rekognition accuracy segmented

**** ******* *** *** severe ****** ******** ***, with '****' ***** *********** at ****** **% ***** low ***** *********** ** under **%.

***** *********** *** ********* in ****** ********* ********, it ***, *** *** away, *** *******, ** two ****** ** ********* versus ******** *** **** running ** ** **:

facial detection fps

*********** ** **** ****** than *** ***** ******** as *** *********** ***** service ******** *** ***** to ** **** ***** to *** ***** *** other ******** *** ** done ******* / ** the **** / ***** the ****** **. ** examine ***** *********** ******* in *** ***** ******* of **** ******.

********

*****, ** ******* ** 100% ******** *** ****** Rekognition, ***** ** ****** strong *********** *** **** is **° **** *** 45° **** **** **** the ******:

[video-to-gif output image]

****, ** ******* ** 59% ******** (******* ~* out ** ** ****** correctly ******* * ****) for ****** ***********, ****** a ***** ********* ** steep **** *** **** tilt **** *** *******'* head ****:

*** ***** ******, ** noted, **** **** **** challenging **** *** ***********. For *******, **** * out ** ***** * frames *** * **** accurately ******** ****** **** a ***** ***** *** the ****** / ********:

*******, *** ** *** few **** ************ *** Rekognition *** ** **** test ********, ****** *** light, *** ******* ****** straight ******* *** ****** but ***** **** *** not ******** ** ***:

Accuracy ***********

*** ******** ****** **** is ************** *** * simplification. *********** **** ********* evaluation **** * ****** called"**** ******* *********" (***), ***** ******** ***********-**-******* on ****** **** ********* datasets ************** ** ****** **** bounding-box ******* (********* ****** "Intersection **** *****" ** IoU) *** ******* **** box ********* ******** * face. ** ****'* ***** library, **** **** ****** has *** **** (***** frame *** * ****, and ***** ** **** one). ***** ****, ** can *** * ****** heuristic *** ********: ** how **** ****** *** a **** ******** (******** / *****-***** = ********). This ** *** *******: in **********, ** **** not ******* *** ***** positives (* **** ** detected ***********, **** ** the **** ** ** the *******'* *****). **** happens ****** - ** we ***** **** *** heuristic. ** ****** ** more ******* **** ********** (lux, *****, ***) ******* having ** ***** ******** boxes *** ***** ***** of *** *******.

FPS ***********

*** ******* **** ****** used *** *********** ***:

  1. ****** *** ***** ** Amazon **
  2. ***** *** *****
  3. ***** *********** (******** ** to *** ** ****, registering *****->***********)
  4. **** *** *****, ********* FPS

* **** ****** ******** for ************ ***** ** to ****** ****** ******** via*******->***********. **** ***** *** the ********** ******** ** a *****-**-***** *****, ***** our ***** ***** *** networking ******** **** *** FPS *********** ********. ** such, *** *** ***********, already **** ***, ***** be **** *****.

***** ** **** ************, this ** * ***** point ******* ***********, ** others ** *** ** a *******. *** *******, one *** *** ***-***** hardware *** ****** ******* performance. *** ***** ****** in **** ******* *** on ****** * ** Intel ** ******* (****-***) devices, ** **** *********** trade-off ** *********-*********. ************, Rekognition ** * ******* service, ***** *********** * model (**** ********'* ****-******-****) takes **** ************* *** setup. ** ***** ****-**** performance **** *********** ** unlikely (****: ** *** still ************* ** **** is ********), *** ***-****-**** applications, *********** ***** ** quite ******.

*******

*** * ****-**** ***** surveillance ***********,*********** ************ ** ***** **** but *** ***** ************ where **** ********** ***** processing ** ******, ** could ** *** **** affordable **** **-**** **********. For *******, *********** ******* for **-**** (*. ********) is $*.** *** * min ** **** ****** video ********. **** **** just **% ******** ** analyze (*.*., *,*** ******* in * *****), *** monthly **** ***** ** ~$50 (*,*** * **% x $*.**) **** *** the **** *********.

** *** ***** ****,****** *********** ****** ******* features********* ****** ***********, ****** analysis (***, ******, *******), object *** ********** *********, etc. **** **** ********* provider ****** ** *** have ** ***** ** likely ** ******** ** compete **** ***** ***********.

********

**** ** *** *** of ****'* *** ****** of **** ******** *****, starting ********** ********* ******,***** ****** ******* ***** 2 / ******** ** Test*** *** ****. ** are ******** * ******* of *** ***** ** the ******** ******. *** feedback, ******** ** *********, please *** ** *** comments.

Comments (7)

John Honovich

IPVM | 12/03/18 01:52pm

*** ***** ** *** ********** ** *** ************,***** *** *** *** ***** **** ****:

deep learning webinar2

***** **** ** ********** ************ ** **** ** *** ******** effort ** ******* ************* ******** *** *********, ** ******** ** our ******* *****.

Undisclosed Manufacturer #1

12/03/18 02:50pm

*** *** ******* **** ******* *** ********** **** ** *********** on? **** ******* ***** ****** ** ** *********** (** **** have *** ******) ** * ***

John Honovich

IPVM | In reply to Undisclosed Manufacturer #1 | 12/03/18 09:02pm

** *** *** ** *** ** ******** *** ** **** try **** ** ****** *****.

Avatar

Matthew Del Salto

BLACKBOX USA | 12/04/18 12:12am

**** **** ***** **** ********* ** **** **** ** **** haven't *******.

Undisclosed #2

12/03/18 07:45pm

** ***** *** *** *** *** **** *** ** **** raw ***** **** ** *** ***** *** **** ***** ** download *** *** **** ***** *** ********* *******?

Undisclosed Manufacturer #3

In reply to Undisclosed #2 | 12/03/18 08:02pm

* ***** **** ***** ** * ****** ***** *** ** compare ********.

John Honovich

IPVM | In reply to Undisclosed #2 | 12/03/18 09:03pm

** ***** ******* *** ******* *** **** *** ******* ***.

Login to read this IPVM report.
Why do I need to log in?
IPVM conducts unique testing and research funded by member's payments enabling us to offer the most independent, accurate and in-depth information.

Related Reports on Low Light

Hikvision Colorvu Camera Tested on Jun 24, 2019
Hikvision says their new ColorVu line captures "vivid chromatic images in darkness", with unconventional white light illuminations whereas most...
Repositionable Multi-Imager Camera Shootout - Avigilon, Axis, Dahua, Hanwha, Hikvision, Panasonic, Vivotek on Jun 19, 2019
Repositionable multi-imager cameras are one of the fastest growing segments in video surveillance, with a slew of new offerings being recently...
False Verkada 'Unrivaled' Low Light Performance Claim Removed on Jun 12, 2019
Verkada falsely claimed that it delivered 'UNRIVALED LOW LIGHT PERFORMANCE' until IPVM questioned. In fact, Verkada's low light performance is...
Camera Course Summer 2019 - Register Now on Jun 12, 2019
Register for the Summer 2019 Camera Course.  This is the only independent surveillance camera course, based on in-depth product and technology...
Avigilon 32MP and 12MP H4 Multisensor Cameras Tested on Jun 11, 2019
Avigilon has released their H4 Multi-Sensor line of cameras claiming "broad scene coverage and high image detail" We bought and tested the...
Vivotek 20MP Multi-Imager Tested on May 30, 2019
Vivotek has released their latest repositionable multi-imager camera, the MA9321-EHTV, a 20 MP model claiming "Supreme Night Visibility", "Optimal...
Panasonic 32MP Multi Imager Camera Tested (WV-X8570N) on May 16, 2019
Panasonic has released their first multi imager models including the 32MP (4x4K) WV-X8570N, claiming "Extreme image quality for evidence capturing...
Hikvision DeepinView Camera Analytics Tested on May 08, 2019
Hikvision is expanding its 'deep learning' offerings with a new camera series called 'DeepinView' claiming false alarm reduction and improved...
Speco Ultra Intensifier Tested on Apr 22, 2019
While ISC West 2019 named Speco's Ultra Intensifier the best new "Video Surveillance Cameras IP", IPVM testing shows the camera suffers from...
Startup Spectral Edge RGB+IR Claims "Unique Low Light Performance Tech" on Apr 03, 2019
This UK startup, born out of university research, is boldly proclaiming that they deliver "Unique low light performance tech for video surveillance...

Most Recent Industry Reports

Hikvision Colorvu Camera Tested on Jun 24, 2019
Hikvision says their new ColorVu line captures "vivid chromatic images in darkness", with unconventional white light illuminations whereas most...
China Subway Facial Recognition System Examined on Jun 24, 2019
A China city of 6+ million people has installed facial recognition-enabled gates in subways, allowing commuters to enter stations by simply showing...
HID Mobile Tested on Jun 21, 2019
HID Global is one of the largest access brands, but their mobile access has had challenges. Indeed, the company has already restructured their...
Genetec Beats Milestone For IHS #1 on Jun 21, 2019
For years, Milestone has touted that they are the #1 VMS. Now, Genetec has beaten them in IHS rankings. But what is this? Even other manufacturers...
Risk of Amazon Alexa Guard: No Battery Or Cell Backup on Jun 20, 2019
Amazon positions its Alexa Guard Service as a "smart home security system" and says it can help you "keep your home safe". However, the...
Exacq Remote Cloud Access Tested on Jun 20, 2019
Remote cloud access has been missing from most VMSes (including Exacq and Milestone). Now, Exacq, after releasing Cloud Drive Storage earlier in...
Briefcam Buys Frost Award* on Jun 20, 2019
Frost 'awards' are well-known and widely disrespected. Now Briefcam is touting their win. The way it has worked for many years is that Frost...
IFSEC 2019 Show Report on Jun 19, 2019
The UK's largest trade show, IFSEC, is underway and IPVM has been examining what is new and happening at the show. Inside, we cover: Huawei...
Repositionable Multi-Imager Camera Shootout - Avigilon, Axis, Dahua, Hanwha, Hikvision, Panasonic, Vivotek on Jun 19, 2019
Repositionable multi-imager cameras are one of the fastest growing segments in video surveillance, with a slew of new offerings being recently...
Genetec Synergis Cloud Link - Complex, Costly and Confusing on Jun 18, 2019
Genetec's Synergis Cloud Link is complex, costly and confusing compared to competitor access control architectures. Inside this note, we examine...

The world's leading video surveillance information source, IPVM provides the best reporting, testing and training for 10,000+ members globally. Dedicated to independent and objective information, we uniquely refuse any and all advertisements, sponsorship and consulting from manufacturers.

About | FAQ | Contact