Amazon Rekognition Facial Detection Tested

By: IPVM Team, Published on Dec 03, 2018

Amazon's Rekognition service has announced improvements, that if accurate, would be a significant advance over conventional facial detection.

amazon recognition

In particular, Amazon claims:

Amazon Rekognition can now detect 40 percent more faces – that would have been previously missed – in images that have some of the most challenging conditions described earlier... These aspects might include pose variations caused by head movement and/or camera movements, occlusion due to foreground or background objects (such as faces covered by hats, hair, or hands of another person in the foreground), illumination variations (such as low contrast and shadows), bright lighting that leads to washed out faces, low quality and resolution that leads to noisy and blurry faces, and distortion from cameras and lenses themselves. [emphasis added]

IPVM developed a dataset of 100+ 3-second video clips:

facial detection dataset 2

These clips are done across a number of common real-world scenarios:

  • Day light (200 lux)
  • Low light (1 lux)
  • Side tilt (from 0° to 90°)
  • Down tilt (from 0° to 30°)

We compiled various combinations (such as 200 lux, 15° side tilt, 22° down tilt or 1 lux, 75° side tilt, 30° down tilt, etc.).

We then ran tests of these 100+ clips across facial detection approaches:

  • HAAR 
  • HOG
  • Openvino (specifically the face-detection-retail-0004 model on Intel i7 and Myriad X chips expanded upon from our NCS2 test)
  • Amazon Rekognition

Inside, we explain full results and key variances in performance.

******'* *********** ******* *** ********* improvements, **** ** ********, would ** * *********** advance **** ************ ****** detection.

amazon recognition

** **********,****** ******:

****** *********** *** *** detect ** ******* **** faces – **** ***** have **** ********** ****** – ** ****** **** have **** ** *** most *********** ********** ********* earlier... These ******* ***** *******pose ********** caused by head movement and/or camera movements, ************ ** ********** ** background ******* (**** ** faces ******* ** ****, hair, ** ***** ** another ****** ** *** foreground),illumination ********** (such as low contrast and shadows), bright lighting that leads to washed out faces, low ******* and resolution that leads to noisy *** ****** *****, and distortion from cameras and lenses themselves. [emphasis added]

**** ********* * ******* of ***+ *-****** ***** clips:

facial detection dataset 2

***** ***** *** **** across * ****** ** common ****-***** *********:

  • *** ***** (*** ***)
  • *** ***** (* ***)
  • **** **** (**** *° ** 90°)
  • **** **** (**** *° to **°)

** ******** ******* ************ (such ** *** ***, 15° **** ****, **° down **** ** * ***, 75° **** ****, **° down ****, ***.).

** **** *** ***** of ***** ***+ ***** across ****** ********* **********:

  • **** 
  • ***
  • ******** (************ *** ****-*********-******-**** ***** ** Intel ** *** ****** X ***** ******** ******** *** **** ****)
  • ****** ***********

******, ** ******* **** results *** *** ********* in ***********.

[***************]

Key ********

****** *********** ****** ********* ******** was *** ****** **** all ****** ****** ********* nearly ** **** ** Openvino:

facial detection overall accuracy

*** ****** *** ********* quite '***' *** **** of *** ******** ** used **** ***** ****, including *** ********* *** bad ******.

*** *******, **** ** segment *** ****** ** key ********** ****, **** for ****** ***********, ***** differences ** *********** ******* are *******:

rekognition accuracy segmented

**** ******* *** *** severe ****** ******** ***, with '****' ***** *********** at ****** **% ***** low ***** *********** ** under **%.

***** *********** *** ********* in ****** ********* ********, it ***, *** *** away, *** *******, ** two ****** ** ********* versus ******** *** **** ******* on ** **: 

facial detection fps

*********** ** **** ****** **** the ***** ******** ** the *********** ***** ******* ******** the ***** ** ** sent ***** ** *** while *** ***** ******** can ** **** ******* / ** *** **** / ***** *** ****** is. ** ******* ***** measurement ******* ** *** final ******* ** **** report.

********

*****, ** ******* ** 100% ******** *** ****** Rekognition, ***** ** ****** strong *********** *** **** is **° **** *** 45° **** **** **** the ******:

[video-to-gif output image]

****, ** ******* ** 59% ******** (******* ~* out ** ** ****** correctly ******* * ****) for ****** ***********, ****** a ***** ********* ** steep **** *** **** tilt **** *** *******'* head ****:

*** ***** ******, ** noted, **** **** **** challenging **** *** ***********. For *******, **** * out ** ***** * frames *** * **** accurately ******** ****** **** a ***** ***** *** the ****** / ********:

 

*******, *** ** *** few **** ************ *** Rekognition *** ** **** test ********, ****** *** light, *** ******* ****** straight ******* *** ****** but ***** **** *** not ******** ** ***:

Accuracy ***********

*** ******** ****** **** is ************** *** * simplification. *********** **** ********* evaluation **** * ****** called"**** ******* *********" (***), ***** ******** ***********-**-******* on ****** **** ********* datasets ********* ***** ** ****** **** bounding-box ******* (********* ****** "Intersection **** *****" ** IoU) *** ******* **** box ********* ******** * face. ** ****'* ***** library, **** **** ****** has *** **** (***** frame *** * ****, and ***** ** **** one). ***** ****, ** can *** * ****** heuristic *** ********: ** how **** ****** *** a **** ******** (******** / *****-***** = ********). This ** *** *******: in **********, ** **** not ******* *** ***** positives (* **** ** detected ***********, **** ** the **** ** ** the *******'* *****). **** happens ****** - ** we ***** **** *** heuristic. ** ****** ** more ******* **** ********** (lux, *****, ***) ******* having ** ***** ******** boxes *** ***** ***** of *** *******.

FPS ***********

*** ******* **** ****** used *** *********** ***:

  1. ****** *** ***** ** Amazon **
  2. ***** *** *****
  3. ***** *********** (******** ** ** the ** ****, *********** an***->***********)
  4. **** *** *****, ********* FPS

* **** ****** ******** for ************ ***** ** to ****** ****** ******** via *******->***********. **** ***** *** the ********** ******** ** a *****-**-***** *****, ***** our ***** ***** *** networking ******** **** *** FPS *********** ********. ** such, *** *** ***********, already **** ***, ***** be **** *****.

***** ** **** ************, this ** * ***** point ******* ***********, ** others ** *** ** a *******. *** *******, one *** *** ***-***** hardware *** ****** ******* performance. *** ***** ****** in **** ******* *** on ****** * ** Intel ** ******* (****-***) devices, ** **** *********** trade-off ** *********-*********. ************, Rekognition ** * ******* service, ***** *********** * model (**** ********'* ****-******-****) takes **** ************* *** setup. ** ***** ****-**** performance **** *********** ** ******** (note: ** *** ***** investigating ** **** ** possible), *** ***-****-**** ************, Rekognition ***** ** ***** useful.

*******

*** * ****-**** ***** surveillance ***********,*********** ************ ** ***** **** but *** ***** ************ where **** ********** ***** processing ** ******, ** could ** *** **** affordable **** **-**** **********. For *******, *********** ******* for **-**** (*. ********) is $0.12 *** * *** of **** ****** ***** analyzed. **** **** **** 10% ******** ** ******* (i.e., *,*** ******* ** a *****), *** ******* **** would ** ~$** (*,*** x **% * $*.**) just *** *** **** analytics.

** *** ***** ****,****** *********** ****** ******* ***************** ****** ***********, ****** analysis (***, ******, *******), object *** ********** *********, etc. **** **** ********* provider ****** ** *** have ** ***** ** likely ** ******** ** compete **** ***** ***********.

********

**** ** *** *** of ****'* *** ****** of **** ******** *****, starting **** ****** ********* ******, ***** ****** ******* ***** 2 / ******** ** Test *** *** ****. ** are ******** * ******* of *** ***** ** the ******** ******. *** feedback, ******** ** *********, please *** ** *** comments.

Comments (7)

John Honovich

IPVM | 12/03/18 01:52pm

*** ***** ** *** interested ** *** ************,***** *** *** *** class **** ****:

deep learning webinar2

***** **** ** ********** fundamentals ** **** ** our ******** ****** ** provide ************* ******** *** education, ** ******** ** our ******* *****.

Undisclosed Manufacturer #1

12/03/18 02:50pm

*** *** ******* **** running *** ********** **** IR *********** **? **** cameras ***** ****** ** IR *********** (** **** have *** ******) ** 1 ***

John Honovich

IPVM | In reply to Undisclosed Manufacturer #1 | 12/03/18 09:02pm

** *** *** ** any ** ******** *** we **** *** **** in ****** *****.

Avatar

Matthew Del Salto

BLACKBOX USA | 12/03/18 03:12pm

**** **** ***** **** integrate ** **** **** if **** *****'* *******.

Undisclosed #2

12/03/18 07:45pm

** ***** *** *** you *** **** *** of **** *** ***** used ** *** ***** for **** ***** ** download *** *** **** their *** ********* *******?

Undisclosed Manufacturer #3

In reply to Undisclosed #2 | 12/03/18 08:02pm

* ***** **** ***** be * ****** ***** way ** ******* ********.

John Honovich

IPVM | In reply to Undisclosed #2 | 12/03/18 09:03pm

** ***** ******* *** dataset *** **** *** decided ***.

Login to read this IPVM report.
Why do I need to log in?
IPVM conducts unique testing and research funded by member's payments enabling us to offer the most independent, accurate and in-depth information.

Related Reports

Facial Recognition Systems Fail Simple Liveness Detection Test on May 17, 2019
Facial recognition is being widely promoted as a solution to physical access control but we were able to simply spoof 3 systems because they had no...
Covert Facial Recognition Using Axis and Amazon By NYTimes on May 20, 2019
What if you took a 33MP Axis camera covering one of the busiest parks in the US and ran Amazon Facial Recognition against it? That is what the...
Carnegie Mellon AI Startup Zensors Profile on Jun 11, 2019
Zensors is a startup formed by Carnegie Mellon graduates from a Carnegie Mellon research project, offering customized models per camera that they...
False Verkada 'Unrivaled' Low Light Performance Claim Removed on Jun 12, 2019
Verkada falsely claimed that it delivered 'UNRIVALED LOW LIGHT PERFORMANCE' until IPVM questioned. In fact, Verkada's low light performance is...
UK Facial Recognition Essex Errors Report on Jul 05, 2019
Facial recognition trials in the UK have generated significant controversy and debate over the past few years. This week, it flared again when Sky...
Verkada People And Face Analytics Tested on Aug 16, 2019
This week, Verkada released "People Analytics", including face analytics that they describe is a "game-changing feature" that "pushes the...
Scylla AI Video Analytics Company Profile on Aug 29, 2019
Scylla, an AI analytics startup, says they are targeting 1 Billion dollar valuation in 5 years and it "is not rocket science" to detect weapons and...
Avigilon Appearance Search Tested on Oct 30, 2019
Avigilon Appearance Search claims that it "sorts through hours of video with ease, to quickly locate a specific person or vehicle of interest...
Gatekeeper Security Company Profile - Detecting Faces Inside Vehicles on Nov 14, 2019
Border security is a common discussion in mainstream US news and politics, as is the use of banned Chinese equipment by US Government agencies....
Avigilon Facial Recognition 'Appearance Alerts' Tested on Dec 18, 2019
Avigilon has released 'Appearance Alerts' in ACC 7.4, which adds real-time facial recognition. To see how this performed, we tested them with...

Most Recent Industry Reports

Motorola / Avigilon Drops ISC West on Feb 26, 2020
Motorola Solutions has pulled out of ISC West 2020 effective immediately, because of coronavirus concerns, IPVM has learned. This is done amidst...
Cancel or Not? Industry Split Over ISC West on Feb 26, 2020
The industry is split, polarized, over whether ISC West 2020 should run or be canceled. New IPVM survey results of 400+ respondents show heated...
Coronavirus Hits Sony, Bosch Says Switch on Feb 26, 2020
Sony's fall in video surveillance has been severe over the past decade. Now, they may be done. In this note, we examine Bosch's new...
Video Surveillance Cameras 101 on Feb 25, 2020
Cameras come in many shapes, sizes and specifications. This 101 examines the basics of cameras and features used in 2020. In this report, we...
Favorite Video Analytic Manufacturers 2020 on Feb 25, 2020
Video analytics is now as hot as ever, driven by the excitement of advancing deep learning offers. But what are actually integrator's...
Latest London Police Facial Recognition Suffers Serious Issues on Feb 24, 2020
On February 20, IPVM visited another live face rec deployment by London police, but this time the system was thwarted by technical problems and...
Masks Cause Major Facial Recognition Problems on Feb 24, 2020
Coronavirus is spurring an increase in the use of medical masks, which new IPVM test results show cause major problems for facial recognition...
Every VMS Will Become a VSaaS on Feb 21, 2020
VMS is ending. Soon every VMS will be a VSaaS. Competitive dynamics will be redrawn. What does this mean? VMS Historically...
Video Surveillance 101 Course - Last Chance on Feb 20, 2020
This is the last chance to join IPVM's first Video Surveillance 101 course, designed to help those new to the industry to quickly understand the...
Vulnerability Directory For Access Credentials on Feb 20, 2020
Knowing which access credentials are insecure can be difficult to see, especially because most look and feel the same. Even insecure 125 kHz...

What IPVM Is

The world's leading authority on video surveillance, with 200,000+ visits monthly.

  • Articles on cameras, video analytics, VMS, and trends
  • Tests showing actual performance and problems
  • Courses in surveillance, access control, etc
  • Camera Calculator for designing systems

Dedicated To Independence

We're dedicated to staying independent and objective. We're the only industry source to refuse any and all advertisements, sponsorship, and consulting from manufacturers. Why?