Gender Video Analytics Shootout

By Rob Kilpatrick, Published Dec 14, 2020, 08:09am EST (Research)

***** ***** ********* *** ****** ************** is ********* *** *** **** **** it ****** ****? *** **** ******** features *** ****** ********* *********** ***** to **** ***** **************?

IPVM Image

** ****** ****** ********* *** **** two ***** **** **** *************' *******:

******, ** *******:

  • **** ******** ******** **** ****** ****** classification?
  • **** *** *** ******** **********?
  • *** **** ********* ****** **************?
  • **** ******** ***** ****** **************?
  • **** **** **** ****** **************?
  • *** ** ****** *** **** ********* impact ********?
  • *** **** *** ****** **************?

*******

***** ** *** *****, *** ******* but *** **** ******* ****** **** length **** *********** ******, ****** ** by *** *** ******* ********* ** whether ******* **** ** ********** ** male ** ******. *******, ***** ********* were ****** *************** *** ********* **** mistakes **** ** ******** ***** ***** not.

**********, **** **** *** *** ***** effect, **** ******** ** ******* ***** and ******* ********** ** **** ** female ***** ** ***** *******, **** no ***** ***** ***** ** **** type.

***** ******* ****** ** ************:

  • ********'* ** ******* *** ********** ****** most ******* ******** ****, *********** ******** with **** **** ** ****** *** short **** ** **** ** **** rates. ***** ******* **** ** **** type, ****** ********, *** ******** ***** were *********** *******.
  • ***** *** ****** ********* *********, **** heavily ********* ****, *** **** ******** style, *********** ******** ** ****** ** female **** *****. ************, **** * face *** ********, ****** ******** **** more ******* **********.
  • *********'* ****-**** **** *** ****** **** performed ****** ******** **** * **** was ********, ****** ****** ***** ********* analysis ** ****** *********, **** **** subjects **** ****** ****. ******* ** the **** *** ******** *** *** camera ** ******* * **** (~*** PPF ** ******), ** *** **** likely ** ********* ******** ******.
  • ******* *********** **** ********** **** ** our ***** *** ******* ******** *****, facial ********, **** ****, ***. ******* was **** *** **** ************ ***** sometimes *** *** ******** ******, ****** detecting ******** ** * ****** ******* of **** ** ******.

Accuracy ********

******** ** ****** ** ** ***** high (**%+) ** **** ************, ** men ********* **** ******* **** **** women, ***, ****** ******** *** ****** who ** *** *** **** ********** and, ** ************ ***** *** ********** does *** ** **, ******** ** likely ** ** ***** ****.

Performance ********

***** ** ** ******** ** *** performance ** *** ** *** ****** recognition *********.

IPVM Image

Avigilon *** ******* ********* **** ******

**** ****** *** **** ******* ******** for *************** ** *** ******* ** all ************* ** ********* (********* *****).

******* *** ******** ******** **** *** heaviest. ** * ****** *** ******** with **** **** **** **** **** likely ********** ** * *****, ********** of ***** *******, **** ** ******* visible ****** **** ** **** *****, shown *****.

IPVM Image

*********, ******** ********** ****** *********** ******* other ******* *** ********** ******** ******* weighted ** **** ******.

IPVM Image

Dahua / ****** ****** **** ******* ** ****** *********

****** ****** ******, ***** *** ****** classify ****** ** **** ****** ********* and **** *********, **** **** ********* more ********. **** ***** ****** *********, hair ** **** ******** ********.

*** *******, ***** ******* ******** ****** by **** ****** **** ***** ****** detection, ***** ***** ********** ** ****** if **** *** **** *** **** versa.

IPVM Image

****** **** ******* ******** ****** ** hair ****** **** ***** ****** *********, resulting ** ******* ***** ********** ** males ** **** *** *****.

IPVM Image

Face ********* **** ******** ****** *********** **** ******

**** ***** **** ********, ***** *** Hanwha ****** ************** ****** **** **** accurate.

*** *******, *****, ****** ********* *** male **** **** **** ** ** a **** **** * **** ** detected *** * ****** **** **** a ****** ** ********.

IPVM Image

*********, ***** ********** **** ****** ** a **** **** * **** ** detected *** * ****** **** *** body ** ********.

IPVM Image

Body **** ******** **** **** **** ** *** *************

**** *** ****** **** ***** **** weighted **** *** *********** **** **** short ** **** **** ** * person. ******, *****, *******, *** ******** would ***** ******** ******** ** **** if **** *** ***** ** **** back, **** **** ***** ****** *******.

IPVM Image

Gender *********** **** ******** ** ***** ***

******* ****** *** ******** **** ****** people ** **** ****** / *** PPF ******, ****** ************** *** ********** at ****** ********* ***** ******* ******** cannot ** *********. *** *******, *** woman ***** ***** *** ********** ********** as ****** ****** ** *** ******, but **** ** * ********, ******* visible **** ****.

IPVM Image

***** ********* ***** ** *** * higher *** **** ********* ****** *** still ***** *** ** ***** ****** at ***** ****, ********* ** ***** classifications.

IPVM Image

Skirts ******* ******** ** ****** *** *****

****** *******, ** * ****** *** detected ******* * *****, ***** ***** automatically ******** **** ** * ******, shown *****. ***** ************* *********** *********** clothing *****.

IPVM Image

************, ****** ******** ****** **** **** hair, ** * ********* ***** * female **** ***** **** *** ******** from ****** ** ***** ********* ** classified ** ****, **** * ***** was ****, *** ******* *** ****** classified ** ******.

IPVM Image

Verkada **** ** **************

******* *** *** ******** ****** ** all ****** **** **** ********. ** some *****, * ****** *** ******** and ***** ** *******'* **** ** people ******** *** *** ***** * gender. *** ***** ********* ********** ****** whenever * ****** ** **** *** detected.

IPVM Image

Hikvision **** *** / ******* ***** ********

********* ****** *********** *** *** **** accurate ** *** ******* *** *** also *** **** ***********, **** ******** faces ** ** ******** **** ** and ** **** *** (~**+).

IPVM Image

Hikvision **** ***** ******

********* *** *** ****** **** ******* compared ** ***** *********. **** ** long **** *** ******* **** *** subject's ****, **** **** ********** ********** as **** ***** ****** ***** ******** them ** ******.

IPVM Image

Versions ****

*** ********* ******** **** **** ****** testing.

  • ******** *.*-***-***-**: *.**.*.**(*****)
  • ******** ******* ******: *.**.*.**
  • ***** ***-*********-***: **.***.*******.*.*, ***** ****: ****-**-**
  • ****** ***-******: *.**.**
  • ********* ***-**********-*: **.*.** ***** ******
  • ******* *****: *** *****

Comments (30)

Hi, what do you think about the fact that Verkada is not classifying gender sometimes? The underlying question is: do video analytics users expect their system to provide a value even when the data is challenging (with high risks of error), or instead to not classify any attribute that is uncertain?

Thanks.

Agree
Disagree
Informative
Unhelpful
Funny

David, thanks for your first comment and a good question. To build on your question, what are people expecting to use gender analytics for? Is it primarily demographics? Or is it "show me men wearing X" or?

A lot of this seems to be more entertainment (i.e., interesting to demo) than actual use. To be clear, I am not doubting that some users make use of gender recognition but I suspect most do not.

Agree: 4
Disagree
Informative
Unhelpful
Funny

I think that not classifying gender all the time might be fine, but it would be better if there was a specific "Unknown" category, instead of simply not classifying. As it stands now, you can search for male or female, but if you want to see non-classified person detections you can only see them by turning all filters off, so you're looking at female, male, and unclassified all at once.

Agree: 5
Disagree
Informative
Unhelpful
Funny

Thanks Ethan. I agree, the system should provide data with high confidence and when it's uncertain, just say so. Just like a witness of an event would be sometimes able to provide information with a high level of accuracy and sometimes would not. IMHO bad data is worse than no data. Allowing the users to search specifically for "unknown" data becomes necessary with this approach.

Agree
Disagree
Informative
Unhelpful
Funny

We have been using an "Undetermined" category for some face objects, such as gender, glasses, face covering.

It has been primarily used in search instead of alerts, the idea being that you start with a search as specific as possible (Male, with glasses, blue shirt, moving in X direction), and then back off some of the specifiers depending on what you are able to get from a given scene.

The ideal case, of course, is that we have enough detail to make a definitive decision on these factors, but the reality that David points out is that many times you cannot make a perfect decision, so we are testing "Undetermined" as a way to articulate when the product knows it is unable to make a hard decision. This can be due to low resolution, shadows and poor lighting, occlusions, or other factors.

IPVM Image

IPVM Image

Note that we have a gender category for both Face and Person, where person takes more general appearance factors into consideration (including hair).

As an example, you can see in this scene the product classifies my face as Undetermined gender, due in part to the low resolution and angle, but when using a Person search, accurately classifies me as Male (insert low-effort jokes here).

IPVM Image

Overall, I think that gender classification is one of the areas where we (the industry as well as Vintra) would likely see a lot of improvement in the next several years.

Agree
Disagree
Informative
Unhelpful
Funny

accurately classifies me as Male

With what percentage confidence? :)

Agree
Disagree
Informative
Unhelpful
Funny: 2

It was 99%, but then I did some testing while holding a Pumpkin Spice Latte and it went down to 51%.

Agree
Disagree
Informative
Unhelpful
Funny: 24

Noted for my future purchases

Agree
Disagree
Informative: 1
Unhelpful
Funny

Extremely underrated comment!

Agree: 1
Disagree
Informative
Unhelpful
Funny

ahahahahahahahahahahhahaha LOL. Nice. And I have had a few pumpkin spice lattes myself as a male.

Agree
Disagree
Informative: 1
Unhelpful
Funny

I think data that is uncertain is useless for an end user. I would prefer manufacturers have a reliable service/product than to have to question it’s veracity. i know false alarms in any type of system contributes to a culture that tends to ignore any output of that system(s). I think Verkada and others of that ilk are best served providing high performing products versus beta testing on their customers. Gender analytics seem like a niche category. I can’t fathom what value do you get from an analytic classifying gender.

Agree
Disagree
Informative
Unhelpful
Funny

Say I am looking for a female with a red shirt and blue jeans. Being able to search the cameras for these descriptions can be useful but not sure how many people are using this feature. This is something that can be done with Avigilon's Appearance Search.

Agree: 1
Disagree
Informative
Unhelpful
Funny

I think we can all agree that a system with 100% accuracy would be the best. But given that this will probably never happen (for a lot of reasons), the question is to know what is the next best option. It doesn't make the system useless if it solves your problem 90% of the time. Forensic search is a common usage of gender analytics.

Agree
Disagree
Informative
Unhelpful
Funny

It doesn't make the system useless if it solves your problem 90% of the time.

David, it might 'work' 90% of the time, as long as most everyone has gender-stereotypical haircuts.

IPVM Image

Edit: To add, I think this could anger people for being mischaracterized. It's one thing for computers to make mistakes, it's another if the viewer looks at it and thinks it's obviously wrong and possibly offensive.

Agree: 2
Disagree
Informative
Unhelpful
Funny

" I can’t fathom what value do you get from an analytic classifying gender."

To know gender of a person crossing camera field of view could be useful for marketing strategy purpose, in some countries to ensure non mixity entrance for cultural reasons, to be alerted if a man attempt to intrude in a women only area...

Agree
Disagree
Informative
Unhelpful
Funny

We were playing around with this a few years ago. My favorite misidentification was when the system identified our Sales Leader (a male in his mid 30's) as a female. We did some digging and it appears that the algorithm gave some extra weight to his skinny jeans. He's been gone from our company for about 2 years, and I still love talking about it.

Agree
Disagree
Informative
Unhelpful
Funny: 5

Its too easy to post jokes on this topic and the pictures involved, so I'm not even going to do it.

Agree: 3
Disagree
Informative: 1
Unhelpful
Funny: 3

You do exist. No one was able to tell me what happened to you.

Agree
Disagree
Informative
Unhelpful
Funny: 1

I don't understand why there is a need to classify on sex. I have been telling manufacturers for a long while that sex classifications cheapen the overall analytics package when they do not work. Until it works near flawlessly I feel like gender classifications are just a "sure, why the hell not" feature that few ask for, few need, and does more harm than good.

For example, a few years back we had a pilot of an otherwise functional facial recognition platform that the customer rejected. The reason was not because the facial recognition analytic was faulty. The rate of correct identification was very high for 2016. The primary reason is that a single high ranking team member was being classified as 5'3 woman when this person was a 5'11" male. Hair length was not the root cause. This misidentification became a running joke to the point it was the entire brand image of the product. While this was not the sole person being misclassified it was definitely the wrong person. The sex and height classification item has since been removed from the facial recognition analytics package.

Agree: 1
Disagree
Informative: 3
Unhelpful
Funny

I don't understand why there is a need to classify on sex

The two main use cases I have seen are for post-event search and for retail analytics applications (which are also primarily post-event).

IMO, we are in the early days of the next phase of analytics, where systems will be able to provide more details about particular attributes of an object, and also start to figure out some context of an overall scene, not just that there are specific objects in the scene.

I would not sell a system today solely based on a demand for gender classifications, as I agree it is unlikely to work reliably enough over to time to satisfy a customer focused on that one specific use case. But when implemented correctly, it can augment other functionality for the user.

Agree: 1
Disagree
Informative: 1
Unhelpful
Funny

Better add Transgender to the filters or the wacko police are coming after you. Did you test a trans?

Agree
Disagree
Informative
Unhelpful: 8
Funny: 2

While this comment is rather insensitive, I do think these features sticking to "appearance" over gender identification leads to a better user experience.

"They were wearing a red shirt and blue jeans" is something that can be designed with a greater accuracy and less bias than "a woman stole my wallet."

Agree: 4
Disagree
Informative: 3
Unhelpful
Funny

A woman stole my wallet

A marriage status detector?

Agree
Disagree
Informative
Unhelpful
Funny: 8

Sorry, it was insensitive. Should have thought twice before hitting post.

Agree
Disagree: 3
Informative
Unhelpful
Funny

😂Too funny. Since no one will say it, i will. All that comes to mind is that YouTube video of the guy/gal in GameStop throwing a fit identifying as “ma’am” 😂😂😂😂

Would it be possible to run that ma’am thru the algorithm to see what it determines? 😂

Agree
Disagree
Informative
Unhelpful: 3
Funny: 1

But then, if the system gets it wrong (in their mind) they will sue

Agree
Disagree
Informative
Unhelpful: 2
Funny

One purpose not yet mentioned was the desire for live video marketing based on the subject in front of the monitor.

By identifying age, gender, race and other possible “style” attributes, you could display several out of thousands of programmed marketing messages.

In my case it would advertise retirement communities, golf pants and heating pads.

By chance is the person on the left available? Asking for a friend.

Agree
Disagree
Informative: 2
Unhelpful
Funny

ACC currently employs age/gender as a facet in the descriptive search feature. With a highly interactive function like descriptive search, the generic age/gender models only have to be good enough to initiate a search and get a reasonable set of images from which to further refine the search. Once an image is selected from initial search results the search quickly converges. The metric of success on searches is not so much initial accuracy but on how quickly an operator can find what they are looking for. For the same reason, gender results are much more accurate in an appearance search that is initiated from live or recorded video. As ACC introduces features such as natural language search or demographic analysis, more accurate gender models will be used.

Agree: 1
Disagree
Informative
Unhelpful
Funny

If you initiate a search and it misses key responses, refining the search certainly won’t make it show what it has already ignored.

I search for female, red shirt, black pants. 20% of the women in the scene are missed. Will further refining it help?

Agree
Disagree: 1
Informative
Unhelpful
Funny

AGAIN. Facial recognition algorithms either meet physics and mathematics requirements or ARE NOT facial recognition technology. That simple!

Agree
Disagree
Informative
Unhelpful
Funny
Subscribe to IPVM Research to read the full report.
Why do I need to subscribe?
The IPVM Research Service includes products tests and shootouts plus competitive and financial analysis, helping decision-makers better evaluate purchasing, partnering, developing, and/or competing against companies in physical security.
Already have an account?
Loading Related Reports