Facial Recognition Tested?

IPVMU Certified

Full testing on face rec software is not something we have done.

It is something we have discussed internally, though. Part of our discussion has involved how to best test performance so it generally indicates field performance.

For example, do we optimize image gathering and minimize variables on the camera side to test software performance, or do we put that at peril in order to use it in challenging spots - in crowds, high cameras/low PPF, variable lighting, and so on.

In other words, If face rec performs good/bad in our tests, will that translate into member applications? It would be bad for everyone if our results failed to apply.

Face rec is one area that generally needs very tightly controlled imaging environments to work well, but we're juggling how to test it in a pragmatic fashion.

If anyone has feedback or ideas, your comments are welcome.

Reactions:

TG

TOMISLAV GOLUB

Dear Brian, thank you for the explanation.I understand the issue involved. Regarding the software, there are many on the market today that I am not sure which to choose or test before.

I will list them a bit later and then if someone has any experience can add comments.

Reactions:

TG

TOMISLAV GOLUB

Ok here it is:

Herta, 3M, fst biometrics, etc...

Any experience?

Reactions:

Brandon Knutson

IPVMU Certified

Brian, I know little about facial recognition software, but I'd only be interested in ones that integrate with major VMS platforms.

Reactions:

(1)

U

Undisclosed #1

I am also in search of a viable facial recognition software to deploy on the IP cameras at my site, to be able to automatically recognize and alert (such as by a pop up box and alarm sound) if known troublemakers/hostiles come onto the property (i.e. matching against a database of photos). I am using NX Witness VMS.

Imagus (imagus.com.au) have facial recognition. They claim to have very good results even with low resolution, poor video. They come from Australia and are used in the US etc.

Last year I received an in-house demo done of what it is like with a PC and a web camera. It worked just fine to spot myself and the presenter in a lighted office. I don't know how exactly this would integrate with NX Witness.

I would very much appreciate real testing/reviews of Face Recognition software, such as Imagus, VCA, AllGo Vision, etc. Any testing would be better than no testing. For me, relevant testing would be in low-light, in daylight and in WDR scenes, outside entrances, at beginning of a driveway/property edge where a camera was, etc. Crowd testing would also be good. But right now any independent testing would be much better than what seems to be available online.

Reactions:

(2)

JH

John Honovich

IPVM

#1, thanks for the questions and sharing your info. Good stuff!

A couple of thoughts:

We have tested some of the consumer facial recognition products, e.g., Face Recognition Camera Tested (Netatmo), Nest Cam IQ Tested, Simplicam Facial Recognition Tested. Overall, did not perform well.
On the commercial side, offerings like NEC, Herta, are typically professional services projects, i.e., they cannot really be tested without their own people optimizing things on-site. Related to that, it tends to be extremely expensive once you factor in all costs in making that happen.
There are not many 'COTS' facial recognition / alerting offerings. For example, you mention VCA, and they offer COTS analytics, but looking at their site and from my previous knowledge of them, I do not think they are doing facial recognition / alerting.
FST Biometrics, as Tomislav has asked, is an access control company so, relative to what #1 is asking, they fall into a different space/application.
Your points about low-light, WDR, outside, crowd testing are very good. The reality is, everything is going to perform much worse in these conditions. I am sure there will be some variance but overwhelmingly I've seen legitimate providers actively try to avoid low-light, WDR, uncontrolled outdoor capture, etc. (I am leaving science fiction things like the Dahua marketing video).

So, IPVM would like to test some facial recognition, I am just not sure which one makes sense. We'd like a product that claims to be capable of running without professional services getting involved (since this significantly drives up end user cost, limiting appeal) and that has some momentum or size behind it, so that people would be more interested in seeing the results. What I would like to avoid is testing a product with low name recognition and then saying it works poorly. That is a waste of time for everyone since few people would not consider it anyway.

Any suggestions on the right choice, please share.

Reactions:

(1)

FL

Fraser Larcombe

Vix Verify - Imagus

Hi, first off I will say up front, I am the sales manager at Imagus, based in Australia. The product you will have tested last year will be a significantly changed product now, the results are faster and more acurrate than before. I will still state though, that for facial recognition to work properly requires the cameras to be placed in the right area and be set up as such too. We do work as a stand alone product but do also integrate into Milestone, I was at an event only yesterday where Milestone was showing it off on their stand, and people were queued up to talk about it. Although the better quality image we have the better, we then have our own database, that reduces the image sizes down significicantly allowing faster searches and extremely quick identification of the person. If you or anyone else would like to try it then I am more than happy to arrange some software to try, however as I mentioned earlier, the better the setup the better the results, seems to be something that is forgotten. I will also state for the record that I left the industry almost 2 years ago as I love solutions but analytics didnt really work properly, I have come back because I was shown the Imagus product!

Reactions:

(1)

FL

Fraser Larcombe

Vix Verify - Imagus

Hi, first off I will say up front, I am the sales manager at Imagus, based in Australia. The product you will have tested last year will be a significantly changed product now, the results are faster and more acurrate than before. I will still state though, that for facial recognition to work properly requires the cameras to be placed in the right area and be set up as such too. We do work as a stand alone product but do also integrate into Milestone, I was at an event only yesterday where Milestone was showing it off on their stand, and people were queued up to talk about it. Although the better quality image we have the better, we then have our own database, that reduces the image sizes down significicantly allowing faster searches and extremely quick identification of the person. If you or anyone else would like to try it then I am more than happy to arrange some software to try, however as I mentioned earlier, the better the setup the better the results, seems to be something that is forgotten. I will also state for the record that I left the industry almost 2 years ago as I love solutions but analytics didnt really work properly, I have come back because I was shown the Imagus product!

Reactions:

U

Undisclosed #1

NEC also claim to be a major leader in Face Recognition.

Reactions:

UE

Undisclosed End User #2

It is my understanding that Panasonic's facial recognition software can integrate to Genetec. To me that would be an interesting test.

Reactions:

JH

John Honovich

In reply to Undisclosed End User #2

IPVM

Panasonic is certainly a bigger brand, what I worry about their facial recognition software is that is has been around for a number of years and I can't recall either Panasonic or anyone else seriously touting their software, so that implies that even they do not think it is good. But happy to hear from any Panasonic people or those who have tried it, if they are confident in it.

Reactions:

KS

KV Swami

In reply to Undisclosed End User #2

Yes, there is an integration with Genetec, but you have to use Panasonic cameras for it to work, so that may be a deal breaker.

Here are more details on the Panasonic / Genetec Integration:

https://security.panasonic.com/products/frs_for_genetec/

I have also been trying to find a facial recognition software that has a deep integration with a VMS but have not been able to find anything completely satisfactory yet.

NEC's NeoFace is integrated with Genetec as well. There is a video demonstration of the integration: https://www.youtube.com/watch?v=lTIoFRrBBcM

Facial Recognition for Biometrics Integrated within Genetec:
- FST (http://www.biometricupdate.com/201703/fst-biometrics-imid-technology-integrated-with-genetec-access-control-system)

- PlugOut an integrator in the Northeast has done quite a bit of work on this as well: http://plugout.com/solutions/facialrec.html

I tested Herta's Facial Recognition a while back and they did have an integration module for Genetec as well, but it only passed basic events through to the VMS.

Reactions:

TG

TOMISLAV GOLUB

Thanks all for the comments. I know, in the end, I will only see how it performs once I buy it, lot of $$$ spent at the startup.

Thank you all for comments.

Reactions:

BP

Bas Poiesz

We have used a Hikvision camera for this, model iDS-2CD8426G0/F-I.

Matchting stats were quite good, multiple faces at the same time.
Guessing the correct age was way off though, not usable.

Then again, it's more about matching the correct face, age is less relevant

Reactions:

(3)

JH

John Honovich

IPVM

In reply to Bas Poiesz

Jonathan, thanks for sharing! For others, here is the IDS-2CD8426G0-F-I datasheet and a iDS-2CD8426G0/F-I detailed deck.

A few questions:

Is the max FoV width 2.5 meters / 8.2 foot? I read that in the doc and was curious to confirm.
How many people did you add into the 'face picture library' / watchlist? Did you say any scaling issues as you added 100 or more people?
Did you try using multiple cameras? Curious how that worked.

Related, we would definitely be interested in testing this unit since (1) it's a large brand and (2) it's being positioned as a product, not a project like NEC.

From what we have seen, we believe the DeepInMind NVR will be released first in North America so our plan is to buy and test that first.

Reactions:

BP

Bas Poiesz

In reply to John Honovich

Hi John

The camera we tested at our exhibition booth, not in combination with other face cameras. We are planning a more in dept test to confirm specs you mention.
The amount of people that can be added was no issue, 10.000 faces with a simple SD Card.

Hikvision gives you the choice, buy the deepinmind NVR and connect a simple camera, or buy the smart camera and use a simple NVR.

So far it is one of the most impressive Hik products we have used!

Reactions:

UI

Undisclosed Integrator #5

In reply to Bas Poiesz

Interesting product. What is the MSRP and where can it be found for purchase in the US?

Reactions:

BP

Bas Poiesz

In reply to Undisclosed Integrator #5

Sorry UI5, I am based in Europe. No MSRP know yet, I expect it in January

Reactions:

U

Undisclosed #1

Thanks John for the good points on this. Good point re the professional services cost aspect.

Reactions:

U

Undisclosed #1

Hi Jonathan, that model iDS-2CD8426G0/F-I sounds good.

Would it work with a VMS, such as Milestone or NxWitness, or does it need the Hikvision VMS?

Reactions:

BP

Bas Poiesz

Hi UD1

Hikvision does the analysis on edge, meaning it's in the camera.
This is a fundamental difference to software solutions who analyze metadata from a random camera with enough detail.

General VMS like milestone take the onvif stream of the image and don't interchange the edge detected data. Intergration time is one factor, and the profit on current models is another factor.

In time I am sure they will have it but they prefer server based analysis.

The camera can work stand-alone and has outputs to create some triggers. In the VMS you could then use the onvif stream.

Kind regards,

Jonathan

Reactions:

(1)

U

Undisclosed #1

In reply to Bas Poiesz

Thanks Jonathan.

Reactions:

Skip Cusack

Historically, there have been a few structured evaluations. You can search for FRVT, FERET, etc. and usually all roads lead to NIST (USA Standards group).

NIST is hosting an FR evaluation now. I've participated in several of these over the years. Preparing to compete is a fire drill on steroids in the best of times, and not entirely congruent with commercial product development. But it is very helpful to place well for all to see, as opposed to buyers simply taking vendors' claims at face value.

www.nist.gov/programs-projects/face-recognition-vendor-test-frvt-1n-2018-evaluation

This from NIST:

In the meantime, our existing evaluation of 1:1 face verification algorithms remains open:

www.nist.gov/programs-projects/face-recognition-vendor-test-frvt-ongoing

Reactions:

(1)

JH

John Honovich

IPVM

In reply to Skip Cusack

Skip, while I have seen the NIST tests for more than a decade, it has never been clear how to map those test results for 'commercial' offerings. For example, what does the NIST test tell buyers about the FST system or the Hikvision system or the Herta system, etc?

Reactions:

Skip Cusack

In reply to John Honovich

Yes, I agree. I think of NIST as a technology test, not a product test. It is still helpful to know something about the potential of the underlying technology when considering a product purchase. The usefulness of the NIST results may be constrained by how the test was structured, but I do give them high marks for being thorough and objective. I suppose one may look at Cognitec's results and infer something about FST's potential performance, I know I would, but it may be wading in too deep for many integrators.

Reactions:

(2)

UM

Undisclosed Manufacturer #4

In reply to John Honovich

Another issue to consider is the relationship between the tested algorithm and the algorithm that a vendor actually deploys, or deployed in the past (if you have an older system). This is especially critical when you consider the advances in facial recognition algorithms over the past decade

Reactions:

UI

Undisclosed Integrator #3

we have seen a couple brands - planning on testing:

http://www.princetonidentity.com/

a little bulky on the reader (doesn't use off the shelf cameras) but they work.

they are the software Samsung Galaxy8 is using for iris recognition technology

also http://bluelinetechnology.com/

They have drivers to a few cameras - based upon Pixels on target to get accurate detection.

they have a case study where all you need is a valid face read to get entry. That way they have a "log" of anyone entering facility.

Reactions:

(1)

RB

Robert Bell

Hi Tomislav,

I am the President of Face4 Systems, a face recognition solutions company.

This is a good discussion. I agree with the comments above. NIST tests technology, and more importantly algorithm performance. Testing a product in the field is another story with its own challenges.

As others have indicated, the positioning and configuration of cameras is very important, and in turn may require professional services of the solution provider. Edge computing (on camera) simplifies the deployment and maintenance and generally will reduce the need for professional services. Ultimately, the proper tools and training should remove the need altogether.

The key to face recognition success is the capture of good quality images suitable for face recognition. In uncontrolled, non-cooperative environments and especially in high traffic environments this is particularly challenging. You will always be fighting quality issues such as resolution, pose angle, blur, occlusion and the randomness of human behavior. Outdoor face recognition can be more challenging. We help our customers focus on the capture zone design to optimize performance.

To get real-life performance data in an operational environment, you may wish to look to operating organizations. We did a large scale evaluation of face surveillance in operation in an airport environment with Canada Border Services Agency. I am aware of good recent work by the Australian Immigration and Border Services in co-operative face capture on the move. I expect both organizations would share their experience.

Reactions:

(1)

TG

TOMISLAV GOLUB

In reply to Robert Bell

Dear Robert, thanks a lot. Yes it is struggling issue to get the best possible image from the target area. Long time ago I believed that there is a software that performs 100% , like the one used in TV shows, CSI etc....was I wrong...

Also some people I work with still have the same idea. I need to explain that there is no such software available. You might get close with the lot of effort.

Reactions:

U

Undisclosed #6

Aimetis (now Senstar) has an affordable top performing facial recognition analytic that is nicely integrated with its Symphony VMS:

http://www.aimetis.com/Analytics/face-recognition.aspx

http://cdn.aimetis.com/public/Library/Face%20Recognition%20Datasheet%2020170523.pdf

Reactions:

UI

Undisclosed Integrator #7

Good discussion so far. I was recently working on an overseas project that required facial recognition for over 100,000 cameras.

Needless to say that the types of technology that could be deployed at this level weren't your stock standard commerically available ones.

I ended up looking at systems from NEC, Qognify and Allevate (Tygart).

All came with considerable price tags. The Allevate one was interesting in that it offered FRAAS (facial recognition as a service), based on the number of searches it does and how big your "watchlist" is.

Might save the huge capex.

http://allevate.com/index.php/2017/10/05/five-serious-criminals-arrested-rio-shopping-mall-first-weeks-use-cloud-face-recognition-system/

Reactions:

Skip Cusack

In reply to Undisclosed Integrator #7

A system this large is very interesting. Did all the cameras share the same watchlist? Are you able to share how large the watchlist was? What sort of correct recognition vs. false alarm did you experience? Was the output a single ID, or a list of potential matches? If you can't get into details, was your impression that the FR delivered value? Did it work in the eyes of the customer?

Reactions:

UI

Undisclosed Integrator #7

In reply to Skip Cusack

Hi Skip,

Personally I don't think this specific project will happen as they have unrealistic expectations on what their cameras can deliver - most are probably in the wrong locations so will be useless for FR.

Watchlist is in the thousands and yes, the intention was for them all to be on the same watchlist with distributed servers to cope with the load.

Testing for this project hasn't commenced yet but i will let you know when it is ready.

Our partner has a similiar deployment in the Middle East so will try an get you some more info.

In our case, I believe it's difficult to quantify value - as it's not a commercially motivated purchase. It's in the category of CT - Counter Terrorism, public safety - well at least that's how it has been dressed up.

I personally think it's a good excuse for mass surveillance so how do you put a "$£€" on that? We have just priced it up as we would any other project - but with lots of margin built in.

Reactions:

Skip Cusack