Facial Recognition Test Results Shared

Hi John, I'll be glad to share our findings with you.

Our primary purpose was to evaluate the FaceVACS-VideoScan (software version 4.8.0 -- today's version is 5.1) for video surveillance (and not for access control).

For our evaluation we placed our video capture device (an uEye GigE IDS UI-6250SE IP camera coupled with a Pentax 1/2" C Mount 8-48mm F1.0 Manual Zoom Lens) next to the coffee corner of our Security Management department. The coffee corner has a frequent movement of employees and visitors. The camera was placed facing the corridor that gives access to the coffee corner, which results in people frontally facing the camera.

The camera's settings were as follows:

• Aperture size: F/2
• Lens focal length: 24 mm
• Focus distance: 6 meters

By using these camera's settings the closest and furthest acceptable sharpness are 5.27 m and 6.96 m, respectively. A total depth of field of approximately 1.70 m, therefore. The Lux (luminous flux) within the depth of field (between 5 and 7 meters) was above 1000 Lux. It is worth highlighting that the IP camera was connected to a PC workstation (Intel Core i5- 2500 3.30GHz processor, 4 GB RAM, 250 GB Hard Disk, Windows 7 Enterprise 64-bit) through a Gigabit switch.

61 Nedap employees participated in our evaluation by allowing their pictures to be enrolled on the watchlist database. We deliberately divided the enrollment pictures into 2 categories: Optimal and Standard. Optimal enrollment pictures (27 employees in total) consisted of photos taken from our employees's faces following Cognitec's guidelines (e.g., frontal pose, proper lighting, etc.). On the other hand, Standard enrollment pictures (34 employees in total) consisted of ordinary photos (Facebook or LinkedIn style) of our employees's faces. These ordinary photos did not fully follow the guidelines aforementioned as it is also the case for most photos of wanted criminals or suspects. The optimal enrollment pictures category contained 3 photos of the same employee with different poses (1 frontal and the other 2 slightly to the sides). Whereas the standard enrollment pictures contained only 1 photo (mostly frontal pose) of a single employee. All photos were stored in a Microsoft SQL Server 2008 R2 database.

After enrolling the images of our employees on the watchlist database, we observed their movement during a period of 24 hours. During this period we observed:

1. The number of correct matches: the evidence image correctly matches the enrolled subject.

2. The number of false matches: the evidence image incorrectly matches a certain enrolled subject.

3. The number of missing matches: the evidence image contained the face of an enrolled subject, but there was no match. That is, the score value was below the global threshold value set up for the purpose of recognition.

We then grouped all evidence images of a certain subject into events. An event consisted of two or more evidence images of the same subject during a period of time. The period of time is determined by the moment the subject enters the camera’s angle of view (i.e., first snapshot is taken) until the moment he/she leaves it (i.e., last snapshot is taken).

In order to determine if an event was successful (i.e., it resulted in a correct match), at least one snapshot belonging to this event should be successful. For example, the subject walked into the camera’s angle of view and 10 snapshots of this person were taken. If at least one snapshot resulted in a correct match, the entire event is considered as a correct match.

Out of 155 events registered during the period of 24 hours, 93 events (~60,00%) were correctly matched, 47 events (~30,32%) were missing alarms, and 15 events (~9,68%) were false alarms.

If you have any further question, do not hesitate to ask!

Sharar, I am not saying the test is unfair! :) I am just trying to get a sense of how it's being used. His conditions may simply be more demanding on the technology than yours.

For instance, if you are getting 99.7% accuracy with an analog camera pointed 45 degrees down into crowds of people in a train station against a 100,000 person watchlist, that sounds too good to be true.

So Sharar, I will ask the same questions to you - what is your setup where you are getting 99.7% accuracy?