Russian Face Rec Suppliers Offer Ethnicity Analytics, Raising Alarm

Four Russian facial recognition firms (AxxonSoft, NtechLab, Tevian, and VisionLabs) who supply the Russian government offer ethnicity analytics, raising concerns the country's police can track minorities.

Reuters published its own investigation based on IPVM's findings, see Reuters: 'Racist' facial recognition sparks ethical concerns in Russia.

Russian facial recognition is already under heavy scrutiny for helping police identify and arrest anti-government protestors. Moscow city government and its facial recognition supplier (NtechLabs) ignored Reuters' requests to comment on the topic. The other three companies gave detailed responses, with one removing the feature for going against its ethics standards.

PRC companies have already attracted criticism for offering China police ethnicity analytics to track Uyghurs. In this post, IPVM investigates ethnicity analytics in Russia.

Four Russian Companies With Ethnicity Detection Solutions

Russia has a significant facial recognition industry presence, with Russian firms expanding internationally and scoring well in NIST rankings for general facial recognition. AxxonSoft, NtechLab, Tevian, and VisionLabs are all Russia-based or founded video analytics providers with large Russia operations. (In April, AxxonSoft informed IPVM of its intention to IPO on the NASDAQ and redomicile in Ireland).

While IPVM found each has marketed or developed ethnicity analytics based on publicly available information from the companies, IPVM and Reuters were unable to confirm whether they are used by Russian police/government.

AxxonSoft "Ethnicity" Parameter in FaceIntellect

AxxonSoft is a Moscow-based VMS manufacturer mainly owned by its founder, Murat Altuev. The company reported 2020 revenue of ~$40 million USD and says they sell software solutions (VMS, PSIM, VSaaS) entirely through the channel. AxxonSoft has around 580 employees, about 200 of which are engineers.

In a guide for integrating its FaceIntellect 8.0 software, there are parameter descriptions that define ethnicity as "Mongoloid", "Negroid", "Caucasoid", and "East Indian":

Axxonsoft said the ethnicity analytics function/offensive terms were "inadvertently" integrated from "3rd party engines", saying they had removed it for "not aligning with [our] ethical standards", emphasizing "our company does not support any kind of discrimination".

Thank you very much for spotting the issue out. Although the integration has been made inadvertently, we consider such functionality as not aligning with the ethical standards of our company. AxxonSoft has 50 offices around the world and representatives of a diverse range of nations in our team. We are definitely not interested in promoting any technologies that could be a basis for ethnic segregation.

Now, we have removed all references to ethnicity analytics from our documentation. We held a dedicated meeting with all product managers to clearly communicate the idea of paying particular attention to the ethical side of functionality in 3rd party products we integrate. Currently we are working on the new version of Face Intellect with disabled ethnicity analytics feature, and it will replace the former version as soon as possible.

Based on the information provided by the engine developers, they cannot differ Central Asian migrants from local citizens. There are 190 nationalities from diverse ethnic groups in Russia, and Face Intellect cannot distinguish between them or differentiate them from migrants. Our company does not support any kind of discrimination, does not develop such solutions, and has no such plans in the future.

These outdated terms is a technical translation of documentation on 3rd party facial recognition engines. [emphasis added]

Axxonsoft said it had no "direct access" to information about end users or actual deployments. AxxonSoft did not say which third-party engine the offensive terms were integrated from.

UPDATE (7/12/2021): AxxonSoft provided a statement to IPVM, addressing the third-party engine:

In Face Intellect, we use Tevian and VisionLabs for facial recognition that included ethnicity attributes. Integration of ethnicity analytics has been made at a technical level. We have already removed all references to ethnicity analytics from our documentation. Race/Ethnicity analytics have already been disabled in Face Intellect.

NtechLab FindFace Code Includes 'Race'

NtechLab is a Russian face recognition and computer vision startup founded in 2016. It initially launched a Clearview AI-like app that scraped Russian social media company VK and then pivoted to surveillance software. It is partly funded by the Russian government's Russian Direct Investment Fund and has a 100,000+ camera analytics deployment in Moscow.

NtechLab's publicly-available descriptions of features for its FindFace solution explicitly mentions the capability of identifying "ethnicity":

Indeed, NtechLab's integration guide for FindFace includes a feature for "race":

NtechLab did not respond to comment requests. If they do, we will add to this report.

UPDATE [7/16/2021] NtechLab provided a statement to IPVM, claiming the ethnicity functionality exists to help "avoid bias" and that the functionality was "never delivered" to a customer.

NtechLab is a team of data scientists and developers passionate about their work and sharing an intention of making the world a better and safer place.

Initially a Russian company, NtechLab has grown into an international business with employees from all over the world, from various ethnic groups. Our target is to make sure that the NTechLab’s video analytics system is the best in world, and we always work on reducing the number of false positives, and now our products use algorithms with 1 false positive per billion (10⁹) comparisons. The additional tools, such as recognition of ethnicity (race) or gender help to avoid bias in facial recognition and ensure the high accuracy of video analytics software for face recognition around the world, taking into account the individual characteristics of the person. So far, at the recent Face Recognition Vendor Test, conducted by the National Institute of Standards and Technology (NIST), NtechLab’s face recognition technology showed the best performance on sophisticated datasets — “wild” and “child” — among all participants for large-scale verification.

Regarding our products, NtechLab has never delivered ethnicity (race) recognition as a functionality or as a scenario for customers. Also, none of our clients has ever included this functionality in the project.

We support comprehensive and transparent dialogue on ethics and regulations regarding the use of face recognition, and we are willing to participate in it, and to contribute with our best practices and knowledge.

Tevian "Ethnicity" Part Of 'Face Classification'

Tevian was founded in 2010 by graduates of Moscow State University, describing its business as "video analysis" focused. In a description for explaining its 'Face Classification' capabilities, Tevian explicitly noted the ability to define individuals by ethnicity. In the example below, a person is classified as "Asian" along with other descriptors such as "Female", "15-25 years", "With glasses" and "Black hair":

In response, Tevian confirmed ethnicity analytics usage, saying it doesn't "work with end-customers" but thought the main use cases were for finding crime suspects (post-incident), retail analytics, and fraud detection (e.g. a bank making sure a customer is not impersonating someone else).

Tevian said the function was "not very popular" and mostly intended as a 'check mark' in a "list of supported algorithms" but conceded "we don't know which of our partners use it":

We provide this functionality along with other algorithms, we don't have any separate license or product for ethnicity classification. So we don’t know, which of our partners use it. According to information we have, it is not very popular functionality. In many cases for our clients and consequently for us it was intended as a 'check mark' in a list of supported algorithms (in comparison with other competitors), but in the end some of our main customers don’t support ethnicity estimation in their VMS solutions, so their end users can’t use it. [emphasis added]

Automated racial profiling would be "frightful" and "dystopia", said Tevian, but doubted it could be used effectively due to poor accuracy and just using face recognition instead:

I think, what frightens the most, is a theoretical use case, when all people are automatically being racially profiled, and law enforcement officers are signaled to check up on ethnic minorities. Fortunately, I don't think it is currently possible because of too broad ethnic groups (which are automatically classified) and low classification precision (which is ok for previously mentioned cases, but is far from enough for this case). But if algorithms improve a lot, it could become theoretically possible, which is frightful. But I think, that in such possible dystopia law enforcement agencies won't use ethnic classification, because they probably will have photos of each person and it would be easier (and more accurate) to use just face recognition. [emphasis added]

VisionLabs' 'Ethnicity Estimator'

VisionLabs is a Russia-founded, Netherlands HQ'd computer vision developer most known for its face recognition. APAC and the Middle East account for 80% of their sales, North America for under 10%, and their partner’s revenue is split 60% commercial revenue 40% government, the company told IPVM.

In a product description for its LunaSDK face recognition solution, VisionLabs explicitly highlights an "Ethnicity estimator" which claims to determine whether a person's race is "Indian", "Asian", "Black", or "White":

Visionlabs said the "ethnicity estimator" was used to track face recognition performance across ethnicities "to guarantee its accuracies", saying it "did not sell a single license" for the product since its inception:

The VisionLabs Ethnicity (race) estimator serves as a way to track the performance of the face recognition engine across various races to guarantee its accuracy. Vision Lab’s face recognition engine quality for various races has been verified by the NIST FRVT Demographic Effects Report.

VisionLabs did not sell a single license of the race estimator starting from its inception. It was developed in the mid-2018.

The race estimator is common feature in most, if not all facial recognition technologies on the market today.

Not a single race estimator license was sold to any commercial or government entity by VisionLabs. [emphasis added]

VisionLabs said "as to our knowledge" Russian authorities "do not use race estimators" from any tech providers, including Visionlabs, saying it has rules "controlling the sale of software for the purpose of discrimination or systemic bias":

As to our knowledge, the Russian authorities do not use race estimators from any of the local (Russian) or global technology providers, including VisionLabs.

As outlined in our Ethics Charter submitted yesterday, we have rules on controlling the sale of software for the purpose of discrimination or systemic bias. As well, we prohibit the use of our technology for military use and require that end-customers not use our products for any military applications [emphasis added]

Race And Ethnicity In Russia

A large percentage of people in Russia – about 20% – are not ethnic Russians. Among this 20% are millions of minorities such as Tatars, Kazakhs, Uzbeks, Chechens, Armenians, Tajiks, Afro-Russians, etc, many who don't have facial features/skin tones that are stereotypically ‘Slavic,’ e.g. below is a Kazakh man and a Russian woman:

Similar to the US, Russian police are regularly criticized by human rights groups and media for ethnic discrimination, e.g. in 2019, Human Rights Watch said police "arbitrarily detained" and "beat" Central Asian migrant workers and in 2017, reports of racial profiling proliferated after a terrorist attack.

"Purpose-Made for Discrimination": Human Rights Groups Concerned

Human rights groups raised concerns over this technology allowing government authorities to track and monitor minorities in Russia and why these companies would offer such capabilities in the first place. Edin Omanovic, advocacy director at Privacy International, said the findings "underline the ugly racism baked into these systems":

The findings underline the ugly racism baked into these systems. Far from being benign security tools which can be abused, such tools are deeply rooted in some of humanity's most destructive ideas and purpose-made for discrimination. [emphasis added]

Matt Mahmoudi, AI researcher at Amnesty International, said this tech means minorities "equated" with "criminality" risk losing rights like the freedom to assemble:

Minorities who might have been equated with a level of deviance or criminality may fall subject to having their rights, for example their freedom to assemble, removed as a result of the extensive usage of this technology [emphasis added]

IPVM has raised similar concerns regarding China manufacturers and their collaboration with the PRC on 'Skin Color' and Uyghur tracking, as building technology that targets ethnic minorities is unethical and should be stopped.

Moscow Officials Respond

Moscow's Department of Technology, which manages the city's surveillance system and uses NtechLabs, responded to Reuters' inquiry talking generically about facial recognition but ignored addressing whether they used ethnicity analytics.

Ethnicity Analytics A Human Rights Risk

Development and offering of ethnicity analytics, as these Russian companies have done, introduces serious ethical issues, with the capability alone raising human rights concerns. Governments outside China adopting ethnicity analytics undermines the trust the public has in video surveillance and is a human rights problem.