The result was generally expected, although the error rates are quite impressive.
The US National Institute of Standards and Technology has launched a series of studies aimed at finding out how well face recognition algorithms work under mask mode. The question is extremely relevant today, and some manufacturers say that they have already developed systems capable of recognizing masked faces. But NIST started in order and in the first study tested 89 algorithms created even before the pandemic began.
The test used more than six million photographs and algorithms had to determine how one image of a person corresponds to another - the most common task in such cases, used, in particular, to unlock smartphones. From a more difficult task - to find a match for one in the entire database - the researchers decided to refuse. Nine variants of masks, differing in shape and color, were digitally superimposed on the images.
As a result, even the best of 89 algorithms, which normally recognized faces with an accuracy of 99.7%, were mistaken when using masks at least 5% of the time. For the majority, however, the error rate ranged from 20% to 50%.
The main reason was the lack of information about the distinctive features of faces, which, in fact, are necessary for algorithms for recognition. At the same time, NIST specialists noticed that the shape and color of the mask affects the level of error - the larger the mask and the higher it covers the nose, the more difficult it is for the algorithm to recognize the image. The number of errors was also higher when using black masks, but, as the authors of the study themselves admit, they did not have enough time to study the "color" issue more thoroughly.
In the next study, NIST specialists intend to test new algorithms that already include the function of recognizing masked faces.