Outside the window is 2022. You drive a self-driving car, as usual, through the city. The car approaches a stop sign, which it has passed by many times, but this time it does not stop in front of it. For you, this stop sign is like the others. But for a car, it is completely different. A few minutes earlier, without warning anyone, the attacker had pasted a small plate on the sign, invisible to the human eye, but which technology cannot fail to notice. That is, a tiny sticker on the sign turned the stop sign into something completely different from the stop sign.
All of this may seem incredible. But a growing area of research is proving that artificial intelligence can be tricked into something like this if it sees some tiny detail that is completely invisible to humans. As machine learning algorithms increasingly appear on our roads, our finances, our healthcare system, computer scientists hope to learn more about how to protect them from such attacks - before someone really tries to fool them.
“This is a growing concern in the machine learning and AI community, especially as these algorithms are used more and more,” says Daniel Lode, assistant professor in the Department of Computer and Information Science at the University of Oregon. “If spam passes through or is blocked by several emails, this is not the end of the world. But if you rely on a vision system in a self-driving car that tells the car how to drive without crashing into anything, the stakes are much higher.”
Whether the machine breaks down or gets hacked, the machine learning algorithms that "see" the world will suffer. And so to the car, the panda looks like a gibbon, and the school bus looks like an ostrich.
In one experiment, scientists from France and Switzerland showed how such disturbances could cause a computer to mistake a squirrel for a gray fox and a coffee pot for a parrot.
How is this possible? Think about how your child is learning to recognize numbers. Looking at the symbols one by one, the child begins to notice some common characteristics: some are taller and slimmer, sixes and nines contain one large loop, and eights contain two, and so on. Once they see enough examples, they can quickly recognize new numbers as fours, eights, or triplets - even if, thanks to the font or handwriting, they don't look exactly like any other fours, eights, or threes they've ever had. seen before.
Machine learning algorithms learn to read the world through a somewhat similar process. Scientists feed the computer hundreds or thousands of (usually labeled) examples of what they would like to find on the computer. When the machine sifts through the data - this is a number, this is not, this is a number, this is not - it begins to notice the features that lead to a response. Soon she may look at the picture and say, "That's five!" with high precision.
Promotional video:
Thus, both human children and computers can learn to recognize a vast array of objects, from numbers to cats, from boats to individual human faces.
But, unlike a human child, a computer does not pay attention to high-level details - like the furry ears of cats or the distinctive angular shape of the four. He doesn't see the whole picture.
Instead, it looks at individual pixels in an image - and the fastest way to separate objects. If the overwhelming majority of units have a black pixel at a certain point and a few white pixels at other points, the machine will very quickly learn to determine them by a few pixels.
Now back to the stop sign. By imperceptibly correcting the pixels in the image - experts call this interference "perturbations" - you can trick the computer into thinking that there is, in fact, no stop sign.
Similar studies from the Evolutionary Artificial Intelligence Lab at the University of Wyoming and Cornell University have produced quite a few optical illusions for artificial intelligence. These psychedelic images of abstract patterns and colors are unlike anything to humans, but are quickly recognized by the computer as snakes or rifles. This suggests how the AI can look at something and not see the object, or see something else instead.
This weakness is common in all types of machine learning algorithms. “One would expect every algorithm to have a hole in the armor,” says Yevgeny Vorobeychik, assistant professor of computer science and computing at Vanderbilt University. “We live in a very complex multidimensional world, and algorithms, by their nature, affect only a small part of it.”
Sparrow is "extremely confident" that if these vulnerabilities exist, someone will figure out how to exploit them. Probably someone has already done this.
Consider spam filters, automated programs that filter out any awkward emails. Spammers can try to get around this barrier by changing the spelling of the words (instead of Viagra - vi @ gra) or adding a list of "good words" that are usually found in normal letters: like "aha", "me", "glad". Meanwhile, spammers can try to remove words that often appear in spam, such as "mobile" or "win".
Where can scammers get to one day? A self-driving car deceived by a stop sign sticker is a classic scenario that was thought up by experts in the field. Additional data can help pornography slip through safe filters. Others may try to increase the number of checks. Hackers can tweak the code of malicious software to evade law enforcement.
Attackers can figure out how to create missing data if they get a copy of a machine learning algorithm that they want to trick. But it doesn't have to be to get through the algorithm. One can simply break it with brute force by throwing slightly different versions of email or images at it until they pass. Over time, it could even be used for a completely new model that knows what the good guys are looking for and what data to produce to fool them.
“People have been manipulating machine learning systems since they were first introduced,” says Patrick McDaniel, professor of computer science and engineering at the University of Pennsylvania. "If people use these methods, we may not even know about it."
These methods can be used not only by fraudsters - people can hide from the X-ray eyes of modern technologies.
“If you’re some kind of political dissident under a repressive regime and you want to conduct events without the knowledge of the intelligence services, you may need to avoid automatic observation methods based on machine learning,” says Lode.
In one project published in October, researchers at Carnegie Mellon University created a pair of glasses that can subtly mislead the facial recognition system, causing a computer to mistake actress Reese Witherspoon for Russell Crowe. It sounds ridiculous, but such a technology might come in handy for anyone desperate to avoid censorship by those in power.
What to do with all this? “The only way to completely avoid this is to create a perfect model that will always be correct,” says Lode. Even if we could create artificial intelligence that surpasses humans in every way, the world can still slip a pig in an unexpected place.
Machine learning algorithms are usually judged on their accuracy. A program that recognizes chairs 99% of the time will be clearly better than one that recognizes 6 chairs out of 10. But some experts suggest another way to assess the algorithm's ability to cope with an attack: the harder the better.
Another solution might be for experts to be able to set the pace for programs. Create your own examples of attacks in the lab based on the capabilities of the criminals in your opinion, and then show them to the machine learning algorithm. This can help it become more resilient over time - provided, of course, that the test attacks are of the type that will be tested in the real world.
“Machine learning systems are a tool for thinking. We have to be intelligent and rational about what we give them and what they tell us,”McDaniel said. "We shouldn't treat them as perfect oracles of truth."
ILYA KHEL
