Tests
Fresh research shows that even subtle changes to digital images intended to confuse computer vision systems can also influence human perception
Computers and people see the world in different ways. Our biological systems and artificial systems in machines do not always pay attention to the same visual signals. Neural networks trained to classify images can be completely fooled by subtle image distortions that humans won’t even notice.
Our discovery highlights the similarity between human and machine vision, but also demonstrates the need for further research to understand the impact that adversarial images have on humans as well as artificial intelligence systems.
What is an adversarial image?
An adversarial image is one that has been subtly altered by a procedure that causes the AI model to confidently misclassify the image’s content. This intentional deception is known as an adversarial attack. Attacks can be targeted so that, for example, an AI model classifies a vase as a cat, or they can be designed so that the model sees everything except the vase.
Left: An artificial neural network (ANN) correctly classifies an image as a vase, but when it is perturbed by a seemingly random pattern across the entire image (center) and the intensity is magnified for illustration purposes – the resulting image (right) is incorrect and sure, incorrectly classified as a cat.
And such attacks can be subtle. In a digital image, each individual pixel in the RGB image has a scale of 0–255, representing the intensity of the individual pixels. The adversarial attack can be effective even when no pixel is modulated more than 2 levels on this scale.
Hostile attacks on real-world physical objects can also be successful, such as causing a stop sign to be misidentified as a speed limit sign. Indeed, security concerns have prompted researchers to explore ways to counter adversarial attacks and mitigate their risks.
What effect do opposing examples have on human perception?
Previous research has shown that people can be sensitive to high-intensity image noise that provides distinct shape cues. However, less is known about the effects of more diverse adversarial attacks. Do people ignore image noise as harmless, random image noise, or can it affect human perception?
We then showed participants a pair of photos and asked a targeted question: “Which image looks more like a cat?” Although none of the photos look anything like a cat, participants were forced to make a choice and typically reported feeling like they were making an arbitrary choice. If brain activations are insensitive to subtle adversarial attacks, we would expect people to select each image on average 50% of the time. However, we found that the selection rate – which we call perceptual bias – was well above chance for a wide range of perturbed image pairs, even when no pixel was corrected by more than 2 levels on a scale of 0–255.
From the participant’s perspective, it may seem that he or she is being asked to distinguish between two virtually identical images. However, the scientific literature is full of evidence that people utilize tender perceptual signals to make choices, signals that are too weak to express confidence or awareness ). In our example, we may see a vase of flowers, but some activity in the brain tells us that there is a hint of a cat in it.
Left: Examples of adversarial image pairs. The top pair of images is subtly distorted, at a maximum size of 2 pixels, causing the neural network to incorrectly classify them as “truck” and “cat” respectively. The volunteer is asked, “What looks more like a cat?” The bottom pair of images are more obviously manipulated, at a maximum size of 16 pixels, to be misclassified as “chair” and “sheep”. This time the question is: “What is more like a sheep?”
The importance of research on the safety and security of artificial intelligence
Our central finding that adversarial images can influence – albeit subtly – human perception raises critical questions in the context of AI security research, but by using formal experiments to study similarities and differences in the behavior of AI visual systems and human perception, we can Operate knowledge to build safer AI systems.
For example, our findings could inform future research aimed at improving the reliability of computer vision models by better matching them to human visual representations. Measuring human susceptibility to perturbations can support assess this adaptation for different computer vision architectures.
Our work also demonstrates the need for further research into understanding the broader impact of technology not only on machines, but also on people. This, in turn, highlights the continuing importance of cognitive science and neuroscience to better understand AI systems and their potential impact as we focus on building safer systems.