Attention Is (Not) All You Need

Humans can show AI models were to focus for image recognition

Image by Pexels.

Deep learning AI models are remarkably good at classifying objects in images, often with very high accuracy, only to fail when they are deployed in the real world. They often succeed for the wrong reasons by jumping over where the fence is lowest. For example, if most training images show boats on water, the AI model may learn to associate water with “boat”. Show it a boat on dry land or a toy boat in water and it may fail. This failure is called dataset bias, and it is more common than many realise. Traditional approaches to address this include collecting thousands of new images, using larger AI models, or manually drawing detailed object outlines.

Accuracy in AI models for image classification, such as CNNs, is highly dependent on the dataset. An accuracy above 80% is generally considered good. But ask yourself this question: would you consider the performance of a human recognising a boat in an image 8 out of 10 times, when only shown images containing boats, a good score?

The paper “Efficient Human-in-the-loop System for Guiding DNNs Attention” by Yi He et al., frames this as a human-AI collaboration effort. The researchers designed a system that lets humans correct the model’s attention using simple clicks on object regions. Previous solutions often relies on developing deeper and deeper models, thus spending more computational resources, whereas this approach simply introduces a human-in-the-loop to perform guided correction.

To understand which parts of an image influenced the AI model’s decision, attention heatmaps (called Grad-CAM) can be generated. These heatmaps show the regions where the AI model relied on to make a decision. The proposed human-in-the-loop system builds on this idea by combining human precision with the AI model’s automatic predictions.

The proposed human-guided system includes an interactive click-based interface that allows users to select images and correct the AI model’s attention with simple mouse clicks. The user clicks directly on the image: left-click indicates where the AI model should focus, and right-click indicates where it should not focus. The system then fine-tunes the AI model so that its attention shifts towards the correct region and away from misleading cues.

Interactive click-based interface lets users pick images and improve model attention with simple mouse clicks.

The authors acknowledge that human time can be expensive, sometimes as costly as developing deeper AI models. Therefore, the proposed human-in-the-loop system includes a smart sampling strategy called attention-based active learning. Instead of asking users to annotate random images, the system selects images where the AI model’s attention is most incorrect or most informative. This reduces redundant work and accelerates improvement. In a user study, participants completed correction tasks up to 27% faster using the click-based interface compared to traditional polygon-based annotation.

In experiments across multiple datasets, including the CelebA dataset where the AI model determines whether a person is wearing lipstick, the attention maps initially show the AI model focusing on unrelated facial regions. With guidance through the proposed human-in-the-loop system, the AI model’s attention clearly shifts to the mouth region.

Grad-CAM attention heatmaps before and after human guidance.

The proposed human-AI system achieved following results:

• Improved classification accuracy.
• Reduced annotation time per image.
• Produced more meaningful attention maps.
• Outperformed both random and other standard active learning methods.

Deep learning AI classification models are powerful pattern recognition systems, but they are not self-aware. They optimise statistical patterns, not understanding. The proposed human-in-the-loop system demonstrates that lightweight human guidance can improve how an AI model makes decisions; not by rebuilding it from scratch and not by collecting enormous amount of new data, but simply by teaching it where to look.

This approach reframes the role of humans. Instead of labelling massive datasets from scratch, humans act as targeted instructors by correcting the AI model’s attention.

Further Reading

• Yi He, Xi Yang, Chia-Ming Chang, Haoran Xie, and Takeo Igarashi. 2023. Efficient Human-in-the-loop System for Guiding DNNs Attention. In Proceedings of the ACM Conference on Intelligent User Interfaces. ACM, New York, NY, USA, 294–306. https://doi.org/10.1145/3581641.3584074