SketchXAI: A First Look at Explainability for Human Sketches

Sketches offer a unique opportunity for explainability research because they are fundamentally different from photos or text—they're built stroke-by-stroke, allowing researchers to study how AI systems understand the sequential, human-centered process of creating visual content.

The paper introduces "stroke location inversion" (SLI) as a new explainability method for sketch-based AI, analogous to existing techniques like heat maps for photos. SLI reveals which strokes are most important to the model's decisions by testing how well the network can reconstruct where strokes should be located in unseen sketches.

Stroke properties matter for explainability: The researchers designed their AI system to specifically account for three key characteristics of sketches—the shape, location, and order of strokes—which are properties that traditional image recognition systems don't need to consider, suggesting that explainability methods should be tailored to how data is actually created.

Better explainability can coexist with improved performance: The sketch-specific design not only advanced explainability but also achieved the best sketch recognition accuracy to date while using fewer computational parameters than previous approaches, suggesting that human-centered AI design principles may benefit both interpretability and efficiency.

This work establishes a foundation for a new research area: By being "the very first" to apply XAI principles to human sketches, the paper opens the door to studying how people and AI can better communicate through a natural, intuitive form of human expression.