XAI-Units: Benchmarking Explainability Methods with Unit Tests

Different explanation methods often disagree on what's important: The paper reveals that various feature attribution methods—tools designed to explain AI decisions—frequently produce conflicting importance scores for the same model input, making it unclear which explanation to trust without independent verification.

We now have a testing framework for explanation methods: XAI-Units provides a standardized benchmark with known "correct answers" by using synthetic models with transparent internal logic, similar to how software engineers use unit tests to verify code quality, enabling objective comparison of different explanation approaches.

Explanation methods fail in predictable ways: The benchmark tests how well explanation methods handle specific model behaviors like feature interactions (when inputs work together), cancellations (when features offset each other), and discontinuous outputs, revealing that different methods have distinct blind spots.

Synthetic testing enables reliable evaluation: By creating procedurally generated models with fully understood mechanisms, researchers can establish clear expectations for what good explanations should look like, moving beyond subjective assessments toward objective performance measurement.

This addresses a critical gap in responsible AI: The paper tackles an important problem for trustworthy AI deployment—currently, organizations cannot reliably determine which explanation method will provide the most accurate insights into their model's decision-making, which is essential for accountability and detecting bias.