Decompose the model: Mechanistic interpretability in image models with Generalized Integrated Gradients (GIG)

We can now "open the black box" of image recognition AI: This research introduces a method to trace exactly how image models process information from input to output across entire datasets, moving beyond previous approaches that only explained individual predictions. This represents a significant step toward understanding the complete internal mechanics of how these systems work.

The method identifies interpretable "concept vectors" that reveal what image models actually learn: By decomposing model layers into Pointwise Feature Vectors and Effective Receptive Fields, researchers can extract meaningful concepts that the model uses to make decisions—offering insight into whether models learn features humans would consider relevant or potentially problematic.

This approach enables dataset-wide auditing rather than case-by-case analysis: Unlike existing methods that focus on explaining single decisions, Generalized Integrated Gradients (GIG) allows researchers to systematically analyze model behavior across entire datasets, making it feasible to identify patterns of bias or unexpected decision-making at scale.

Better interpretability supports more responsible AI deployment: By understanding the "semantic significance" of how image models operate, organizations can more confidently audit systems before deployment, identify potential failure modes, and ensure models are making decisions based on appropriate features rather than spurious correlations or biases.