Mechanistic Interpretability of Fine-Tuned Vision Transformers on Distorted Images: Decoding Attention Head Behavior for Transparent and Trustworthy AI

Different parts of AI models have different jobs: Early layers in vision transformers focus on low-level details like edges and text, while deeper layers concentrate on task-critical information. Removing early layers caused only a 0.11% performance drop, but removing deeper layers caused a 0.34% drop—showing that not all components contribute equally to decisions.

AI models can be debugged by examining individual components: By analyzing individual "attention heads" (specialized processing units), researchers identified which parts of the model were focusing on irrelevant information like axis labels and color bars rather than the actual task, enabling targeted improvements to model reliability.

Some AI components specialize in specific tasks, while others handle multiple things: The study found "monosemantic" heads that focused exclusively on relevant features (like chirp patterns in spectrograms) versus "polysemantic" heads that scattered attention across multiple unrelated regions, revealing functional specialization that can be leveraged for transparency.

Understanding model internals helps identify vulnerabilities: By decomposing the transformer into interpretable pieces, researchers could spot which components were distracted by extraneous content, allowing developers to address potential failure points before deployment—a key step toward more trustworthy AI systems.

Mechanistic interpretability makes AI systems more auditable: This approach moves beyond treating AI as a "black box" by revealing how models make decisions at a granular level, enabling stakeholders to verify that models are using appropriate reasoning rather than relying on spurious correlations.