DINO-QPM: Adapting Visual Foundation Models for Globally Interpretable Image Classification

Foundation models can be made more interpretable without sacrificing performance: DINO-QPM achieves better accuracy than standard DINOv2 linear probes while providing human-understandable explanations, demonstrating that interpretability and high performance aren't necessarily trade-offs in AI systems.

Visual AI explanations can be grounded in specific image regions: By using average-pooling of patch embeddings rather than standard classification tokens, DINO-QPM can pinpoint which parts of an image (e.g., relevant object parts) the model actually uses for its decisions, making explanations spatially localized and verifiable.

Lightweight adapters offer a practical path to interpretability: Rather than retraining entire models, DINO-QPM works as a lightweight adapter on top of frozen foundation models, suggesting that organizations can add interpretability layers to existing AI systems without expensive retraining or computational overhead.

Reducing background noise improves explanation trustworthiness: The paper's use of sparsity loss to minimize spatial scatter and background noise indicates that interpretable AI systems should actively filter out irrelevant information, helping ensure that explanations focus on genuinely important features rather than spurious correlations.

New evaluation metrics are needed for interpretability claims: The introduction of a "Plausibility metric" alongside other interpretability measures suggests that assessing whether AI explanations are actually trustworthy requires specialized evaluation methods beyond standard accuracy benchmarks.