Explainable embeddings with Distance Explainer

Embeddings are hard to understand, but now there's a tool to help: Machine learning models often compress complex information into "embeddings"—numerical representations where the meaning isn't obvious to humans. This paper introduces Distance Explainer, a new method that can explain why two pieces of data (like images or image-caption pairs) are considered similar or different by the model, making these "black box" decisions more transparent.

The method works by strategically hiding information: Distance Explainer generates explanations by selectively masking (hiding) parts of the input data and measuring how this affects the model's similarity judgments. By testing many different masking combinations and ranking them by their impact on distance calculations, the method identifies which features actually matter for the model's decisions—similar to how you might figure out which ingredients are essential to a recipe by leaving them out one at a time.

Rigorous testing shows the explanations are reliable: The researchers validated Distance Explainer using established XAI metrics including Faithfulness (do explanations accurately reflect what the model does?), Robustness (are explanations consistent?), and Randomization tests (do they avoid random noise?). Testing on real-world models like CLIP and ImageNet datasets demonstrated the method maintains high quality across these measures, suggesting users can trust these explanations.

This fills a gap in AI transparency: While explainability methods exist for many AI systems, very few address the specific challenge of explaining embedded vector spaces—a common architecture in modern AI. By tackling this gap, the work helps make increasingly important AI systems (like those used in image search and multimodal AI) more auditable and trustworthy.