Learning Quantifiable Visual Explanations Without Ground-Truth

Better ways to measure explanation quality: The paper addresses a major challenge in explainable AI—the lack of reliable methods to evaluate whether explanations are actually good. The researchers propose a new metric based on testing what information is truly necessary and sufficient for a model's decisions, which they show aligns better with human judgment than existing evaluation methods.

Explanations can be improved through training: Rather than just extracting explanations from existing models, the researchers demonstrate that explanation quality can be enhanced by training a specialized adapter module on top of any AI model. This approach generates what they call "causal explanations" of how models make decisions without requiring changes to the original model or reducing its performance.

Practical applicability to existing systems: The framework works with "black-box" models—meaning it can be applied to AI systems already in use without needing access to their internal workings or retraining them from scratch. This makes the approach potentially useful for improving transparency in real-world AI applications across different industries.

Balancing two aspects of good explanations: The metric formally evaluates both "sufficiency" (does the explanation include everything important to the decision?) and "necessity" (does it avoid including irrelevant information?). This dual focus helps ensure explanations are neither incomplete nor cluttered with unnecessary details.