Faithful and Plausible Explanations of Medical Code Predictions

Explainability requires balancing two competing demands: The paper identifies a key tension in medical AI—explanations must be both faithful to how the model actually works AND understandable to doctors. This means explanations can't simply be accurate; they must also make sense to clinical experts who need to trust the system.

Proxy models offer a practical solution for high-stakes settings: Rather than trying to explain complex AI systems directly, the researchers trained a simpler "proxy model" that mimics the original model's predictions while providing clearer explanations. This approach allows hospitals and clinics to maintain predictive accuracy while improving interpretability for doctors reviewing individual cases.

The approach was tested on a real clinical task: The study evaluated their method on assigning ICD codes (medical diagnosis and procedure codes) to clinical notes, demonstrating that explanations from the proxy model accurately reflected how the original model made decisions—an important validation for clinical use.

Both individual and aggregate explanations matter in practice: Domain experts need explanations at two levels: understanding why a specific patient received a particular code assignment, AND understanding overall patterns in how the model behaves across many cases. The paper addresses both needs, which is essential for clinical oversight and quality assurance.

This work addresses a critical barrier to AI adoption in medicine: Without trustworthy explanations, clinicians may reject AI tools regardless of their accuracy. By making model decisions interpretable, this approach helps bridge the gap between technical performance and real-world clinical acceptance.