ExplainabilityTransparencyTrustworthinesscs.LG
This looks like what? Challenges and Future Research Directions for Part-Prototype Models
Khawla Elhadri, Tomasz Michalski, Adam Wróbel, Jörg Schlötterer, Bartosz Zieliński, Christin Seifert
February 13, 2025
arXiv.org
3 citations
Key Takeaways
-
Prototype quality remains the central bottleneck: Part-Prototype Models explain decisions by comparing inputs to learned prototypes ("this looks like that"), but current systems struggle with producing high-quality and appropriate numbers of prototypes, which directly undermines both their accuracy and their explanatory value.
-
Interpretability alone isn't enough for adoption: Despite being designed for human understanding, PPMs have not yet become competitive alternatives to post-hoc explanation methods, suggesting that built-in interpretability must be paired with strong predictive performance and practical usability to gain real-world traction.
-
Evaluation standards are inconsistent across research: The survey identifies "non-standardized evaluation" as a key methodological problem, meaning different PPM studies use different metrics and benchmarks, making it difficult to compare approaches and hindering progress in the field.
-
Models need to work across different tasks and contexts: Current PPMs show limited generalization beyond the specific tasks they were trained on, which is a significant barrier to deploying these interpretable systems in diverse real-world applications.
-
Human-centered design is essential for the future: The survey identifies "frameworks for human-AI collaboration" and "aligning models with human concepts" as critical research directions, emphasizing that truly responsible AI requires understanding how humans actually use and interpret these explanations, not just making them technically interpretable.