ExplainabilityTransparencyAccountabilitycs.LG
From Confusion to Clarity: ProtoScore -- A Framework for Evaluating Prototype-Based XAI
Helena Monke, Benjamin Sae-Chew, Benjamin Fresz, Marco F. Huber
November 11, 2025
Conference on Fairness, Accountability and Transparency
2 citations
Key Takeaways
-
Prototype-based explanations use representative examples to help people understand AI decisions, which is particularly important in high-stakes fields like healthcare, finance, and law where understanding why an AI made a decision is crucial for appropriate trust and fairness validation.
-
There was no standardized way to objectively compare different prototype-based explanation methods, leading to subjective evaluations that hindered progress in the field—ProtoScore addresses this gap by providing a consistent framework for assessment across different data types, especially time series data.
-
The ProtoScore framework integrates established quality properties (Co-12 properties from Nauta et al.) to enable fair comparisons between prototype methods and other XAI approaches, helping practitioners select the most appropriate explanation method without needing expensive user studies.
-
Neural networks' complexity and opacity create real challenges in high-stakes decision-making, making standardized evaluation tools like ProtoScore essential for ensuring that explanation methods actually work as intended before deploying AI systems in sensitive domains.
Abstract
The complexity and opacity of neural networks (NNs) pose significant challenges, particularly in high-stakes fields such as healthcare, finance, and law, where understanding decision-making processes is crucial. To address these issues, the field of explainable artificial intelligence (XAI) has developed various methods aimed at clarifying AI decision-making, thereby facilitating appropriate trust and validating the fairness of outcomes. Among these methods, prototype-based explanations offer a promising approach that uses representative examples to elucidate model behavior. However, a critical gap exists regarding standardized benchmarks to objectively compare prototype-based XAI methods, especially in the context of time series data. This lack of reliable benchmarks results in subjective evaluations, hindering progress in the field. We aim to establish a robust framework, ProtoScore, for assessing prototype-based XAI methods across different data types with a focus on time series data, facilitating fair and comprehensive evaluations. By integrating the Co-12 properties of Nauta et al., this framework allows for effectively comparing prototype methods against each other and against other XAI methods, ultimately assisting practitioners in selecting appropriate explanation methods while minimizing the costs associated with user studies. All code is publicly available at https://github.com/HelenaM23/ProtoScore .