ExplainabilityTransparencyAccountabilitycs.DB
Explaining Black-Box Clustering Pipelines With Cluster-Explorer
Sariel Ofek, Amit Somech
December 29, 2024
Proceedings of the VLDB Endowment
6 citations
Key Takeaways
-
Clustering results are often unexplainable "black boxes": The paper identifies that existing explainability tools from AI are largely ineffective at helping people understand why clustering algorithms group data the way they do, creating a significant gap in responsible AI practices for this common data analysis technique.
-
A new tool makes cluster explanations practical and efficient: Cluster-Explorer automatically generates clear, rule-based explanations (such as "customers in this cluster have high spending AND frequent purchases") that describe what makes each cluster distinct, addressing a task that previously required manual, time-consuming analysis.
-
Explanations are validated to be both accurate and useful: The tool was tested on 98 real clustering results and through user studies, demonstrating it produces higher-quality explanations faster than existing AI explainability methods, suggesting it could be reliably deployed in practice.
-
Transparency doesn't require sacrificing algorithm choice: Rather than forcing organizations to use only "interpretable-by-design" clustering methods (which may not work well for their data), this approach allows them to use the best clustering algorithm for their needs while still being able to explain the results—balancing performance with accountability.
-
Explainability enables better decision-making with clustered data: By making clustering results understandable to non-technical stakeholders, this tool supports more responsible use of clustering in real-world applications like customer segmentation or data analysis, where decisions based on clusters affect people.