HOLMES: HOLonym-MEronym based Semantic inspection for Convolutional Image Classifiers

HOLMES provides "what and where" explanations: Unlike traditional saliency methods that only show where a model looks, HOLMES identifies both the specific parts (meronyms) a CNN focuses on and their locations in images, offering more complete transparency into classification decisions.

Part-based decomposition improves interpretability: By breaking down object categories into their component parts (e.g., identifying "wheels" and "engine" for "car"), HOLMES makes CNN decisions more understandable to non-experts without requiring densely annotated training data.

Individual parts significantly impact predictions: The research found that removing a single part typically reduces model confidence by roughly 50%, demonstrating that HOLMES successfully identifies features the model genuinely relies on rather than spurious correlations.

Automated construction reduces annotation burden: HOLMES uses ontologies, web scraping, and transfer learning to automatically build part-detectors, avoiding the need for extensive manual labeling that typically limits explainability research to small datasets.

Feasibility demonstrated across object categories: Testing on diverse object types (animals, tools, vehicles) showed the approach works broadly, with explanations typically including at least two identifiable parts per classification, suggesting practical applicability across different domains.