ExplainabilityTransparencyTrustworthinesscs.LG
si4onnx: A Python package for Selective Inference in Deep Learning Models
Teruyuki Katsuoka, Tomohiro Shiraishi, Daiki Miwa, Shuichi Nishino, Ichiro Takeuchi
January 29, 2025
arXiv.org
4 citations
Key Takeaways
-
Statistical rigor improves AI reliability: The package addresses a critical gap where common explanation methods like CAM (Class Activation Mapping) and VAE-based anomaly detection identify important image regions, but without statistical testing to confirm these regions are genuinely meaningful rather than artifacts.
-
Type I error control prevents false conclusions: si4onnx implements hypothesis testing with controlled type I error rates, meaning it reduces the risk of incorrectly claiming an AI system has identified something significant when it hasn't—a key requirement for trustworthy AI systems.
-
Bridges explanation and validation: While XAI (Explainable AI) techniques show what a model is looking at, selective inference adds the crucial step of verifying those explanations are statistically sound, addressing a major gap between interpretability and reliability.
-
Practical accessibility across frameworks: By supporting PyTorch and TensorFlow models through the ONNX format, the package makes statistical validation techniques available to practitioners using mainstream deep learning tools, lowering barriers to responsible AI implementation.
-
Applicable to high-stakes domains: The focus on anomaly detection and image analysis suggests relevance for critical applications (medical imaging, security systems) where false positives from unreliable explanations could have serious consequences.