Enhancing Deep Neural Network Reliability with Refinement and Calibration

Confidence scores don't always reflect reality: Deep neural networks often express high confidence in predictions they get wrong, which can mislead users into trusting unreliable decisions. This paper addresses a critical gap between what models claim to know and what they actually know correctly.

There's a trade-off between two types of reliability: The research identifies that many existing methods improve "calibration" (how well confidence matches actual correctness) but worsen "refinement" (the ability to distinguish between correct and incorrect predictions). RefCal's key innovation is optimizing both simultaneously rather than sacrificing one for the other.

Post-processing tricks can be misleading: Some calibration improvements are merely cosmetic—adjusting confidence scores after training without genuinely improving the model's understanding. This paper emphasizes the importance of building true reliability into models during training, not just adjusting outputs afterward.

Significant real-world performance gains are possible: On imbalanced datasets (a common real-world scenario), RefCal achieved an Expected Calibration Error of 0.08 compared to 0.22 for existing methods—a nearly 3x improvement—while maintaining competitive accuracy, demonstrating that reliability improvements don't require sacrificing performance.

Responsible AI requires measuring what matters: The paper highlights that practitioners should evaluate multiple reliability metrics (accuracy, calibration, and refinement) rather than relying on any single measure, ensuring AI systems are genuinely trustworthy rather than appearing trustworthy.