Consistency-Guided Temperature Scaling Using Style and Content Information for Out-of-Domain Calibration

Calibration matters as much as accuracy for trustworthy AI: The paper highlights that while most research focuses on improving model accuracy when facing new domains, calibration performance—ensuring a model's confidence matches its actual correctness—is equally critical for systems users can trust.

Temperature scaling alone fails for new domains: The standard calibration technique (temperature scaling) works well when testing on familiar data but breaks down when encountering new domains, because it typically requires validation data from those unseen domains, which isn't available in real-world scenarios.

Overconfidence is the core problem: The researchers identified that models become overconfident when making inconsistent predictions across similar samples, and this overconfidence is the main barrier to proper calibration when facing domain shifts.

Style and content consistency can guide better calibration: By using information about how samples are similar in both their visual style and semantic content, the proposed method (CTS) provides mutual supervision that improves calibration without needing access to the new domain's data beforehand.

Practical solution with no accuracy trade-off: The method achieves better calibration performance using only source domain data while maintaining the same accuracy levels, making it immediately applicable to real-world trustworthy AI systems without additional data collection or model retraining.