What's Wrong with Your Synthetic Tabular Data? Using Explainable AI to Evaluate Generative Models

XAI reveals hidden problems that standard metrics miss: The paper demonstrates that conventional synthetic data quality metrics often provide conflicting results and fail to explain why synthetic data differs from real data. By applying explainable AI techniques to a classifier trained to distinguish real from synthetic data, the authors uncovered issues that standard evaluation techniques overlooked—including unrealistic dependencies and missing patterns.

Feature importance and visualization tools pinpoint specific weaknesses: Rather than just reporting that synthetic data is "different," the paper shows how XAI methods like permutation feature importance, partial dependence plots, and Shapley values can identify exactly which features are problematic and how they deviate from real data patterns, enabling targeted improvements.

Transparency in synthetic data evaluation matters for responsible deployment: The paper emphasizes that understanding why synthetic data fails quality checks—not just whether it passes them—is essential for responsible use. This interpretability is particularly important since synthetic data is increasingly used in sensitive applications like healthcare and finance where unexplained quality issues could propagate downstream harms.

Counterfactual explanations offer actionable insights for improvement: By showing what would need to change in the synthetic data to make it indistinguishable from real data, counterfactual explanations provide concrete guidance for improving generative models, moving beyond diagnostic information toward practical solutions.