Evaluation of Black-Box XAI Approaches for Predictors of Values of Boolean Formulae

Explainability can be objectively measured: Rather than relying on subjective judgments about whether an AI explanation is "good," this research demonstrates that explainability can be formally evaluated using causal reasoning principles, providing a concrete standard for comparing different explanation methods.

Current XAI tools have measurable performance gaps: The study benchmarked state-of-the-art explainability tools against a rigorous standard and found significant variation in their accuracy, suggesting that organizations should evaluate which explanation method works best for their specific use case rather than assuming all tools perform equally.

Specialized tools outperform general approaches: The newly developed B-ReX tool substantially outperformed existing black-box explainability methods on Boolean function prediction tasks, indicating that tailored XAI solutions designed for specific problem types can deliver more reliable explanations than one-size-fits-all approaches.

Tabular data deserves focused attention: By concentrating on tabular data and a well-defined prediction task, the researchers created a controlled environment to rigorously test XAI methods—suggesting that responsible AI deployment benefits from domain-specific evaluation rather than generic benchmarks.