XAutoML: A Visual Analytics Tool for Understanding and Validating Automated Machine Learning

Trust in AI requires transparency: The paper found that users don't trust AutoML systems despite their competitive performance, highlighting that accuracy alone is insufficient—people need to understand how and why a system made its decisions to have confidence in it.

Different experts need different explanations: A requirements analysis with 36 professionals (domain experts, data scientists, and AutoML researchers) revealed "vastly different expertise in ML" and varying informational needs, suggesting that responsible AI tools must be designed to serve diverse audiences, not just technical specialists.

Visual explanations significantly improve understanding: User study participants using XAutoML achieved "significantly increased understanding of ML pipelines produced by AutoML," demonstrating that interactive visualizations combined with explainability techniques can make complex automated systems more comprehensible to non-experts.

Transparency of the entire process matters: The tool explains not just the final ML pipeline but "the complete AutoML procedure," suggesting that responsible AI requires visibility into optimization choices and trade-offs, not just the final model's predictions.

Explainability tools enable broader participation: By integrating with JupyterLab and allowing experienced users to create custom visualizations, the approach demonstrates how transparency tools can empower different stakeholders to validate and understand AI systems according to their specific needs.