ComplAI: Theory of A Unified Framework for Multi-factor Assessment of Black-Box Supervised Machine Learning Models

Responsible AI requires multiple dimensions of assessment beyond accuracy: The ComplAI framework evaluates machine learning models across five key areas—explainability, robustness, performance, fairness, and behavior under data drift—recognizing that a model's ability to make correct predictions alone is insufficient for responsible deployment.

High-stakes decisions demand transparency and fairness evaluation: The paper emphasizes that AI systems making consequential decisions about people's lives (such as loan approvals, insurance coverage, and job applications) must be transparent and fair, not just accurate, making multi-factor assessment essential for responsible AI.

A unified "Trust Factor" enables model comparison: Rather than evaluating models in isolation, ComplAI provides a single Trust Factor metric that allows developers to compare different models across multiple responsibility dimensions, facilitating better decision-making about which model to deploy.

Practical tools are needed for responsible AI implementation: The framework is designed as an actionable tool that helps developers connect their models, visualize performance across different responsibility metrics, and identify specific areas for improvement—bridging the gap between responsible AI principles and real-world implementation.

Robustness and drift susceptibility matter for real-world deployment: Beyond fairness and explainability, the framework assesses how models behave when data changes over time, addressing a practical concern that responsible AI systems must remain trustworthy as conditions evolve.