Computing the Collection of Good Models for Rule Lists

Multiple equally good AI models can make very different predictions: The paper demonstrates that for the same problem (predicting recidivism using the COMPAS dataset), thousands of rule-based models can perform equally well statistically, yet make different decisions about individuals. This "prediction multiplicity" means that choosing one model over another is often an arbitrary choice with real consequences for people.

Current AI development practices may miss important model diversity: Traditional machine learning approaches find a single "best" model, but this paper shows that examining the full collection of good models reveals large variations in fairness properties across models. Organizations deploying AI systems may be unaware that alternative models with similar accuracy could treat different groups more fairly.

Efficient algorithms now make it practical to explore model alternatives: The CorelsEnum algorithm presented here can enumerate tens of thousands of good rule-list models in about 16 minutes, compared to finding only 40 models in 100 minutes with previous methods. This computational breakthrough makes it feasible for practitioners to actually study model diversity rather than just theoretically acknowledging it exists.

Interpretable models (rule lists) are particularly suitable for responsible AI analysis: Rule lists—simple if-then decision rules—allow both the algorithm and human auditors to understand and compare how different good models make decisions, making it easier to identify fairness concerns and choose models aligned with organizational values.

Responsible AI requires moving beyond single-model evaluation: The findings suggest that responsible AI deployment should include analyzing collections of good models to understand prediction multiplicity and fairness trade-offs, rather than treating the first acceptable model found as the final solution.