Interpretable ML for Imbalanced Data

Imbalanced datasets in high-stakes fields like medicine and autonomous driving create a "double transparency problem": Deep learning models are already difficult to interpret, and when trained on imbalanced data where classes are skewed, the relationships between classes become even more unclear, reducing user trust in critical applications.

Current explainability tools fall short for real-world imbalanced data: Existing XAI techniques focus on converting models to simpler forms or explaining individual predictions, but they don't adequately examine the underlying data complexities (such as class overlap, outliers, and sub-concepts) that are common in imbalanced datasets and crucial for understanding model behavior.

A new framework is needed for modern deep learning on imbalanced data: The paper addresses a gap by proposing techniques designed specifically for large, high-dimensional, multi-class imbalanced datasets—moving beyond older methods that only handled binary classification and shallow learning models.

The framework serves dual audiences: Both model users (who need to understand and trust predictions) and algorithm developers (who need to improve performance) can use these techniques to identify class prototypes, sub-concepts, outliers, and key features that drive model decisions.

Identifying borderline instances is critical: The framework helps detect instances that fall on the boundary between classes, which are particularly important for understanding where models may struggle and where performance is most uncertain.