Deletion and Insertion Tests in Regression Models

Feature importance rankings need rigorous testing: The paper demonstrates that insertion and deletion tests provide a quantifiable way to evaluate whether explainability methods correctly identify which features actually drive a model's predictions, moving beyond subjective assessments of importance.

Different explanation methods have measurably different performance: When tested on real datasets, Kernel SHAP performed best at identifying truly important features, but integrated gradients achieved nearly equivalent results while being significantly faster to compute—suggesting a practical trade-off between accuracy and computational cost that organizations must consider.

Popular ranking methods don't always align with optimal feature ordering: The finding that sorting variables by Shapley values (a theoretically-grounded approach) doesn't necessarily produce the best feature ordering for insertion-deletion tests reveals that even principled methods may not capture what matters most for understanding predictions in practice.

Technical choices matter for specific data types: The paper identifies that integrated gradients struggles with binary variables because it must interpolate between discrete values, highlighting how explanation methods require careful adaptation depending on the types of features in your data—a practical consideration for responsible AI implementation.

Regression models need tailored evaluation approaches: The paper extends insertion-deletion testing from classification to regression problems, emphasizing that explainability methods and their evaluation criteria must be specifically designed for the task at hand rather than simply adapted from other domains.