GECOBench: A Gender-Controlled Text Dataset and Benchmark for Quantifying Biases in Explanations

AI explanations can inherit gender biases from training data: The paper demonstrates that when language models like BERT are trained on biased data, the explanations they generate for their predictions—not just the predictions themselves—can reflect and perpetuate those biases, a problem that wasn't previously well-understood.

Standard fine-tuning may not automatically fix explanation bias: While fine-tuning a pre-trained model can improve its predictions, the paper shows that explanation quality depends heavily on which layers are fine-tuned, suggesting that standard practices may leave biased explanations intact even when overall model performance improves.

We now have tools to measure bias in AI explanations: GECOBench provides the first rigorous benchmark for testing whether explainability methods (techniques meant to show why AI makes decisions) are themselves biased, enabling organizations to audit their AI systems more thoroughly.

Retraining embedding layers is particularly important: The research found that fine-tuning or completely retraining the embedding layers—the foundational components that represent words—is especially effective at reducing bias in explanations, offering a concrete technical recommendation for practitioners.

Explainability alone doesn't guarantee fairness: The paper highlights a critical gap: even when AI systems provide explanations for their decisions, those explanations can be misleading if they're influenced by hidden biases, underscoring that responsible AI requires scrutiny of both predictions and the reasoning provided for them.