Fairness Evaluation in Text Classification: Machine Learning Practitioner Perspectives of Individual and Group Fairness

How fairness is measured significantly influences which models practitioners choose: The study found that when ML practitioners were presented with different fairness metrics (group fairness vs. individual fairness), it directly changed which models they considered acceptable, suggesting that the choice of evaluation approach is not neutral and can lead to different conclusions about the same model.

Practitioners rely heavily on personal experience rather than systematic frameworks: Rather than using standardized fairness evaluation methods, the 24 ML practitioners studied tended to base their fairness assessments on their own personal experiences and intuitions, highlighting a gap between available fairness toolkits and how practitioners actually evaluate models in practice.

Testing model sensitivity to identity-related language is a key fairness concern: Practitioners focused on understanding how models behaved when identity-related words were added, removed, or changed in text—a practical approach to detecting whether models make unfair predictions based on demographic information.

How practitioners define and group identity categories matters for fairness evaluation: The study found that practitioners formed their own custom groupings of identity tokens (demographic categories) when testing fairness, suggesting that standardized identity groupings may not capture the nuanced ways bias actually manifests in real-world applications.

Current fairness tools may not match how practitioners actually work: The findings suggest that interactive tools designed to help practitioners evaluate fairness in text classification should be redesigned to align with practitioners' actual strategies and workflows, rather than assuming they will adopt formal fairness metrics as intended.