Subverting machines, fluctuating identities: Re-learning human categorization

Static identity categories in AI systems can reinforce existing power imbalances. The paper argues that machine learning systems typically treat identity as fixed and essential (e.g., permanently categorizing someone as belonging to a specific demographic group), which "immobilizes existing identity categories and the power differentials that co-occur" rather than recognizing how identity actually changes through social interaction.

Current AI fairness practices may inadvertently perpetuate the problem. The authors critique "emergent AI fairness practices" for continuing to rely on the same static identity paradigm, suggesting that well-intentioned fairness interventions could still lock people into rigid categorical boxes rather than acknowledging identity's fluid nature.

Identity should be understood as relational and dynamic, not fixed traits. Drawing on critical theory, the paper proposes that identity is "a doing rather than a being"—something that forms and changes through ongoing interaction—rather than a collection of permanent attributes that AI systems should simply detect and classify.

AI systems need feedback loops to better reflect how identity actually works. The paper identifies "the absence of iterative feedback to our models" as a key problem, suggesting that AI systems designed with circular, adaptive processes could better capture how human identity continuously evolves rather than remaining static.

Rethinking AI architecture could enable more responsible identity modeling. The authors sketch potential technical approaches like "multilevel optimization and relational learning" that could allow machines to represent identity as "a relationship perpetually in flux," though they acknowledge many open questions remain.