Misfitting With AI: How Blind People Verify and Contest AI Errors

AI visual assistance technologies have significant blind spots: The study found that AI-powered visual assistance tools fail consistently with complex document layouts, diverse languages, and cultural artifacts—gaps that disproportionately affect blind users who depend on these systems for daily access to information.

Blind users employ sophisticated verification strategies that AI designers often overlook: Rather than passively accepting AI outputs, the 26 blind participants used creative non-visual verification methods, including experimenting with the technology, leveraging their own skills, strategically involving sighted people, and cross-referencing multiple devices—suggesting that accessible AI requires understanding how disabled users actually problem-solve.

Current explainable AI (XAI) features are designed without disabled users in mind: The paper critiques existing XAI approaches as "vision-dependent," meaning they rely on visual explanations that don't help blind users verify or understand AI errors, highlighting a fundamental gap in how the field approaches transparency and accountability.

AI systems embed harmful assumptions about disability: The research reveals that AI VAT reflects problematic design assumptions that treat blindness as a deficit to overcome rather than recognizing "disabled ways of knowing" as valuable—a mindset shift necessary for truly responsible AI development.

Responsible AI requires designing for contestation, not just explanation: Participants identified the need for features that allow users to challenge and contest AI errors, suggesting that accessible and responsible AI must empower users to dispute incorrect outputs, not just understand why systems made mistakes.