The Impact of Presentation Style on Human-In-The-Loop Detection of Algorithmic Bias

Presentation format significantly influences how decision-makers review algorithmic bias: The study found that when bias detection tools used a recommendation list format, participants were less likely to investigate performance measures beyond those automatically flagged as biases, compared to when visual cues were used—suggesting that how information is presented can narrow or broaden human scrutiny.

Recommendation lists may create tunnel vision in bias detection: Participants using the recommendation list prototype gave "less consideration" to unmeasured performance metrics, indicating that automated suggestions can inadvertently discourage comprehensive human review—a critical concern since important biases might exist outside the tool's detection scope.

Visual cues encourage more thorough investigation: The visual cues prototype prompted participants to investigate a broader range of performance measures, suggesting this presentation style may better support the comprehensive bias detection needed for responsible AI deployment.

One-size-fits-all bias detection tools are insufficient: The researchers propose that bias detection tasks should be characterized along two dimensions—information load and comprehensiveness—implying that organizations should match their presentation style to their specific context rather than relying on a single approach for all scenarios.

Human oversight requires thoughtful interface design: This research demonstrates that the technical capability to detect bias is only part of the solution; how that information is communicated to human decision-makers fundamentally shapes whether they will catch and address algorithmic bias in practice.