Explained, yet misunderstood: How AI Literacy shapes HR Managers' interpretation of User Interfaces in Recruiting Recommender Systems

Current recruiting dashboards lack meaningful transparency: The study found that dashboards used in practice do not adequately explain AI results and keep AI elements opaque, raising concerns about how recruitment decisions are currently being made without proper visibility into AI reasoning.

Explainability doesn't automatically improve understanding: While adding explainable AI (XAI) features made dashboards feel more helpful and trustworthy to users, it did not actually increase HR managers' objective understanding of how the AI system worked—and in some cases made accurate understanding worse, particularly with complex explanations.

AI literacy matters significantly: The benefits of AI explanations depend heavily on users' existing AI knowledge. Only HR managers with high AI literacy benefited from feature importance overlays, suggesting that one-size-fits-all explanations won't work across different user groups.

Complexity can backfire: More complex explanation styles (like counterfactuals and model criteria) did not help users understand AI recommendations better, indicating that adding more information isn't necessarily the solution to improving transparency in recruitment systems.

Training and tailored approaches are essential: The research emphasizes that organizations need both targeted AI literacy training for HR managers and customized explanation strategies suited to different user knowledge levels to ensure fair and transparent AI adoption in hiring decisions.