Grounded but Misleading: Evaluating Semantic Alignment in AI-Generated Security Explanations

Grounding alone doesn't guarantee helpful explanations: The research found that AI-generated security explanations can cite real evidence (appear "grounded") while still weakening the intended risk message. In human tests, risk-diluting explanations scored 3.66/5 on perceived grounding but only 3.00/5 on helpfulness, showing that users can be misled even when explanations reference actual detector findings.

How evidence is framed matters as much as what evidence is cited: The paper demonstrates that the semantic framing of evidence—how it's interpreted and presented—can redirect users away from appropriate risk concerns, even when the underlying facts are accurate. This suggests current AI explanation systems may inadvertently undermine security awareness.

Current evaluation methods miss semantic misalignment: Existing approaches to evaluating AI explanations often focus on whether evidence is cited but fail to check whether that evidence actually supports the intended risk interpretation. The researchers argue that trustworthy AI systems need verification methods that go beyond surface-level grounding checks.

Users struggle to detect misleading but grounded explanations: Human evaluators gave relatively high grounding scores to explanations that actually provided poor reasoning support for security decisions, suggesting people cannot easily distinguish between genuinely helpful and subtly misleading AI-generated explanations without additional safeguards.

This matters for real-world security: As AI systems increasingly help users identify online scams, explanations that appear credible but semantically weaken risk interpretation could leave people vulnerable to social engineering attacks they should have recognized as dangerous.