Literary Narrative as Moral Probe : A Cross-System Framework for Evaluating AI Ethical Reasoning and Refusal Behavior

Existing AI ethics tests measure appearance over substance: Current evaluation frameworks primarily assess whether AI systems produce responses that sound ethical rather than demonstrating genuine moral reasoning capacity, meaning many systems may be passing ethics tests through surface-level performance rather than authentic understanding.

Literary narratives reveal what simpler tests miss: By using complex, unresolvable moral dilemmas from science fiction as test material, researchers found that this approach is "structurally resistant to surface performance" - suggesting that nuanced, ambiguous scenarios expose limitations that straightforward ethics questions cannot.

AI systems show consistent behavior regardless of transparency: The study found "zero delta across all 16 dimension-pair comparisons" when comparing how systems performed under blind versus declared conditions, indicating that knowing they're being evaluated doesn't change how AI systems approach moral reasoning.

AI systems fail in distinctive ways that correlate with capability: The research identified "five qualitatively distinct reflexive failure modes," including instances where systems misidentify themselves or falsely claim to have capabilities they lack, with these failure patterns becoming more sophisticated in more capable systems.

Independent verification matters: Multiple AI judges (Gemini Pro and Copilot Pro) showed perfect agreement in ranking system performance, suggesting that using different AI systems to evaluate each other can provide reliable cross-validation of moral reasoning assessments.