More Questions than Answers? Lessons from Integrating Explainable AI into a Cyber-AI Tool

XAI tools aren't automatically user-friendly for non-experts: Despite being marketed as accessible to non-technical users, popular explanation techniques like SHAP and LIME "are lost in translation" when cybersecurity analysts without AI expertise try to interpret them, suggesting a significant gap between XAI theory and real-world usability.

Current XAI methods slow down critical workflows: Post hoc explanations that focus on individual predictions are too localized and time-consuming for real-time cybersecurity work where "accurate assessment and timeliness are paramount," indicating that current XAI approaches may actually hinder rather than help in high-stakes domains.

Domain experts need different explanations than AI researchers expect: Cybersecurity analysts require "higher-level, easy-to-digest explanations" that integrate smoothly into their existing work processes, not the detailed technical breakdowns that XAI researchers typically prioritize.

There's a mismatch between XAI design and practical deployment: The paper identifies "unaddressed gaps in practical and effective XAI," suggesting that many XAI techniques are developed in laboratory settings without sufficient consideration for how they'll actually be used by professionals in time-sensitive, mission-critical environments.

Large Language Models may offer a path forward: The authors suggest that emerging LLM technology could help bridge current XAI limitations by potentially translating complex AI explanations into more accessible, contextual language that fits naturally into expert workflows.