AnomalyExplainer Explainable AI for LLM-based anomaly detection using BERTViz and Captum

Visual explanations significantly improve security analyst trust: The framework uses BERTViz and Captum to create visual representations of how the AI model makes decisions, helping security analysts understand why an anomaly was flagged rather than just accepting a black-box prediction—addressing a key barrier to AI adoption in cybersecurity.

RoBERTa substantially outperforms other models for anomaly detection: On the HDFS dataset, RoBERTa achieved 99.6% accuracy and demonstrated superior anomaly detection capabilities compared to larger models like Falcon-7B and Mistral-7B, suggesting that bigger models aren't always better for specialized security tasks.

Explainable AI reduces manual work and speeds response times: By combining visual explanations with natural language reports based on attention outputs, the framework reduces the manual effort required to investigate alerts and accelerates remediation—directly improving operational efficiency in cybersecurity teams.

User testing validates practical value: Security analysts confirmed the chatbot interface was easy to use and that the explanations genuinely improved their understanding of detected anomalies, indicating the framework translates technical XAI advances into real-world usability benefits.

The framework addresses false positives, a major pain point: By providing clear explanations for each detection, the system helps analysts quickly distinguish between genuine threats and false alarms, reducing alert fatigue that undermines trust in AI-assisted security tools.