Can Global XAI Methods Reveal Injected Behaviours in LLMs? SHAP vs Rule Extraction vs RuleSHAP

LLMs encode default beliefs that can drive misinformation: The research confirms that large language models contain built-in assumptions (such as "joy is positive" or "math is complex") that can shape how they frame information in ways that amplify misinformation related to valence framing, information overload, and oversimplification.

Current explainability tools struggle to reveal complex behavioral patterns in LLMs: Existing global XAI methods like SHAP and RuleFit were designed for numerical data and fail to adequately detect more complex, non-linear behavioral triggers in language models—with RuleFit missing non-univariate triggers entirely—highlighting a significant gap in our ability to audit LLM behavior.

A new method (RuleSHAP) shows promise for detecting hidden behavioral rules: The researchers developed RuleSHAP, which combines SHAP analysis with rule extraction and achieved 82% improvement over existing methods at identifying injected behavioral triggers, suggesting a practical pathway for surfacing and understanding what hidden "rules" guide LLM outputs.

Behavioral triggers can be deliberately injected and then recovered: By intentionally embedding behavioral triggers of varying complexity into GPT and Llama models via system instructions, the researchers demonstrated that these hidden behaviors can be systematically detected—a finding relevant to both identifying malicious manipulation and understanding unintended model biases.

Better tools for detecting LLM behavior are essential for responsible deployment: The research addresses a critical need: without better methods to extract and understand the implicit rules driving LLM outputs, organizations cannot adequately audit these systems for harmful biases or misinformation-amplifying behaviors before deployment.