Attention Gathers, MLPs Compose: A Causal Analysis of an Action-Outcome Circuit in VideoViT

Video AI models develop "hidden knowledge" beyond their training task: The research found that VideoViT models trained simply to classify success vs. failure develop sophisticated internal circuits to represent this outcome information, suggesting AI systems may learn complex semantic representations that aren't directly required for their stated purpose—a finding relevant to understanding what models actually "know" versus what they're designed to do.

AI decision-making involves distributed, redundant processing that's hard to disable: The study revealed that success/failure signals are computed through a division of labor where attention mechanisms gather evidence and MLP blocks compose concepts across multiple layers (5-11). This distributed architecture explains why removing single components doesn't break the model, raising concerns about the difficulty of controlling or auditing AI decision pathways.

Mechanistic interpretability can reveal internal AI reasoning: By reverse-engineering the model's internal circuits using activation patching and ablation techniques, researchers identified exactly which components contribute to specific decisions. This demonstrates that explainable AI methods can move beyond "black box" descriptions to show how models actually process information—critical for trustworthiness and accountability.

Model resilience through redundancy may complicate safety measures: The redundant circuit design means the model can recover partial success signals even when individual components are removed, suggesting that simple safeguards or modifications to AI systems may be insufficient if the underlying computation is distributed across multiple pathways.