Do LLMs trust AI regulation? Emerging behaviour of game-theoretic LLM agents

LLM agents show more distrust than expected: When embedded in game-theoretic models, LLMs tend to adopt "pessimistic" and defective stances more readily than traditional game-theoretic agents would predict, suggesting AI systems may be naturally inclined toward non-cooperation in regulatory scenarios.

User trust is critical but fragile: The research finds that full user trust can incentivize effective regulation, but "conditional trust" (partial or skeptical trust) may actually undermine the regulatory system, indicating that maintaining strong public confidence in AI governance is essential for its success.

Reputation and trust create a virtuous cycle: Establishing positive feedback between user trust and regulator reputation appears to be the key mechanism for encouraging AI developers to build safer systems, suggesting regulatory frameworks should prioritize transparency and demonstrated competence.

Results vary by LLM model: The specific LLM used in testing affects the level at which trust emerges in the system, indicating that AI regulation strategies may need to account for differences between different AI systems rather than applying one-size-fits-all approaches.

Implications for AI-assisted regulation: The findings provide guidance for designing AI regulation systems and offer caution about using strategic LLM agents to aid regulation itself, since their inherent tendency toward distrust could bias regulatory outcomes.