Language Models Exhibit Inconsistent Biases Towards Algorithmic Agents and Human Experts

LLMs show contradictory attitudes toward human experts versus algorithms: When directly asked about trustworthiness, the eight language models tested gave higher ratings to human experts, mirroring human psychology. However, when shown actual performance data and asked to make incentivized bets, the models disproportionately chose algorithms—even when those algorithms performed demonstrably worse than the human experts.

This inconsistency mirrors but inverts human "algorithm aversion": While humans are known to distrust algorithmic recommendations (algorithm aversion), LLMs appear to exhibit the opposite bias in practice. This suggests that LLMs may have learned contradictory patterns from their training data about how to evaluate human versus algorithmic decision-makers.

Stated preferences don't predict actual behavior in LLMs: The gap between what LLMs say they trust (humans) and what they actually choose (algorithms) reveals that direct questioning about trustworthiness may not accurately reflect how these models will behave in real decision-making scenarios, raising concerns about relying on stated preferences in AI safety evaluations.

High-stakes decisions require careful consideration of these biases: The paper emphasizes that these inconsistent biases "need to be carefully considered when [LLMs are] deployed in high-stakes" applications, suggesting that organizations using LLMs for important decisions should test actual performance rather than assuming the models will behave consistently with their stated preferences.