SafetyBiasTrustworthinesscs.AI
Can Risk-taking AI-Assistants suitably represent entities
Ali Mazyaki, Mohammad Naghizadeh, Samaneh Ranjkhah Zonouzaghi, Amirhossein Farshi Sotoudeh
October 9, 2025
arXiv.org
Key Takeaways
-
AI assistants don't consistently match human risk preferences: The study found that popular language models like DeepSeek Reasoner and Gemini-2.0-flash-lite show only partial alignment with how humans actually approach risk decisions, meaning they may not reliably represent human values when used in decision-support systems.
-
Risk behavior in AI systems can be manipulated: The research demonstrates that AI assistants' risk aversion is "manipulable"—meaning their tendency to take or avoid risks can be shifted, which raises concerns about whether users might be inadvertently nudged toward riskier or more conservative choices than they would naturally prefer.
-
Gender and context matter for risk decisions: The study examined how risk preferences vary by gender and across different decision-making roles, finding that AI systems struggle to account for these nuanced human differences, which is important since real-world decisions often depend on context and individual circumstances.
-
Current measurement methods need improvement: The paper identifies gaps in how we measure whether AI systems truly replicate human risk behavior, suggesting that existing evaluation methods ("bio-centric measures") are insufficient for ensuring responsible AI deployment in high-stakes decision contexts.
-
Better AI design is essential for trustworthy decision support: The authors call for advances in how AI systems are built to more accurately mirror human risk preferences, arguing this is necessary before deploying these tools in real-world risk management scenarios where poor alignment could lead to harmful outcomes.