Towards Trustworthy AI: Characterizing User-Reported Risks across LLMs "In the Wild"

Real-world risks differ significantly from lab studies: User-reported problems with LLMs "in the wild" don't always match what researchers find in controlled settings, suggesting that companies and researchers need to pay more attention to how people actually experience these systems in daily life rather than relying solely on laboratory testing.

Different AI chatbots have distinct risk profiles: Each major LLM chatbot shows a unique "risk fingerprint"—for example, users report more safety and fairness concerns with GPT, privacy issues with Gemini, and security concerns with Claude—indicating that risks are not evenly distributed across platforms and each system may require tailored safeguards.

Reliability is the most pressing user concern: "Valid and Reliable" risks (meaning the AI produces accurate, consistent information) emerged as the most frequently mentioned problem across all platforms, suggesting this should be a top priority for developers and policymakers focused on trustworthy AI.

Common versus rare risks manifest differently for users: Frequently reported risks like fairness are experienced as direct personal harms to users, while less common risks like privacy and explainability appear as complex trade-offs users must navigate—meaning communication and support strategies should differ based on risk type.

User-centered research is essential for responsible AI: The gap between system-centered laboratory findings and actual user experiences demonstrates that developing trustworthy AI requires listening directly to users about the problems they encounter, not just relying on expert assessments.