Epistemic Trustworthiness in Generative AI: A Normative Framework for Warranted Reliance in High-Stakes Workflows

Epistemic trustworthiness requires three distinct conditions beyond accuracy: The paper argues that current AI evaluation frameworks focusing on accuracy, fairness, explainability, and safety miss a crucial question—whether users are justified in relying on AI outputs for their own reasoning. The authors propose three jointly necessary conditions: epistemic humility (representing limits), epistemic access (enabling inspection and contestation), and resistance to epistemic injustice (recognizing users as legitimate knowledge holders).

AI systems must communicate their competence limits, not just their outputs: Epistemic humility requires generative AI to actively represent and communicate what it cannot do reliably. This goes beyond standard confidence scores or disclaimers—it demands that systems help users understand the boundaries of their competence in context-specific ways, particularly in high-stakes domains like legal reasoning and medicine.

Users need meaningful ability to question and contest AI decisions: Epistemic access means systems must enable users to inspect, question, and challenge outputs in real-world workflows. This is distinct from explainability alone; it requires that professionals can actually engage with and contest AI reasoning within their professional context, rather than being presented with outputs as black boxes.

AI deployment risks marginalizing professional expertise and lived experience: The resistance to epistemic injustice condition warns that AI systems can undermine users' recognition as legitimate knowledge holders—particularly problematic when AI outputs override professional judgment or patient/client knowledge in high-stakes settings like healthcare and law.

Warranted reliance differs fundamentally from user trust: The framework distinguishes between whether users feel they can trust AI (behavioral trust) versus whether they are in doing so (epistemic warrant), addressing a critical gap in responsible AI governance for professional contexts.