Epistemic Integrity in Large Language Models

LLMs frequently express false information with high confidence: The paper demonstrates that large language models often present inaccurate statements in assertive language, creating a dangerous mismatch between how certain they sound and whether they're actually correct—a problem the researchers call "epistemic miscalibration."

Current methods for detecting this problem are inadequate: The researchers developed a new measurement approach that reduces error rates by over 50% compared to existing benchmarks, suggesting previous tools for identifying when models are overconfident were significantly flawed and missing the scale of the issue.

The misalignment between confidence and accuracy is severe: Human evaluations confirmed that there is a stark gap between how confidently LLMs linguistically present information and their actual accuracy rates, indicating this isn't a minor calibration issue but a fundamental problem affecting user trust.

This poses real risks at scale: Because LLMs are increasingly used as information sources by millions of people, their tendency to state false information confidently could systematically mislead users across many domains—from healthcare to news to education.

Better diagnosis is the first step toward solutions: While the paper focuses on measuring and identifying the problem rather than fully solving it, the researchers frame their framework as essential groundwork for developing more trustworthy AI systems that either express appropriate uncertainty or provide more reliable information.