RFEval: Benchmarking Reasoning Faithfulness under Counterfactual Reasoning Intervention in Large Reasoning Models

Nearly half of reasoning outputs are unfaithful: RFEval found that 49.7% of outputs from large reasoning models contain reasoning that doesn't actually drive their answers, even when those answers are correct—a critical gap between what models claim they're doing and what they're actually doing.

Accuracy alone cannot be trusted as a reliability measure: The research demonstrates that a model can produce correct answers with flawed reasoning. Once accounting for model and task differences, the connection between accuracy and faithful reasoning is "weak and statistically insignificant," meaning high accuracy scores may mask underlying trustworthiness problems.

Current AI training methods may undermine reasoning integrity: Adding reinforcement learning objectives to standard fine-tuning actually reduced reasoning faithfulness in some cases, even when accuracy stayed the same—suggesting that how we train these models matters as much as how capable they are.

Math and coding tasks are particularly vulnerable: Unfaithfulness concentrates in "brittle, convergent domains" like mathematics and code, meaning users should be especially cautious about trusting the reasoning in these high-stakes areas where errors have clear consequences.

Trustworthy AI requires new evaluation standards: The paper establishes that auditing AI systems for reliability must go beyond checking if answers are correct—we need to verify that the reasoning process is structurally sound and actually causes the model's conclusions.