Evaluating the Impact of Explainable AI on Trust in AI-Assisted Code Review

Explanations increase trust but may encourage healthy skepticism: When developers received detailed explanations alongside AI code review suggestions, they reported the highest trust levels (3.99 out of 5), but this didn't translate to the highest agreement with recommendations. This suggests that understanding why an AI system made a decision helps developers trust it, while also empowering them to critically evaluate whether that decision is actually correct.

Moderate explanations may offer the best practical balance: Code review systems providing moderate explanations (feedback without detailed reasoning) achieved the highest agreement rate at 89.22%, suggesting this level of transparency may be optimal for real-world deployment—offering enough context for developers to make informed decisions without overwhelming them.

Lack of transparency undermines both trust and reliance: Systems with no explanations resulted in the lowest levels of both trust and agreement, indicating that developers are reluctant to accept AI recommendations when they cannot understand the reasoning behind them.

Explainable AI doesn't significantly impact efficiency: The study found that explanation level did not significantly affect the time developers spent reviewing code, suggesting that adding transparency features to AI-assisted code review tools doesn't create a meaningful productivity burden.

One-size-fits-all transparency may not be ideal: The divergence between trust and agreement across different explanation levels suggests that developers may need customizable transparency options rather than a single standardized approach to AI explanations in code review.