Why AI-Generated Text Detection Fails: Evidence from Explainable AI Beyond Benchmark Accuracy

High benchmark accuracy masks real-world failure: While the researchers' detection model achieved a competitive F1 score of 0.9734 on standard benchmarks (PAN CLEF 2025 and COLING 2025), performance "degraded significantly" when tested on different datasets and AI generators, revealing that benchmark scores don't guarantee reliable detection in practice.

Detectors exploit dataset quirks rather than detecting AI: Using explainable AI analysis (SHAP), the study found that the most influential features for detection "differ markedly between datasets," indicating that systems are learning dataset-specific stylistic patterns rather than identifying fundamental characteristics of machine-generated text.

The most useful features are also the most fragile: There is a "fundamental tension" in AI text detection—the linguistic features that work best for distinguishing AI text on benchmark data are simultaneously the most vulnerable to domain shifts, formatting changes, and other variations, making them unreliable for real-world deployment.

Current detection systems lack generalization: The research demonstrates "substantial generalisation failure" across different domains and AI generators, suggesting that today's AI-generated text detectors cannot be trusted to work reliably when encountering new types of AI systems or text in different contexts than their training data.

Interpretability reveals the problem: By applying explainable AI techniques rather than relying solely on accuracy metrics, the researchers uncovered that detectors are fundamentally unreliable—a finding that would have remained hidden if only benchmark performance numbers were considered.