Cross-Dialect NER for Bangla Regional Dialects Using Leave-One-Dialect-Out Cross-Validation and Explainable AI

Regional dialects significantly impact AI performance: The study found that models trained on Standard Bangla struggle with regional dialects, with performance varying dramatically across regions—achieving 97.26% accuracy on Mymensingh dialect but only 82.38% on Chattogram, demonstrating that "one-size-fits-all" language AI models may fail for speakers of non-standard varieties.

AI systems rely too heavily on surface-level patterns: Using explainability tools (LIME), researchers discovered that the AI model makes decisions based primarily on the word itself rather than understanding context from surrounding sentences, suggesting the system may not truly comprehend language meaning in the way humans do.

Testing methodology matters for fairness: The "Leave-One-Dialect-Out" testing approach—where models are evaluated on dialects they've never seen during training—revealed real-world performance gaps that standard testing methods might miss, highlighting the importance of rigorous evaluation for responsible AI deployment.

Multilingual models show promise but aren't perfect: While Multilingual-E5 outperformed other systems, even the best model struggled significantly with Chattogram dialect, indicating that current AI technology has meaningful limitations when serving linguistically diverse populations.

Explainability is essential for understanding AI limitations: By making the model's decision-making process visible, researchers identified a critical weakness (over-reliance on word form) that could inform improvements and help practitioners understand when and why the system might fail.