EPT Benchmark: Evaluation of Persian Trustworthiness in Large Language Models

Safety is a critical vulnerability across major LLMs: The EPT Benchmark found "significant deficiencies in the safety dimension" when evaluating leading models like ChatGPT, Claude, Gemini, and others, indicating that even widely-used AI systems may not adequately protect users from harmful outputs.

Cultural values matter for AI trustworthiness: The research demonstrates that evaluating LLMs requires "culturally grounded evaluation criteria" beyond generic benchmarks, as models trained on Western-dominant datasets may not align with Persian ethical and cultural values—a principle applicable to other non-Western languages and cultures.

Trustworthiness requires multi-dimensional assessment: Rather than focusing on a single metric, the EPT Benchmark evaluates six interconnected aspects (truthfulness, safety, fairness, robustness, privacy, and ethical alignment), suggesting that responsible AI requires comprehensive evaluation across multiple dimensions of trustworthiness.

Human evaluation remains essential: The study combined both "automated LLM-based and human assessments," highlighting that algorithmic evaluation alone is insufficient and that human judgment—particularly from culturally informed evaluators—is necessary to catch nuanced failures in trustworthiness.

Transparent benchmarking supports accountability: By making their dataset publicly available, the researchers enable independent verification and ongoing evaluation of model trustworthiness, demonstrating how open-source benchmarks can advance responsible AI development across different languages and cultural contexts.