A Finetuned SpeechLLM for Joint Multi-Granular L2 Assessment and Natural-Language Rationales

Multi-level assessment improves interpretability: The model evaluates language proficiency across multiple levels simultaneously—sentence, word, and phoneme—while explaining its reasoning in plain language, addressing a key limitation of traditional automated assessment systems that only assign scores without explanation.

Natural language explanations have reliability limits: While the model's rationales are plausible and trustworthy at the sentence level, the paper finds that explanations become less faithful at the word and phoneme level, with "sparse and weakly aligned" references to specific errors—suggesting caution when relying on AI explanations for detailed feedback.

Hybrid training approach balances performance and interpretability: The combination of supervised fine-tuning and Bounded Direct Preference Optimization allows the system to maintain competitive accuracy while generating explanations, demonstrating that interpretability doesn't necessarily require sacrificing performance.

Validation of AI explanations requires multiple checks: The researchers evaluated rationale quality using two separate measures—self-consistency (whether explanations match predictions) and faithfulness (whether explanations match ground truth)—highlighting that responsible AI deployment should verify that explanations are both internally consistent and factually grounded, not just plausible-sounding.