Beyond saliency: enhancing explanation of speech emotion recognition with expert-referenced acoustic cues

Current emotion recognition explanations fall short of real understanding: Existing methods highlight which parts of speech recordings matter to AI models, but they don't explain why those parts matter or whether they correspond to actual acoustic features that emotion experts recognize as meaningful indicators of emotion.

Connecting AI decisions to expert knowledge builds trust: The proposed framework links what the AI model focuses on to established acoustic markers of emotion (such as pitch, intensity, or speech rate variations) that speech scientists have identified as genuine emotion indicators, making explanations more credible and interpretable.

"What" and "why" both matter for transparency: Simply showing which spectrogram regions are important is insufficient; explanations must also clarify the magnitude and nature of acoustic cues within those regions to help users understand the reasoning behind the model's emotion classification.

This approach moves toward more trustworthy AI systems: By grounding AI explanations in theory-driven acoustic science rather than purely statistical patterns, the framework provides a foundation for building speech-based emotion recognition systems that users and stakeholders can meaningfully understand and appropriately trust.

Explainability improvements are measurable: Testing on standard datasets demonstrated that this expert-referenced approach produces more understandable and plausible explanations compared to traditional saliency-based methods alone.