Explanations for Automatic Speech Recognition

Explaining AI transcriptions is harder than explaining image classifications: The paper identifies that assessing whether speech-to-text outputs are correct is more complex than judging image labels, and existing explanation methods weren't designed for variable-length sequences like transcriptions, requiring new approaches to make ASR systems interpretable.

Researchers developed methods to identify which audio segments cause specific transcriptions: The study adapted explanation techniques from image analysis (Statistical Fault Localisation and Causal methods) to pinpoint minimal and sufficient portions of audio that lead to particular transcription outputs, helping users understand what the system "heard."

Explanation quality was tested across multiple commercial and open-source systems: The research evaluated their methods on Google's API, Sphinx, DeepSpeech, and 100 audio samples from the CommonVoice dataset, suggesting the findings apply across different types of ASR systems rather than just one proprietary tool.

Building trust requires moving beyond accuracy metrics: By providing explanations for why an ASR system produced a specific transcription, the paper argues this transparency helps users develop justified confidence in the system—addressing a key gap in responsible AI where users often cannot understand system failures or successes.