Word-Level ASR Quality Estimation for Efficient Corpus Sampling and Post-Editing through Analyzing Attentions of a Reference-Free Metric

Quality estimation can improve transparency in AI systems: The paper demonstrates that NoRefER, a reference-free quality metric, can identify word-level errors in speech recognition without needing a correct transcript, making it a practical tool for understanding where and why AI systems make mistakes.

AI tools can make human work more efficient: By prioritizing which transcriptions need human review, NoRefER helps post-editors focus their effort on the most problematic outputs, potentially reducing the time and cost of manual correction workflows.

Explainability requires analyzing how AI makes decisions: The research shows that examining the attention mechanisms (how the model "focuses" on different parts of input) of quality estimation metrics reveals valuable insights into model behavior, supporting the broader goal of making AI decision-making more interpretable.

Better datasets lead to better AI performance: The paper shows NoRefER can help build higher-quality training datasets by identifying and annotating problematic examples, suggesting that responsible AI development requires careful attention to data quality, not just model architecture.

Reproducibility strengthens responsible AI practices: The authors' commitment to releasing all source code publicly enables other researchers to verify findings and build upon this work, supporting the principle that responsible AI development should be transparent and collaborative.