Responsible AI Considerations in Text Summarization Research: A Review of Current Practices

Limited stakeholder consideration: The review of 333 summarization papers found that "relatively few papers engage with possible stakeholders or contexts of use," which prevents researchers from identifying potential harms to real-world users and communities affected by their systems.

Responsible AI discussions are rare in summarization research: Despite increased encouragement from AI venues to address ethical considerations, text summarization—a widely-used NLP task—has been "largely overlooked by the responsible AI community," indicating a significant gap between general AI ethics discussions and specific application areas.

Mismatch between research goals and actual outcomes: The analysis identified "mismatches between stated and realized research goals," suggesting that researchers may claim to address certain objectives but fail to follow through in their actual work, potentially misleading readers about a system's capabilities or limitations.

Evaluation practices don't capture real-world impacts: Current evaluation methods in summarization research appear insufficient for understanding "downstream adverse impacts," meaning papers may report positive technical results without assessing whether systems actually work fairly and safely for all users in practice.

Limited discussion of work limitations: The review found inadequate discussion of both "limitations of prior work and their own work," which means potential problems with summarization systems—such as bias or failure modes—may not be transparently communicated to practitioners who might deploy these systems.