ExplainabilityTransparencyAccountabilitycs.HC
Understanding Stakeholders' Perceptions and Needs Across the LLM Supply Chain
Agathe Balayn, Lorenzo Corti, Fanny Rancourt, Fabio Casati, Ujwal Gadiraju
May 25, 2024
arXiv.org
10 citations
Key Takeaways
-
One-size-fits-all transparency doesn't work: The study of 71 stakeholders across the LLM supply chain found that different groups have fundamentally different needs regarding explainability and transparency, meaning that generic transparency tools and documentation are insufficient for responsible AI development.
-
Forgotten stakeholders have critical information needs: The research surfaced "often forgotten stakeholders" whose needs are typically overlooked in AI transparency efforts, suggesting that responsible AI practices must actively identify and include voices beyond just developers and researchers.
-
Three key questions must be answered together: Rather than treating explainability and transparency as standalone concerns, the paper emphasizes that responsible design requires simultaneously addressing "who" needs information, "what" information they need, and "why" they need it—a more holistic framework than current practices typically employ.
-
Current explainability research has blind spots: Existing studies and tools for AI explainability fall short because they don't account for the diverse stakeholders in the LLM supply chain, indicating that much of the current responsible AI work may be solving problems for the wrong audiences or missing critical perspectives.
-
Supply chain perspective is essential: By examining stakeholders across the entire LLM supply chain rather than in isolation, the research highlights that responsible AI requires understanding how different groups (from data annotators to end-users) interact with and are affected by these systems.