TransparencyAccountabilityEthicscs.HC
Data Cards: Purposeful and Transparent Dataset Documentation for Responsible AI
Mahima Pushkarna, Andrew Zaldivar, Oddur Kjartansson
April 3, 2022
Conference on Fairness, Accountability and Transparency
393 citations
19 influential
Key Takeaways
-
Dataset documentation is critical for responsible AI: The paper emphasizes that understanding a dataset's origins, development, ethical considerations, and evolution is "a necessary step for the responsible and informed deployment of models, especially those in people-facing contexts and high-risk domains."
-
Data Cards provide structured transparency: The authors propose Data Cards as "structured summaries of essential facts about various aspects of ML datasets" that document key information including upstream sources, data collection and annotation methods, intended use, and decisions affecting model performance—making this information accessible to stakeholders throughout a dataset's lifecycle.
-
Documentation should be user-centered: The paper treats dataset documentation "as a user-centric product in and of itself," recognizing that clear, concise, and comprehensive documentation requires consistency and comparability across datasets to be truly useful for responsible AI development.
-
Complexity increases with scale and multi-modal data: As models become larger and capable of handling multiple types of data (text, images, etc.), "the complexity of understanding multi-modal datasets that give nuance to models rapidly increases," making systematic documentation approaches like Data Cards increasingly necessary.
-
Stakeholders need transparency about data decisions: Data Cards explain the "processes and rationales that shape the data and consequently the models," helping stakeholders understand how upstream choices in collection, annotation, and evaluation methods influence downstream model behavior.