AccountabilityTransparencyTrustworthinesscs.SE
Human-aligned AI Model Cards with Weighted Hierarchy Architecture
Pengyue Yang, Haolin Jin, Qingwen Zeng, Jiawen Wen, Harry Rao, Huaming Chen
October 8, 2025
SIGSOFT FSE Companion
Key Takeaways
-
Documentation gaps are widespread and costly: The research analyzed 240 open-source projects and found that existing model documentation is inconsistent, incomplete, and imbalanced across platforms, leading to model underutilization and hindering responsible AI adoption.
-
Static documentation isn't enough: Current frameworks like Model Cards and FactSheets rely heavily on qualitative, static information that makes it difficult for users to rigorously compare different models—a key barrier to responsible decision-making about which AI systems to use.
-
A new framework balances multiple dimensions: CRAI-MCF organizes AI model information into eight modules covering technical, ethical, and operational aspects, grounded in Value Sensitive Design principles to ensure documentation reflects human values and real-world needs.
-
Quantitative comparison is now possible: The framework introduces measurable "sufficiency criteria" that enable practitioners to objectively evaluate and compare different LLMs under a unified standard, rather than relying on subjective assessments.
-
Better documentation supports better choices: By providing comprehensive, comparable, and actionable information about AI models, this framework helps practitioners assess, select, and deploy LLMs with greater confidence and integrity—ultimately supporting more responsible AI adoption.