Generative Responsible AI Data Evaluation Schema (GRAIDES) for AI Assurance in Local Government

Fragmented evaluation data is a major barrier to AI trustworthiness: The paper identifies that evaluation data for AI systems is currently "fragmented across systems, inconsistently structured and difficult to compare," making it hard for organizations to assess whether AI systems are actually safe and performing well.

A standardized framework can improve AI accountability: GRAIDES provides an open-source data model that centralizes how organizations track and measure AI performance, enabling more consistent and reproducible evaluation practices across different AI vendors and systems.

Human-AI alignment matters for responsible deployment: The Westminster City Council case study demonstrates the importance of measuring whether AI systems agree with human evaluators, and importantly, detecting when there is "systematic disagreement between evaluators"—revealing potential biases or inconsistencies that could affect fairness.

Local governments can implement practical AI assurance: The paper provides concrete "blueprints for code, architecture and statistical evaluation," suggesting that even resource-constrained organizations like local councils can establish systematic approaches to monitoring and assuring their generative AI systems.

Measurable evidence is essential for public trust: The research emphasizes that "trust in the application of generative AI relies on well-governed measurable evidence of performance and safety," positioning transparent, data-driven evaluation as foundational to responsible AI governance rather than optional.