A Generative Approach for Semantic Auditing of Electronic Health Records

AI systems in healthcare are only as good as their underlying data: The paper demonstrates that Electronic Health Records often contain inconsistencies with established medical knowledge, yet current quality checks fail to catch these semantic errors—a critical gap since unreliable data directly undermines the trustworthiness of clinical AI systems.

Manual data validation doesn't scale: Traditional approaches to checking medical data quality rely on labor-intensive manual rules that cannot adapt to different contexts or datasets, creating a bottleneck that prevents healthcare organizations from systematically auditing the massive amounts of data that modern AI systems require.

AI can help audit AI: The researchers propose using Large Language Models to automatically generate context-aware tests that identify when observed patient data contradicts established medical evidence, offering a scalable alternative to manual inspection that could be applied across multiple datasets and cohorts.

The methodology identified real problems in existing datasets: When tested on three datasets, the approach generated dozens of quality tests per cohort and found actual discrepancies between what the data showed and what epidemiological evidence predicts—suggesting this method could catch genuine data problems that current systems miss.

Responsible AI requires "semantic" not just "syntactic" validation: Beyond checking whether data is formatted correctly, healthcare organizations need to verify that data makes medical sense—this paper provides an initial framework for that deeper level of scrutiny necessary for trustworthy clinical AI.