An AI Implementation Science Study to Improve Trustworthy Data in a Large Healthcare System

Data quality is foundational to trustworthy AI in healthcare: The study demonstrates that before AI systems can be reliably deployed in clinical settings, healthcare organizations must systematically evaluate data quality across multiple dimensions—including missing data patterns, redundancy, timeliness, and consistency—using frameworks like METRIC that embed trustworthy AI principles from the ground up.

Bridging the research-to-clinic gap requires practical infrastructure: Shriners Children's modernized its data warehouse to a standardized format (OMOP CDM v5.4) and built Python-based quality assessment tools compatible with their existing systems, showing that real-world AI implementation demands investment in infrastructure that works within actual organizational constraints, not just theoretical best practices.

Hybrid implementation strategies outperform one-size-fits-all approaches: The research compared systematic infrastructure-wide AI strategies with use-case-specific approaches (tested on Craniofacial Microsomia diagnosis) and found that blending both methods—rather than choosing one—better advances AI adoption while maintaining trustworthiness and clinical relevance.

Regulatory compliance and clinical adoption are interconnected challenges: The study identifies that strict regulations, gaps between research environments and real clinical settings, and difficulties in evaluating whether AI systems actually work in practice all create barriers that must be addressed simultaneously, not sequentially, for successful implementation.

Standardized data formats enable trustworthy AI evaluation: By adopting the FHIR standard and OMOP common data model, the healthcare system created a foundation where data quality and AI performance could be systematically assessed and compared across different clinical use cases.