dashi: A Python library for Dataset Shift Characterization to Support Trustworthy AI Development and Deployment

Dataset shifts can severely compromise AI safety: The paper emphasizes that changes between training and test data—whether occurring over time or across different locations—can degrade model performance and compromise data quality, with particularly serious consequences in healthcare where patient safety and fundamental rights are at stake.

Current tools for detecting dataset shifts are inadequate: Despite well-established theoretical foundations for understanding different types of shifts (covariate, prior, and concept shifts), the paper identifies a significant gap: there is a "lack of accessible and comprehensive software tools" to actually analyze and measure these shifts in practice.

dashi provides both detection and characterization capabilities: The library offers two complementary approaches—unsupervised methods that identify where data has shifted using statistical techniques, and supervised methods that measure how those shifts actually affect model performance—allowing organizations to understand both when and why their AI systems might fail.

Multi-source and temporal monitoring is essential for deployment: The tool's ability to track dataset shifts across both time periods and different data sources reflects a key insight: responsible AI deployment requires ongoing monitoring across multiple dimensions, not just one-time validation before launch.

Accessible tools enable proactive risk management: By making dataset shift analysis available as an open-source library, dashi helps organizations move from reactive problem-solving (discovering failures after deployment) to proactive risk management during development and operational stages.