Incorporating Expert Guidance in Epidemic Forecasting

Expert knowledge can improve AI forecasts while maintaining safety guarantees: The paper demonstrates that incorporating expert guidance into epidemic forecasting reduced prediction errors by up to 17% on test data, showing that human expertise and machine learning can work together effectively rather than as competitors.

AI safety frameworks have practical applications beyond theoretical concerns: By adapting the Seldonian optimization framework—a tool designed for AI safety—to epidemic forecasting, the authors show that formal methods for ensuring responsible AI behavior can be implemented in real-world public health applications.

Systematic incorporation of expert feedback addresses a critical limitation of data-driven models: The paper identifies that while machine learning methods for forecasting have become more sophisticated, they typically cannot systematically integrate expert judgment, which the authors successfully address through their proposed approach.

Probabilistic guarantees on model behavior are achievable: The framework not only improves accuracy but also "bounds the probability of undesirable behavior," meaning decision-makers can have formal confidence that the AI system will behave within acceptable parameters—an important consideration for public health applications where forecasting errors have real consequences.

Domain-specific constraints matter: The study focused on two types of expert guidance (smoothness and regional consistency of errors), suggesting that responsible AI implementation requires understanding what "good behavior" means within specific fields rather than applying one-size-fits-all solutions.