On Identifying Why and When Foundation Models Perform Well on Time-Series Forecasting Using Automated Explanations and Rating

Foundation models for forecasting have significant limitations in real-world applications: The research found that general-purpose foundation models like Chronos and Llama underperform compared to traditional feature-engineered models (like Gradient Boosting) in volatile or sparse domains such as energy and automotive sales, suggesting these popular AI systems should not be universally trusted for all forecasting tasks.

Model performance is highly context-dependent: Foundation models only excelled in "stable or trend-driven contexts" like finance, while struggling with unpredictable data patterns. This finding highlights the critical importance of matching model selection to specific use cases rather than assuming one model works everywhere.

Explainability matters for responsible deployment: The paper emphasizes that understanding why models succeed or fail is essential because "time series forecasting models are increasingly used to generate information that informs real-world actions with equally real consequences." Feature-engineered models provided more interpretable explanations, making them safer choices when decisions affect real outcomes.

Current AI transparency tools are insufficient: Despite using explainable AI (XAI) methods and Rating Driven Explanations, the researchers note that the "opaque nature" of these models remains a "known limitation," suggesting that existing explanation techniques may not fully address the complexity users need to understand before relying on forecasting models.

Users need better guidance on model selection: The research demonstrates that practitioners should evaluate multiple model architectures across their specific domain rather than defaulting to trendy foundation models, requiring more accessible tools to help non-technical users make informed choices about which forecasting system to deploy.