AGRI-Fidelity: Evaluating the Reliability of Listenable Explanations for Poultry Disease Detection

Multiple AI models can reach similar accuracy while relying on different clues: The paper reveals that different high-performing AI systems for detecting poultry diseases may focus on entirely different acoustic signals, meaning a model's explanation of why it made a decision isn't automatically trustworthy just because it performs well.

Environmental noise can fool explanation systems: In real farm settings, stationary background sounds like ventilation systems can trick standard explanation metrics into highlighting these irrelevant noises as important diagnostic signals, creating explanations that appear reliable but are actually misleading.

Current explanation evaluation methods have blind spots: Existing techniques for validating AI explanations fail to catch when models are using "shortcuts"—like relying on background noise rather than actual disease indicators—because they don't penalize these redundant patterns.

Cross-model consensus improves reliability: AGRI-Fidelity's approach of comparing explanations across multiple models and checking for agreement helps filter out spurious patterns, providing a more trustworthy assessment of which acoustic signals genuinely indicate poultry disease versus which are just environmental artifacts.

Responsible AI deployment requires specialized evaluation: For real-world applications like agricultural disease detection, generic explanation metrics are insufficient; domain-specific reliability frameworks are needed to ensure AI systems are actually learning the right patterns rather than exploitable shortcuts.