ExplainabilitySafetyTransparencycs.AI
Fighter Jet Navigation and Combat using Deep Reinforcement Learning with Explainable AI
Swati Kar, Soumyabrata Dey, Mahesh K Banavar, Shahnewaz Karim Sakib
February 19, 2025
International Conference on Unmanned Aircraft Systems
Key Takeaways
-
High task completion rates don't guarantee safe AI: The fighter jet agent achieved over 80% task completion, but this metric alone doesn't reveal whether the AI made decisions that would be acceptable in real-world military applications—highlighting the importance of looking beyond performance numbers when evaluating AI systems.
-
Explainability requires comparing actual choices to alternatives: The researchers used "counterfactual" analysis (comparing what the AI actually did versus what it could have done instead) to understand why the jet made specific decisions. This approach demonstrates that transparency in AI requires more than just observing outcomes; it requires examining the reasoning behind choices.
-
Multi-objective AI systems need careful balance: The AI had to simultaneously navigate, reach targets, and decide whether to engage or evade enemies through a single reward function. This illustrates a real challenge in responsible AI: when systems must optimize multiple goals, the way those goals are weighted can significantly influence behavior and outcomes.
-
Simulation environments have limitations for real-world deployment: The study used a customized Pygame simulation rather than real fighter jet systems. This reminds us that AI trained in simplified environments may not account for real-world complexity, uncertainty, and consequences—an important consideration before deploying such systems in actual military contexts.