ExplainabilitySafetyTransparencycs.CV
Show Why the Answer is Correct! Towards Explainable AI using Compositional Temporal Attention
Nihar Bendre, Kevin Desai, Peyman Najafirad
May 15, 2021
IEEE International Conference on Systems, Man and Cybernetics
8 citations
Key Takeaways
-
Explainability improves safety in critical applications: The paper demonstrates that AI systems used in safety-critical domains like autonomous vehicles and cybersecurity need to show their reasoning, not just provide answers. The proposed model addresses this by explaining why it arrives at particular conclusions, making it more trustworthy for high-stakes decisions.
-
Current AI systems struggle with complex reasoning: The authors note that existing Visual Question Answering models "fail to better complex questions," suggesting that black-box AI systems may not be reliably handling nuanced or multi-step problems—a significant concern for responsible deployment in real-world scenarios.
-
Compositional reasoning enables more reliable AI: By using "compositional temporal attention" to break down complex questions into simpler, interpretable modules, the model achieves better performance on benchmark datasets (VQA2.0 and CLEVR) while remaining transparent about its decision-making process.
-
Transparency and performance can go hand-in-hand: Rather than treating explainability as a trade-off with accuracy, this research shows that building interpretability into the model's architecture actually improves its performance, suggesting that responsible AI design doesn't necessarily compromise effectiveness.