AI2030 Responsible AI Tools Marketplace

Key Takeaways

Interpretability built into the model itself is feasible: Rather than explaining AI decisions after-the-fact, this research demonstrates that AI systems can be designed to generate their own explanations as part of their normal operation—in this case, by identifying which parts of an image are relevant to answering a question—while maintaining competitive performance on benchmark tasks.
Explainability doesn't require sacrificing accuracy: The paper shows that an "intrinsic" interpretable approach to Visual Question Answering achieves competitive results on the GQA dataset, challenging the common assumption that making AI more transparent necessarily means accepting worse performance.
Human evaluation is essential for validating explanations: The researchers validated their automatically-generated explanations through human assessment and developed quantitative metrics that correlate with human judgment, highlighting that automated metrics alone may not capture whether explanations are actually meaningful to people.
Graph-based approaches can make AI reasoning more transparent: By representing images as graphs and generating subgraphs as explanations, the model makes its reasoning process more traceable—showing which visual relationships and objects it considered when answering questions, rather than treating the decision-making as a "black box."
Standardized evaluation methods for explanations are still developing: The paper's contribution of metrics that align with human assessment of explanation quality addresses a gap in responsible AI—we still need better ways to automatically measure whether AI explanations are actually good and trustworthy.

Abstract

The large success of deep learning based methods in Visual Question Answering (VQA) has concurrently increased the demand for explainable methods. Most methods in Explainable Artificial Intelligence (XAI) focus on generating post-hoc explanations rather than taking an intrinsic approach, the latter characterizing an interpretable model. In this work, we introduce an interpretable approach for graph-based VQA and demonstrate competitive performance on the GQA dataset. This approach bridges the gap between interpretability and performance. Our model is designed to intrinsically produce a subgraph during the question-answering process as its explanation, providing insight into the decision making. To evaluate the quality of these generated subgraphs, we compare them against established post-hoc explainability methods for graph neural networks, and perform a human evaluation. Moreover, we present quantitative metrics that correlate with the evaluations of human assessors, acting as automatic metrics for the generated explanatory subgraphs. Our implementation is available at https://github.com/DigitalPhonetics/Intrinsic-Subgraph-Generation-for-VQA.

Accepted at LREC-COLING 2024

Key Takeaways

Interpretability built into the model itself is feasible: Rather than explaining AI decisions after-the-fact, this research demonstrates that AI systems can be designed to generate their own explanations as part of their normal operation—in this case, by identifying which parts of an image are relevant to answering a question—while maintaining competitive performance on benchmark tasks.
Explainability doesn't require sacrificing accuracy: The paper shows that an "intrinsic" interpretable approach to Visual Question Answering achieves competitive results on the GQA dataset, challenging the common assumption that making AI more transparent necessarily means accepting worse performance.
Human evaluation is essential for validating explanations: The researchers validated their automatically-generated explanations through human assessment and developed quantitative metrics that correlate with human judgment, highlighting that automated metrics alone may not capture whether explanations are actually meaningful to people.
Graph-based approaches can make AI reasoning more transparent: By representing images as graphs and generating subgraphs as explanations, the model makes its reasoning process more traceable—showing which visual relationships and objects it considered when answering questions, rather than treating the decision-making as a "black box."
Standardized evaluation methods for explanations are still developing: The paper's contribution of metrics that align with human assessment of explanation quality addresses a gap in responsible AI—we still need better ways to automatically measure whether AI explanations are actually good and trustworthy.

Abstract

Accepted at LREC-COLING 2024

Intrinsic Subgraph Generation for Interpretable Graph based Visual Question Answering

Key Takeaways

Abstract

Paper Info

Responsible AI Topics

Links

Intrinsic Subgraph Generation for Interpretable Graph based Visual Question Answering

Key Takeaways

Abstract

Paper Info

Responsible AI Topics

Links