In Defence of Visual Analytics Systems: Replies to Critics

Visual analytics systems face legitimate scientific rigor concerns: The paper identifies 36 common criticisms of VA research through interviews with 24 researchers, suggesting that concerns about research quality in this field are widespread and warrant serious consideration rather than dismissal.

Criticism categories span methodological and practical domains: The criticisms collected range across multiple dimensions of VA work, indicating that challenges to VA systems aren't limited to one area but affect how these tools are designed, evaluated, and deployed—relevant for responsible AI practitioners assessing visualization-based explainability tools.

Researcher responses reveal defensive gaps in the field: By interviewing 17 researchers about responses to criticisms, the authors found that the field's ability to address concerns may be "imperfect, provocative, and controversial," suggesting that visual analytics for AI explainability may have unresolved vulnerabilities that users should be aware of.

The paper advocates for higher scientific standards in VA research: The authors explicitly recommend improving "scientific values and rigor" in VA systems, implying that current standards may be insufficient—an important consideration for organizations relying on visual analytics for AI transparency and decision-making.

Inclusive critical evaluation is needed: The authors encourage readers to approach their findings with "an inclusive and critical eye," acknowledging that their own analysis is imperfect and that the field needs ongoing dialogue rather than settled conclusions about best practices.