FairnessExplainabilityTrustworthinesscs.CY
Adapting SQuaRE for Quality Assessment of Artificial Intelligence Systems
Hiroshi Kuwajima, Fuyuki Ishikawa
July 31, 2019
2019 IEEE International Symposium on Software Reliability Engineering Workshops (ISSREW)
44 citations
6 influential
Key Takeaways
-
Traditional software quality frameworks need significant adaptation for AI systems because AI behavior emerges from training data rather than logical design, making conventional quality assessment approaches ineffective for machine learning applications.
-
New quality dimensions beyond traditional software are essential, including fairness and explainability—concepts that don't apply to conventional systems but are critical for responsible AI deployment in industrial applications.
-
The paper bridges ISO/IEC 25000 (SQuaRE) standards with AI ethics requirements, specifically incorporating the European Commission's Ethics guidelines for trustworthy AI into traditional software quality concepts to create a holistic framework for evaluating AI systems.
-
A comprehensive quality assessment approach must account for both technical ML characteristics and ethical considerations, recognizing that responsible AI requires evaluation criteria that address both how well systems perform and whether they operate fairly and transparently.
Abstract
More and more software practitioners are tackling towards industrial applications of artificial intelligence (AI) systems, especially those based on machine learning (ML). However, many of existing principles and approaches to traditional systems do not work effectively for the system behavior obtained by training not by logical design. In addition, unique kinds of requirements are emerging such as fairness and explainability. To provide clear guidance to understand and tackle these difficulties, we present an analysis on what quality concepts we should evaluate for AI systems. We base our discussion on ISO/IEC 25000 series, known as SQuaRE, and identify how it should be adapted for the unique nature of ML and $\textit{Ethics guidelines for trustworthy AI}$ from European Commission. We thus provide holistic insights for quality of AI systems by incorporating the ML nature and AI ethics to the traditional software quality concepts.