TrustworthinessSafetyTransparencycs.SE
Developing and Operating Artificial Intelligence Models in Trustworthy Autonomous Systems
Silverio Martínez-Fernández, Xavier Franch, Andreas Jedlitschka, Marc Oriol, Adam Trendowicz
March 11, 2020
Research Challenges in Information Science
31 citations
2 influential
Key Takeaways
-
Trust requires continuous monitoring: The paper identifies that users lack trust in AI systems during adverse or unknown conditions. The authors propose addressing this through ongoing operational monitoring and self-adaptation capabilities, suggesting that trust in autonomous systems isn't built once during development but must be maintained through real-world performance tracking.
-
AI development and software engineering need better integration: A key gap exists between how AI models are developed and how traditional software systems are engineered. The paper advocates for an integrated agile process that coordinates both activities, recognizing that treating AI model development separately from broader system development creates operational problems.
-
One-size-fits-all AI models don't work in practice: The research emphasizes the need for continuous deployment of "context-specific instances" of AI models across distributed autonomous systems, indicating that a single AI model cannot reliably serve all operational environments and that systems must adapt to local conditions.
-
DevOps principles should extend to AI systems: The authors propose a "holistic DevOps-based lifecycle" for AI-based autonomous systems, suggesting that the continuous integration and deployment practices proven effective in software engineering should be adapted and applied to AI model management and operation.
-
Industrial AI faces unique challenges: The paper synthesizes challenges specific to real-world industrial settings—beyond academic environments—highlighting that responsible AI deployment requires addressing practical operational constraints, not just theoretical concerns.