TrustworthinessAccountabilitySafetycs.AI
Developing trustworthy AI applications with foundation models
Michael Mock, Sebastian Schmidt, Felix Müller, Rebekka Görge, Anna Schmitz, Elena Haedecke, Angelika Voss, Dirk Hecker, Maximillian Poretschkin
May 8, 2024
arXiv.org
3 citations
Key Takeaways
-
Foundation models introduce unique trustworthiness risks that differ from traditional AI systems and must be specifically evaluated when developing applications based on them, rather than applying standard AI testing approaches alone.
-
The EU's AI Regulation creates new compliance requirements for both the foundation models themselves and the applications built on top of them, requiring developers to understand regulatory obligations at both levels.
-
A risk-based, application-specific approach to testing trustworthiness—as outlined in Fraunhofer IAIS's AI Assessment Catalog—can be adapted to foundation model contexts to systematically identify and mitigate potential harms before deployment.
-
Foundation models' broad capabilities across text, speech, and image processing create both opportunities and challenges for responsible AI development, necessitating careful evaluation of how these general-purpose systems perform in specific use cases.
-
Organizations developing AI applications with foundation models should conduct trustworthiness assessments that account for inherited risks from the underlying model while also evaluating application-specific risks and compliance with emerging regulatory frameworks.
Abstract
The trustworthiness of AI applications has been the subject of recent research and is also addressed in the EU's recently adopted AI Regulation. The currently emerging foundation models in the field of text, speech and image processing offer completely new possibilities for developing AI applications. This whitepaper shows how the trustworthiness of an AI application developed with foundation models can be evaluated and ensured. For this purpose, the application-specific, risk-based approach for testing and ensuring the trustworthiness of AI applications, as developed in the 'AI Assessment Catalog - Guideline for Trustworthy Artificial Intelligence' by Fraunhofer IAIS, is transferred to the context of foundation models. Special consideration is given to the fact that specific risks of foundation models can have an impact on the AI application and must also be taken into account when checking trustworthiness. Chapter 1 of the white paper explains the fundamental relationship between foundation models and AI applications based on them in terms of trustworthiness. Chapter 2 provides an introduction to the technical construction of foundation models and Chapter 3 shows how AI applications can be developed based on them. Chapter 4 provides an overview of the resulting risks regarding trustworthiness. Chapter 5 shows which requirements for AI applications and foundation models are to be expected according to the draft of the European Union's AI Regulation and Chapter 6 finally shows the system and procedure for meeting trustworthiness requirements.