Multimodal Composite Association Score: Measuring Gender Bias in Generative Multimodal Models

Gender bias is embedded in popular image generation models: The research found that both DALL-E 2 and Stable Diffusion contain gendered associations—meaning these widely-used tools systematically link certain concepts to specific genders, reflecting biases from their internet-sourced training data.

Current bias auditing methods don't work well for modern AI systems: Traditional approaches to measuring bias are too time-consuming and impractical for large multimodal models (systems that process both text and images) that can accept unlimited types of inputs, highlighting a gap in how we evaluate newer AI technologies.

A new measurement tool makes bias detection more practical: The Multimodal Composite Association Score (MCAS) offers a scalable, accessible method for measuring gender bias across different types of generative models, making it easier for researchers and companies to audit their systems without requiring extensive manual review.

The problem affects multiple popular platforms: Since both DALL-E 2 and Stable Diffusion showed gendered associations, this suggests the issue is widespread across leading image generation tools that millions of people use, potentially influencing the content these systems create.

Better bias measurement is essential for responsible AI development: By providing tools to quantify and identify biases in multimodal models, this research supports more systematic approaches to understanding and addressing fairness issues before these systems are deployed to the public.