Unboxing Diffusion Models for the Arts: Interactive Model Bending and Practice-Based Explainability

Artists need transparency to use AI as a creative tool, not just a black box: The paper demonstrates that when diffusion models' internal structures are made visible and manipulable, artists can inspect, modify, and debug them as part of their creative process—moving beyond treating AI as an opaque end-to-end tool.

"Model bending" allows predictable creative control: By manipulating specific components within Stable Diffusion 1.5's pipeline, the researchers found that different interventions produce "relatively consistent families of visual effects," enabling artists to develop practical understanding of how different model parts influence generated images.

Practice-based explainability differs from technical explanation: Rather than focusing on how to explain AI to non-experts in technical terms, the paper argues that explainability in creative contexts should prioritize enabling hands-on experimentation and intervention—letting artists learn through doing rather than reading documentation.

Interactive interfaces democratize model exploration: The integration of layer selection and intervention controls into ComfyUI's node-based workflow shows that accessible interfaces can give artists practical tools to understand and shape AI behavior without requiring deep technical expertise.

Responsible AI in creative fields requires material engagement: The research suggests that responsible deployment of large-scale AI systems in arts contexts means designing them to be inspectable and modifiable by practitioners, rather than presenting them as fixed, unchangeable systems.