Designing a Direct Feedback Loop between Humans and Convolutional Neural Networks through Local Explanations

Local explanations (heatmaps) are popular but frustrating for ML engineers: The study found that while engineers value heatmap-based explanations for understanding CNN decisions, the process of manually detecting problems is exhausting and relies too heavily on guesswork rather than systematic methods.

There's a critical gap between diagnosis and fixing: Engineers struggle to translate what they learn from examining explanations into actual model improvements. DeepFuse addresses this by creating a direct feedback loop where engineers can annotate problems and automatically steer the model to avoid similar mistakes.

Interactive tools can make model improvement more efficient: In testing with 12 experienced engineers, DeepFuse enabled participants to create more accurate and "reasonable" models compared to current state-of-the-art approaches, suggesting that better tool design can reduce the labor burden of responsible AI development.

Systematic, case-based reasoning improves practice: Rather than relying on intuition, DeepFuse guides engineers through structured case-by-case analysis of problematic model behaviors, which participants found could meaningfully improve their current workflows and model quality.

Responsible AI requires better human-AI collaboration tools: The research demonstrates that explainability alone isn't enough—practitioners need interactive systems that connect explanation, diagnosis, and correction in a seamless workflow to build more trustworthy AI systems.