AI2030 Responsible AI Tools Marketplace

Key Takeaways

Feedback timing matters for human reflection: The research distinguishes between two types of reflection—thinking through decisions as they happen versus reviewing them afterward. Current AI design focuses heavily on the first type, but this study shows that feedback after decisions are made (reflection-on-action) is equally important for improving human-AI teamwork.
Simple feedback can backfire: When feedback only pointed out when humans overrode AI suggestions, it reduced how carefully people thought through their decisions in the moment. This suggests that one-sided feedback interventions, while well-intentioned, may have unintended consequences that undermine other aspects of good decision-making.
Balanced feedback preserves multiple thinking processes: The study found that "balanced feedback"—which presumably addressed both overrides and appropriate reliance on AI—maintained both types of reflection without the negative side effects. This offers a concrete design principle: feedback should be comprehensive rather than narrowly targeted.
Team performance doesn't always improve with behavior change: Interestingly, while feedback changed how much humans relied on AI, overall team performance remained unchanged in some cases. This highlights that the goal shouldn't simply be changing human behavior, but ensuring those changes actually lead to better outcomes in real-world applications.
Context matters for AI collaboration: The research identifies "context-dependent effects," meaning the same feedback intervention works differently depending on the situation, underscoring that responsible AI design requires careful consideration of specific domains and decision-making environments rather than one-size-fits-all solutions.

Abstract

As artificial intelligence (AI) increasingly supports human decision-making in high-stakes domains, designing effective human-AI collaboration becomes critical. Successful collaboration requires humans to reflect on decision-making processes and adjust their strategies accordingly. Current research typically investigates interventions, such as explainable AI, that target reflection during the decision-making process (reflection-in-action). Less attention has been given to retrospective examination of decisions (reflection-on-action). This work explores reflection-on-action by studying the impact of feedback on reliance behavior and human-AI team performance. Specifically, we investigate feedback interventions targeting overrides of AI advice through a behavioral study (N=244). We find that such feedback influences reflection-on-action, while unidirectional feedback risks reducing subsequent reflection-in-action. Furthermore, we identify opposing shifts in reliance behavior that leave overall human-AI team performance unchanged but show context-dependent effects. Balanced feedback did not show these risks, preserving both concurrent and retrospective reflection, offering insights for designing for appropriate reliance on AI.

Key Takeaways

Feedback timing matters for human reflection: The research distinguishes between two types of reflection—thinking through decisions as they happen versus reviewing them afterward. Current AI design focuses heavily on the first type, but this study shows that feedback after decisions are made (reflection-on-action) is equally important for improving human-AI teamwork.
Simple feedback can backfire: When feedback only pointed out when humans overrode AI suggestions, it reduced how carefully people thought through their decisions in the moment. This suggests that one-sided feedback interventions, while well-intentioned, may have unintended consequences that undermine other aspects of good decision-making.
Balanced feedback preserves multiple thinking processes: The study found that "balanced feedback"—which presumably addressed both overrides and appropriate reliance on AI—maintained both types of reflection without the negative side effects. This offers a concrete design principle: feedback should be comprehensive rather than narrowly targeted.
Team performance doesn't always improve with behavior change: Interestingly, while feedback changed how much humans relied on AI, overall team performance remained unchanged in some cases. This highlights that the goal shouldn't simply be changing human behavior, but ensuring those changes actually lead to better outcomes in real-world applications.
Context matters for AI collaboration: The research identifies "context-dependent effects," meaning the same feedback intervention works differently depending on the situation, underscoring that responsible AI design requires careful consideration of specific domains and decision-making environments rather than one-size-fits-all solutions.

Abstract

Think Once, Reflect Twice: How Feedback Shapes Human-AI Collaboration

Key Takeaways

Abstract

Paper Info

Responsible AI Topics

Links

Think Once, Reflect Twice: How Feedback Shapes Human-AI Collaboration

Key Takeaways

Abstract

Paper Info

Responsible AI Topics

Links