How Researchers Navigate Accountability, Transparency, and Trust When Using AI Tools in Early-Stage Research: A Think-Aloud Study

AI's confident tone masks uncertainty: The study found that LLM-based tools present information with unwarranted confidence, making it harder for researchers to identify which outputs need careful verification. Since researchers bear ultimate accountability for their work, this obscured uncertainty creates a significant problem for responsible research practice.

Provenance and transparency are compromised: Researchers struggle to trace where AI-generated information comes from because the tools don't clearly show their retrieval and content construction processes. This opacity makes it difficult to establish transparent, verifiable foundations for scholarly work—a core requirement for responsible AI in research.

Trust in AI is fragile and context-dependent: The study revealed that researchers' trust in AI tools is easily broken and varies significantly depending on the specific task and context. Rather than building stable confidence, researchers experienced trust that could quickly erode when tools made errors or produced unreliable outputs.

Researchers develop workarounds to restore judgment: When facing these accountability, transparency, and trust challenges, the 15 researchers in the study developed compensatory strategies to maintain their own scholarly judgment. This suggests that without better AI design, researchers must invest extra effort to use these tools responsibly.

Early-stage research is particularly vulnerable: The study focused on literature exploration, synthesis, and ideation—foundational research activities where scholarly judgment is critical. These early decisions shape entire research projects, making the integration of opaque AI tools especially consequential for research integrity.