Auditing Student-AI Collaboration: A Case Study of Online Graduate CS Students

Students want AI as a collaborative tool, not a replacement: The research found that graduate CS students prefer AI systems that support their learning rather than fully automate tasks, revealing a gap between current AI capabilities (which tend toward over-automation) and what students actually want from these tools in academic contexts.

Hallucinations and unreliability are major concerns: Among the primary risks students identified, AI systems' tendency to generate plausible-sounding but false information ("hallucinated outputs") emerged as a significant barrier to trust, suggesting that improving accuracy and transparency about AI limitations is critical for educational adoption.

Students want clearer boundaries and control: The study found that students have specific preferences about when and how they use AI across different academic tasks, indicating that one-size-fits-all AI integration fails to meet their needs—instead, systems should allow students to set their own boundaries around automation levels.

Current AI design doesn't match educational goals: The research identified meaningful gaps between what existing AI systems are designed to do and what students believe would actually support their learning, suggesting that AI tools developed for general audiences may undermine rather than enhance student agency and skill development.

Student input is essential for responsible AI in education: By directly asking students about their concerns and preferences, this audit demonstrates that designing trustworthy educational AI requires listening to end-users rather than assuming how these tools should be integrated into academic work.