Making the Invisible Visible: A Framework for Reflective AI Use in Software Engineering Education

Hidden learning processes need visibility: Traditional assessment of student work only captures final outputs, leaving educators blind to how students actually interact with AI tools. The AI Journal framework addresses this by tracking prompts, outputs, and student decision-making, revealing that critical learning happens when students evaluate whether AI suggestions fit their specific context—not just when they catch obvious errors.

Students vary significantly in how they verify AI outputs: The study found meaningful differences in how first-year students verify and refine AI-generated work, suggesting that responsible AI use isn't automatic. This variation highlights the need for explicit teaching about verification strategies rather than assuming students will naturally develop sound practices.

Reflection tools can support responsible AI practices: By requiring students to document their verification strategies, intervention decisions, and confidence levels, the AI Journal framework helps develop evaluative judgment—the ability to critically assess when and how to trust AI assistance. This structured reflection appears to be a practical way to build responsible AI habits early in education.

Context matters more than perfection in AI evaluation: Students' most important learning moments involved assessing whether AI outputs were suitable for their specific requirements and feasible to implement, rather than simply identifying technical errors. This suggests responsible AI use depends on understanding contextual fit, not just technical accuracy.

Lightweight tracking can work across different AI tools: The framework is described as "model-agnostic," meaning it can work with any generative AI system, making it adaptable as technology evolves and potentially scalable across different educational contexts.