Towards responsible AI for education: Hybrid human-AI to confront the Elephant in the room

AI in education is often oversimplified: The paper identifies a critical problem where different types of AI are lumped together under one label, with companies promoting large language models as universal solutions despite their distinct purposes and limitations. This lack of clarity obscures what AI can and cannot do in educational contexts.

Current AI systems ignore crucial aspects of how students actually learn: AI models for education frequently neglect motivation, emotion, and metacognition—essential elements of the learning process—and fail to account for how these factors vary across different contexts, potentially leading to ineffective or misleading educational interventions.

Students and educators are largely absent from AI development: The paper highlights that domain knowledge and input from key stakeholders (teachers, students, parents) are insufficiently integrated into the design of AI educational systems, raising concerns about whether these tools actually address real educational needs.

Technical methods are being misapplied to educational data: The research identifies specific methodological problems, including using machine learning models designed for static data on temporal (time-based) educational information, and evaluating these models with inappropriate metrics—errors that can produce unreliable results.

Explanations from AI systems may be misleading: The paper warns that common techniques used to explain how AI makes decisions are unreliable, yet are being deployed in education where transparency is critical for building trust with students, teachers, and families.