AI-Powered Detection of Inappropriate Language in Medical School Curricula

AI can efficiently identify outdated and harmful language in medical training materials at scale: The study analyzed approximately 500 documents and over 12,000 pages of medical curricula—a volume that would be "prohibitively costly and impractical" to review manually—demonstrating that automated systems can make comprehensive audits of educational content feasible.

Smaller, specialized AI models outperform larger general-purpose models for this task: Contrary to common assumptions about bigger AI systems, fine-tuned small language models significantly outperformed larger models like Llama-3 70B, suggesting that targeted AI solutions may be more effective than general-purpose systems for identifying specific types of problematic language.

Training data quality matters more than model size: Adding unflagged excerpts as negative examples improved the specialized classifiers' performance by up to 25%, indicating that how AI systems are trained—not just their size—is critical for catching harmful language in medical education.

Inappropriate language in medical materials has real consequences: The paper emphasizes that outdated, exclusionary, or non-patient-centered terms in medical training can "significantly influence clinical training, patient interactions, and health outcomes," connecting curriculum language directly to patient care quality.

Systematic review of medical curricula is now possible: By automating the detection of inappropriate language categories, institutions can now systematically identify and update problematic content in their training materials, supporting more equitable and evidence-based medical education.