MedFailBench: A Clinician-Built Open-Source Benchmark for Medical AI Safety Boundary Inspection

Safety failures are distinct from knowledge gaps: Unlike traditional medical AI benchmarks that test whether models know correct answers, MedFailBench specifically measures how and why AI systems fail in clinical contexts—a crucial distinction for understanding real-world risks in healthcare settings.

Six specific safety failure categories have been identified: The benchmark categorizes medical AI errors into six types: missed urgent escalation, unsafe remote dosing, unsafe discharge reassurance, evidence fabrication, unsafe protocol execution, and source support gaps—providing clinicians and developers with a structured taxonomy to understand failure modes.

Severity levels matter for clinical decision-making: The benchmark uses a 1-5 severity scale for errors, recognizing that not all AI failures pose equal risk; this allows stakeholders to prioritize which safety boundaries are most critical to address in medical AI systems.

Clinician involvement is essential for safety evaluation: The benchmark was built by a clinician and includes clinician review of cases, emphasizing that medical AI safety assessment requires domain expertise rather than purely technical evaluation—an important principle for responsible AI development in healthcare.

Transparency and reproducibility are prioritized: The open-source release includes automated screening pipelines, public access via Hugging Face, and explicit statements about what the benchmark does not include (no patient data, no clinical validation claims, no model rankings), setting a responsible standard for how safety benchmarks should be shared.