Segment Anything in Pathology Images with Natural Language

Natural language makes AI more accessible to clinicians: PathSegmentor allows pathologists to identify tissue structures using simple text descriptions rather than requiring technical spatial inputs like points or boxes, potentially lowering barriers to AI adoption in clinical settings.

Foundation models can reduce annotation burden: By training on a large, diverse dataset (PathSeg with 275k image-mask-label triples across 160 categories), the model generalizes better to new cases, addressing a major practical challenge in clinical AI where labeled data is expensive and time-consuming to create.

Broader applicability improves clinical utility: PathSegmentor outperforms specialized models designed for single tasks, suggesting that more general-purpose AI systems may be more practical for real-world pathology labs that need to analyze diverse tissue types and structures.

AI transparency supports clinical decision-making: The model enhances interpretability through feature importance estimation and imaging biomarker discovery, providing pathologists with evidence-based explanations for AI recommendations rather than "black box" outputs—a critical requirement for responsible clinical AI deployment.

Robust performance on external data builds trust: The model's strong generalization to datasets outside its training set (demonstrated through external validation) suggests it may be more reliable and trustworthy for deployment across different hospitals and labs with varying imaging equipment and protocols.