CRISS: A Retrieval-Augmented AI Chatbot for Assisting Cancer Registrars

RAG technology significantly improves accuracy and trustworthiness: When the AI system was trained on cancer registry standards (retrieval-augmented generation), it provided citations supporting its answers 62% of the time on easy questions, 56% on medium questions, and 59% on hard questions—compared to only 26-29% for systems without this grounding, demonstrating that connecting AI to authoritative sources makes responses more reliable.

Specialized AI tools can reduce expert burden without replacing human judgment: CRISS was designed to assist cancer registrars with complex, frequently updated coding standards while preserving "human oversight of final abstraction decisions," suggesting that AI can handle routine information retrieval and training support while keeping humans in control of critical decisions.

Different AI models have different strengths depending on question complexity: The study found that proprietary models (like GPT and Gemini) performed better on straightforward questions, while open-weight models excelled on harder, more nuanced registry questions—indicating that organizations should evaluate multiple AI approaches rather than assuming one solution fits all use cases.

Domain-specific knowledge bases are essential for specialized professional tasks: The researchers built CRISS using a custom knowledge base of national cancer registry standards rather than relying on general-purpose AI, highlighting that responsible AI deployment in specialized fields requires tailored training data and expertise.