Bridging the Gap on AI-Assisted Scientific Software Development Through Transparency and Traceability

AI use in scientific software is already widespread but largely invisible: Researchers are actively using large language models to write code, generate test cases, and draft documentation without formal acknowledgment or guidance, creating potential risks for scientific integrity and reproducibility.

Safety-critical scientific software requires special governance: Tools used in nuclear energy, fusion research, and other high-stakes domains must meet strict Quality Assurance standards (such as NQA-1) that demand traceability and documented procedures—standards that current ad hoc AI use threatens to undermine.

The real question is not whether to use AI, but how to use it responsibly: Rather than banning AI-assisted development, the scientific community should establish clear governance frameworks that maintain quality standards while allowing beneficial AI tools to be used transparently and with proper oversight.

Verification and validation cases are an ideal starting point for AI governance: Because these test cases have known correct answers, errors can be objectively identified and fully audited, making them a practical proving ground for developing trustworthy AI-assisted development practices before expanding to other software components.

Transparency and traceability are essential safeguards: The paper demonstrates that structured frameworks requiring documented procedures and full auditability of AI-generated artifacts can help maintain scientific software quality while enabling responsible AI adoption.