Toward Automated and Trustworthy Scientific Analysis and Visualization with LLM-Generated Code

LLMs can help democratize scientific coding, but current reliability is limited: Open-source LLMs frequently fail to generate correct, executable code for scientific analysis tasks without human intervention, with failures often stemming from ambiguous instructions and lack of domain-specific knowledge—meaning scientists cannot yet fully trust autonomous code generation for critical research workflows.

Three practical strategies can significantly improve LLM code quality: The researchers identified that combining data-aware prompt clarification, retrieval-augmented prompts (providing relevant context), and iterative error repair substantially increased execution success rates and output quality, suggesting that thoughtful human-AI collaboration rather than full automation is currently the most responsible approach.

The problem of ambiguous prompts is central to trustworthiness: Many failures occurred because natural language descriptions of scientific tasks were inherently unclear to the models, highlighting that responsible deployment requires either clearer communication protocols or better tools to help scientists specify their needs precisely.

Benchmark testing is essential for responsible AI deployment in science: The researchers created a reusable benchmark suite of real-world research tasks to systematically evaluate LLM performance, demonstrating that rigorous, domain-specific testing—rather than general benchmarks—is necessary before deploying AI tools in high-stakes scientific contexts.

Human oversight remains necessary for scientific integrity: While LLM-assisted code generation shows promise for accelerating discovery, the findings underscore that human validation and iterative refinement are currently indispensable safeguards to ensure the correctness and trustworthiness of AI-generated scientific workflows.