Explain the Black Box for the Sake of Science: the Scientific Method in the Era of Generative Artificial Intelligence

Explainability is essential for scientific discovery: The paper argues that understanding how AI systems make decisions—not just their outputs—is crucial for leveraging AI in scientific research. By examining the "principles" AI uses, scientists can validate or challenge these decisions against domain expertise, potentially uncovering new knowledge.

AI should complement, not replace, human scientific reasoning: The authors emphasize that "human complex reasoning for scientific discovery remains of vital importance" in the current era. Rather than having AI autonomously discover science, the paper proposes AI should serve as a tool that scientists can interrogate and learn from.

Disagreement between AI and experts can drive progress: The paper identifies that when AI interpretability reveals principles that diverge from expert expectations, this disagreement can "spark further scientific investigations" rather than being viewed as a failure. This suggests responsible AI development should treat explainability as a dialogue between human and machine reasoning.

A new field is emerging: "Explainable AI for Science": The authors propose that domain experts should formulate scientific hypotheses based on understanding AI system interpretability. This represents a shift from using AI as a black box predictor to using transparent AI as an active participant in the scientific process.

Reproducibility and validation remain central: The paper grounds its argument in the scientific method's core principle of reproducibility and experimental validation, suggesting that responsible AI for science must maintain these standards rather than abandoning them for efficiency.