Explanations from Large Language Models Make Small Reasoners Better

Smaller AI models can match or exceed much larger models when trained on explanations: The research demonstrates that small models trained using explanations from large language models (LLMs) can outperform GPT-3 (a 175-billion parameter model) by up to 9.5% accuracy, suggesting that explanation-based training is more efficient than simply scaling up model size.

Explanations improve both accuracy and interpretability simultaneously: The study shows that when small models are trained to generate explanations alongside their predictions, they achieve stronger reasoning capabilities while also producing human-understandable justifications for their answers—addressing two key responsible AI concerns at once.

This approach makes AI deployment more practical and affordable: By enabling smaller models to perform at the level of massive models, this method reduces computational costs and energy consumption associated with AI systems, making responsible AI more economically feasible for real-world applications.

Multi-task learning with explanations is more effective than traditional fine-tuning: The research consistently outperforms standard fine-tuning approaches across multiple reasoning tasks, suggesting that teaching models to explain their reasoning is a fundamentally better training strategy than conventional methods.

Explainability is validated through human evaluation: The paper includes human assessment confirming that the generated explanations are genuinely high-quality and useful, not just appearing reasonable—an important validation that the system actually achieves the goal of explainable AI rather than just claiming to.