AI2030 Responsible AI Tools Marketplace

Key Takeaways

Environmental reporting is virtually absent from AI research: An analysis of 5,285 papers accepted to NeurIPS 2025 found that reporting of environmental impact is "nearly non-existent," indicating the AI research community lacks standardised practices for carbon accounting despite growing calls for responsible AI.
LLMs have significant and largely unmeasured environmental costs: The paper highlights that as Large Language Models become more capable and widespread, their environmental footprint grows substantially, yet the field has no consistent way to measure or report these impacts.
New standardised metrics and tools are now available: The researchers have defined standardised sustainability metrics for evaluating training efficiency and created carbonbenchmark, a software tool that researchers can easily use to track and report the carbon emissions of their AI models.
The field should shift focus from accuracy alone to efficiency: The paper introduces the "Smallest Model that Achieves the Job" (SMAJ) framework, which proposes that the AI community should stop pursuing marginal accuracy improvements when they come at disproportionate environmental costs, and instead prioritise computational efficiency and environmental accountability alongside traditional performance metrics.

Abstract

As the capabilities and ubiquity of Large Language Models (LLMs) grow, so does their environmental footprint. Despite calls for responsible AI, the machine learning community lacks standardised practices for carbon accounting. Our automated literature review of the 5,285 papers accepted to NeurIPS 2025 reveals that reporting of environmental impact is nearly non-existent. To catalyse a shift toward sustainable AI, we define standardised sustainability metrics for evaluating model training efficiency, accompanied by simple heuristics to estimate the carbon cost of LLM inference. We implement these metrics in carbonbenchmark, a drop-in software solution for tracking and reporting emissions. Finally, to combat the pursuit of marginal accuracy gains at disproportionate environmental costs, we formalise the `Smallest Model that Achieves the Job' (SMAJ), a framework which challenges the field to prioritise computational efficiency and environmental accountability alongside traditional `State-of-the-Art' (SotA) accuracy.

Key Takeaways

Environmental reporting is virtually absent from AI research: An analysis of 5,285 papers accepted to NeurIPS 2025 found that reporting of environmental impact is "nearly non-existent," indicating the AI research community lacks standardised practices for carbon accounting despite growing calls for responsible AI.
LLMs have significant and largely unmeasured environmental costs: The paper highlights that as Large Language Models become more capable and widespread, their environmental footprint grows substantially, yet the field has no consistent way to measure or report these impacts.
New standardised metrics and tools are now available: The researchers have defined standardised sustainability metrics for evaluating training efficiency and created carbonbenchmark, a software tool that researchers can easily use to track and report the carbon emissions of their AI models.
The field should shift focus from accuracy alone to efficiency: The paper introduces the "Smallest Model that Achieves the Job" (SMAJ) framework, which proposes that the AI community should stop pursuing marginal accuracy improvements when they come at disproportionate environmental costs, and instead prioritise computational efficiency and environmental accountability alongside traditional performance metrics.

Abstract

Beyond State-of-the-Art: Standardising Environmental Impact Metrics for AI Research

Key Takeaways

Abstract

Paper Info

Responsible AI Topics

Links

Beyond State-of-the-Art: Standardising Environmental Impact Metrics for AI Research

Key Takeaways

Abstract

Paper Info

Responsible AI Topics

Links