news
- Computing CO2 cost: The article highlights the environmental impact of large language models (LLMs) on climate change, with a focus on the CO2 emissions generated during model inference. By integrating carbon emission estimates into the Open LLM Leaderboard, we can provide transparency and encourage model creators to balance performance with environmental responsibility.
- General Trends: The study analyzed over 2,700 models and found that smaller models tend to have a lower CO2 cost, making them appealing for use cases where energy efficiency is paramount. Community fine-tunes and merges tend to be more CO2-efficient than official models, and official models with instruction tuning tend to have higher CO2 emissions.
- Detailed Insights: The article provides a detailed analysis of high-parameter and compact language models, highlighting the importance of fine-tuning and community involvement in reducing CO2 emissions. The study also shows that community fine-tunes can significantly reduce CO2 emissions, particularly in the case of calme-2.1-qwen2-72b.
- Conclusion: The article concludes that by integrating carbon emission estimates into the Open LLM Leaderboard and encouraging model creators to balance performance with environmental responsibility, we can reduce the environmental impact of large language models on climate change.
Since June 2024, we have evaluated more than 3,000 models on the Open LLM Leaderboard, a worldwide ranking of open language models performance. Even though we’re trying to run evaluations without wasting resources (we use the spare cycles of our cluster, in other words the GPUs which are active but waiting between jobs), this still represents quite a big amount of energy spent for model inference!
In the last year, people have become more and more aware that using large language models (LLMs) to generate text has a significant environmental impact, beyond the already important impact of training. Recent research (see the Towards Greener LLMs article) highlights the challenges of managing resources efficiently at inference due to dynamic and diverse workloads.
By integrating carbon emission estimates into the Open LLM Leaderboard, we aim to provide transparency to users about the carbon impact of various model evaluations and hopefully encourage model creators to balance performance with environmental responsibility.
Read the full post at Hugging Face.
