Misinformations about power consumption

[samber]

Hey there ✌️

It appears that your estimate of GPT-4o's energy consumption (in Wh) is significantly different from currently circulating figures, such as those reported by TechCrunch:
👉 https://techcrunch.com/2025/02/11/chatgpt-may-not-be-as-power-hungry-as-once-assumed/

From what I can tell, your estimate seems to be based mostly on an assumed quantization level, parameter count, and a guessed model architecture, without access to verified infrastructure data or detailed deployment parameters.

The discrepancy is so large that it borders on misinformation, if not disinformation, especially if it spreads unchecked.

How can we work together to correct or clarify this?

Thanks in advance for your response.

[adrienbanse]

Hey @samber

Many thanks for the message and the feedback. 🙏

All articles that report the estimates you are mentioning are citing a study from epoch.ai (including this TechCrunch article). To the best of my knowledge, this study is the only study to report such "small" estimates, and the latter is itself based on a ton of assumptions that one can criticise (c.f. for example this post). Moreover, epoch.ai was previously funded by OpenAI, so there's a potential conflict of interest here..

Our methodology is indeed mostly based on the parameters you are mentioning, because we see from actually measured energy that the latter are the most impactful (c.f. also our methodology). Now, I agree that our methodology is also based on assumptions, and we tried to disclose those as much as possible, together with possible limitations (see "Assumptions and limitations" section of our methodology).

I really appreciate your proposition to "work together to correct or clarify this", and I would be happy to collaborate on trying to refine our estimates. To me, the only thing that we can do is ask the providers for more transparency, and try to reproduce production environment of these models to then measure the energy consumption and validate (or not) our methodology. However, as you know, for now there is a huge lack of transparency from the providers so I'm afraid we are kind of stuck here... Note however that we've already validated EcoLogits on other models, see for example this discussion.

If there's a precise step in our methodology with which you don't agree, or if you have any idea and/or suggestion to refine the estimates, feel free to write them here or to contact us directly! 😃

[samuelrince]

Hi @samber,

Completely agree with what @adrienbanse responded here. We know we have some improvements to do on the methodology and the benchmarking data we use. But we use what’s available.

The issue I have with the calculation made by Epoch AI is they imply a very high and unrealistic throughput of around ~500 tokens/s in their calculation, where today we are more around ~50-60 tokens/s on the API. So I believe there are some strong hypotheses there as well.

One way to improve this is by making a more realistic GPU energy consumption benchmark at inference, taking into account batching, attention optimizations and more. If you have the expertise and some time to help us build that, please let us know; we are interested.

[samber]

Bigger batch is slower and ...cheaper (in € and Wh).

[samber]

Some updates:

• Sam Altman wrote chatgpt average prompt consumes ~0.34Wh -> https://blog.samaltman.com/the-gentle-singularity
• Google wrote gemini median prompt consumes ~0.24Wh -> https://arxiv.org/pdf/2508.15734#

There is no information on how big "average" or "median" is, and on the distribution across small vs big models. Still, these figures suggest that your methodology may overestimate hyperscalers’ power consumption by a factor of ~100.

[adrienbanse]

Hey @samber, thanks for the update, we're aware of this, and we're currently working hard on updating our methodology.

Google's paper indeed suggest that our methodology is not reflecting reality right now, but it also suggests that our estimations were in the right order of magnitude when it was developed (> 1 year ago, c.f. the 33x improvement ratio). This is quite good news because it also means that, if we update the benchmark (H100, batching, new quantization, etc.) and some assumptions that do not reflect the reality anymore, then we can expect to get closer to the real numbers.

[samber]

I haven’t read the Mistral study on carbon emissions, but the results suggest that their platform is still far from the level of industrialization achieved by hyperscalers.

1y ago, Gemini was in pre-release.