Google researchers looked at how much energy, water and carbon emissions that a run-of-the-mill Gemini text query generates
It has become conventional wisdom that using an AI chatbot like ChatGPT for queries requires far more energy than using a regular search engine, because of the advanced compute involved. However, while there are estimates — a common one is 10x more energy for an AI query than a web search — the companies who actually have to calculate such things (and pay for them) have largely kept mum. And what about quantifying the other impacts of AI, to come up with a figure that more accurately reflects the total environmental cost of a query?
In a new paper, researchers from Google do just that. They measured measuring the energy usage, carbon emissions, and water consumption of Google’s own Gemini AI assistant, in a large-scale production environment.
“Our approach accounts for the full stack of AI serving infrastructure—including active AI accelerator power, host system energy, idle machine capacity, and data center energy overhead,” the researchers said in their technical paper.
Their conclusion? The median text prompt through Gemini apps consumes 0.24 watt-hours of energy, plus the equivalent of five drops of water. (More on the carbon emissions part in a second.)
The researchers said that energy consumption is less than what gets consumed by watching nine seconds of TV, and also noted that the number is “substantially lower than many public estimates.” So, nine seconds of TV and five drops of water per text query. That doesn’t sound like much … until you start thinking about the fact that Google’s Gemini has more than 400 million monthly active users, who are making multiple queries per day and often asking for image or video generation.
“While these impacts are low compared to other daily activities, reducing the environmental impact of AI serving continues to warrant important attention,” the researchers wrote.
The rising public adoption of generative AI is shifting the conversation around the environmental impact of AI, to include not just energy-intensive model training, but the environmental footprint of AI model inference and serving. “With these AI models now serving billions of user prompts globally, the energy, carbon emissions, and water impacts associated with generating responses at scale represents a significant and rapidly growing component of AI’s overall environmental cost,” according to the Google researchers.
Measuring AI’s impacts is not always clear-cut
There have been other attempts to quantify the energy usage of AI. For example, Salesforce and open-source community Hugging Face use the AI Energy Score, which focuses on the comparable energy efficiency of different models so that developers can make choices about which one(s) to use. There is also the ML.ENERGY benchmark, which came out of a research group at the University of Michigan. Both of those have leaderboards which show relative rankings of different models.
Other research has explored, for example, the difference in energy consumption between asking AI to generate text, images or video — with the last two being far more energy intensive. Often, research has focused solely on energy usage, rather than including other well-known impacts of generative AI usage, such as water use. But fundamentally, as the Google researchers write in their paper: “The field lacks first-party data from the largest AI model providers.”
They also pointed to another crucial issue when it comes to figuring out the environmental impact of AI: Among the research community at-large, there is disagreement on which energy-consuming activities should be included in analysis of AI queries, resulting in wide variation of estimates on power consumption. And, the Google researchers said, some of the narrower approaches are missing significant sources of energy use.
Amin Vahdat, VP/GM of AI and infrastructure at Google Cloud and Jeff Dean, chief scientist for Google DeepMind and Google Research, elaborated in a blog post on the research.
“Many current AI energy consumption calculations only include active machine consumption, overlooking several of the critical factors discussed above,” Vahdat and Dean wrote. “As a result, they represent theoretical efficiency instead of true operating efficiency at scale. When we apply this non-comprehensive methodology that only considers active TPU and GPU consumption, we estimate the median Gemini text prompt uses 0.10 Wh of energy, emits 0.02 gCO2e, and consumes 0.12 mL of water. This is an optimistic scenario at best and substantially underestimates the real operational footprint of AI.”
Instead, the Google researchers calculated their numbers on the basis that: “Characterizing and optimizing the environmental impact of AI model serving requires a comprehensive view of energy consumption — including the power drawn by the host machine’s CPU and DRAM, the significant energy consumed by idle systems provisioned for reliability and low latency, and the full data center overhead as captured by the Power Usage Effectiveness (PUE) metric.” This, they added, “accounts for all material energy sources.”
Their work resulted in quantifying measurements in three ways:
- Energy usage per prompt.
- The market-based emissions per prompt, generated by the use of the grid and the associated compute hardware.
- Water consumption per prompt, primarily for data center cooling.
Let’s circle back to that emissions-per-prompt metric, which the Google researchers found was 0.03 grams of carbon dioxide equivalent (gCO2e) for that median text prompt. That was calculated on the basis of the “local grid energy mix of the consumed electricity, and the embodied emissions of the compute hardware,” according to the research paper.
The local energy grid in particular is a highly variable metric across countries and regions, because it depends on how much green energy is available locally and is used by the model provider. The Google researchers looked at the previous calendar-year’s average annual grid emission factors across Google data centers, in order to have a full year of data to use in the calculations — and to be able to work in credits for green energy procurement, which Google has prioritized.
Those numbers matter. The researchers found that over one year, “Google’s software efficiency efforts and clean energy procurement have driven a 33x reduction in energy consumption and a 44x reduction in carbon footprint for the median Gemini Apps text prompt.”
Vahdat and Dean wrote: “We believe this is the most complete view of AI’s overall footprint.”
Read the blog post here, which includes links to the research paper; plus additional commentary here from Ben Gomes, Google’s chief technologist for learning and sustainability.
