This month’s topic request comes from our apprentice Amirreza, and we are asking whether climate protection and the use of large AI models are compatible.
Anyone who has recently delved deeper into CO2 consumption will certainly have become aware of the environmental impact of generative AI. Studies indicate that the electricity demand for AI data centers will be eleven times higher by 2030 than in 2023. A threefold increase in water demand is also predicted. A simple query to ChatGPT, for example, requires ten times as much energy as a simple Google search. The longer the generated response from a language model, the higher the energy consumption. In April of this year, Sam Altman commented on the immense costs caused by technically unnecessary polite phrases such as “please” and “thank you”. There is talk that large tech companies intend to operate their own nuclear power plants in the future.
All of this sounds as if companies striving to keep their CO2 footprint low would have to forgo generative AI. But is there really no alternative?
In fact, before deploying generative AI, companies should ask themselves some economic and ecological questions, for example: Is the use of generative AI proportionate? Can the tasks the model is intended to solve not be solved by any other technology?
Apart from that, there are also ways to influence the climate impact of generative AI. An important factor here is, of course, the choice of operator and its location. Because there are indeed operators who run AI systems in climate-neutral data centers. For example, we at Splitbot rely on data centers that are powered by renewable energies and sensibly reuse the generated waste heat. Furthermore, we offer our clients the option to operate KOSMO on-premise. This is the ideal solution if your own IT or building is already climate-neutral.
Another exciting aspect is the training of the models themselves. Scientists have discovered that during the training of AI models, parts of the calculations are performed unnecessarily quickly. The computational work during AI training is distributed across multiple GPUs – but unevenly. This leads to less utilized GPUs having to “wait” for the more heavily used ones. Since this waiting time occurs anyway, the fast calculation is unnecessary. By adjusting the computing speed, electricity consumption could be reduced here. The researchers directly provided the suitable solution: The open-source software Perseus controls the GPU frequency of each individual calculation, thereby keeping power consumption as low as possible.
Sources: https://t3n.de/news/ki-stromverbrauch-energie-perseus-1656380/
https://reset.org/sprachmodelle-nachhaltig-nutzen-sparsamer-genai-gruen/
