Tools like ChatGPT run on large language models (LLMs), which are artificial neural networks trained on vast amounts of data scraped from the web. These models provide AI-generated answers to text-based prompts.
Speaking at the Fortune Brainstorm AI Singapore conference last week, Seth Dobrin suggested that the future could belong to small language models (SLMs). These models are tailor-made for specific applications and require far less energy to operate.
“These massive large models are not what the technology was built for. They’re cool, they’re fun, but that’s not the solution,” Dobrin, a general partner at venture capital fund 1infinity Ventures, told conference participants. “Look at using small, task-specific models."
Experts have been warning that AI will not reach its full potential until it solves its energy addiction.
Arm Holdings, a designer of power-efficient microchips for handheld devices, predicted earlier this year that GenAI could consume a quarter of all the electricity in the United States by 2030.
It’s not just energy either. Many data centres also use water along with air to cool the servers as they process terabytes of data in seconds.
Dobrin noted that too few people are aware of the ecological impact when they use ChatGPT or Claude today.
“For every 25 to 50 prompts, depending on how big they are, you use about half a litre of water — just through evaporation. We absolutely need new paradigms of cooling.”
As miniaturisation advances and process technology shrinks to 2 and 3 nanometre nodes, the thermal properties of circuits are becoming a bigger problem. Fans and air conditioners cannot transport the heat away fast enough, and even cooling plates attached directly to chips are not effective beyond a certain computational speed.
Sustainable Metal Cloud CEO Tim Rosenfield warned:“We’re getting less and less efficient with every step, and this has coincided with the rise of AI, which unfortunately uses the most energy-intensive and hottest chips out there.”
He believes SMC may have the answer. Its flexible, modular HyperCube data centre hardware can roughly halve carbon emissions compared to a conventional air-cooled H100 HGX system by submerging servers directly in a bath of oil, a process known as immersion cooling.
Oil is far more effective than air at extracting heat and is also less electrically conductive than water, making it better suited for server racks that will run cutting-edge 2nm and 3nm AI training and inference chips.
While immersing servers in nonconductive liquids can help dissipate growing volumes of heat quickly and efficiently, this technology comes with challenges in terms of upfront investment costs, maintenance, and repair.
Venture capitalist Dobrin had one more piece of unconventional advice to minimise AI's substantial carbon footprint—beyond using new cooling technologies.
He suggests forgetting the GenAI hype for a moment and asking yourself whether something else might be equally suited for the task.
“Focus on the use case—what problem you are trying to solve? Do you really need generative AI?”