Why liquid-cooled storage is changing the AI infrastructure equation

Behind every AI assistant, recommendation, search result and intelligent application is infrastructure working to deliver data at the speed modern AI demands. AI infrastructure is scaling faster than the assumptions on which it was built. Racks are denser. GPUs are more powerful. And the thermal demands of modern AI workloads are testing the limits of cooling approaches that were never designed for this environment. The result is an infrastructure challenge with real consequences: When components run too hot, they throttle, pipelines stall, and systems built to deliver AI at speed and scale quietly fail to do so. For Micron, that means designing storage solutions that not only deliver performance, but sustain it under the thermal demands of modern AI infrastructure.

This evolution is already underway in every major data center, and it is reshaping how operators think about every layer of the stack, including storage. Understanding where cooling fits into the efficiency and performance equation of modern AI systems is increasingly essential for anyone building, operating or planning at scale.

Why AI is accelerating the evolution of cooling

Data center cooling has always been a core engineering discipline. Modern data center facilities already deploy a range of effective approaches, including hot-aisle and cold-aisle containment, direct-to-chip liquid cooling and fully liquid-cooled servers. These approaches have served the industry well. But the infrastructure demands of AI are proportional to the breakthroughs it enables, from compressing drug discovery timelines from years to months¹, to catching cancers earlier through AI-powered medical imaging².

AI is driving rack power to levels that are forcing the next step in that evolution. Top-end AI server GPUs have already crossed 1,200 watts per chip³. Next-generation rack-level power projections reach as high as 600 kilowatts per rack⁴, or roughly enough electricity to power hundreds of average homes. At those power levels, cooling is no longer just about managing heat. It becomes a critical factor in keeping AI deployments running efficiently and ensuring systems can deliver the experiences users expect. At those densities, the economics and thermal requirements of AI infrastructure are accelerating the adoption of more advanced cooling across every layer of the stack, including storage.

It is not only GPUs and CPUs driving the thermal equation. Every component in the server contributes to the power and heat budget, and each plays a role in improving it.

The role of storage (and liquid cooling) in thermal and performance outcomes

Solid-state drives (SSDs) are not thermally passive components in a modern AI server. AI servers are increasingly dense, packing many high-performance NVMe™ drives — a type of SSD optimized for the demanding speed and throughput requirements of AI workloads — into tight chassis configurations, with each drive often drawing up to 25 watts or more.

At scale, that concentrated heat load becomes significant.

When a solid-state drive exceeds its optimal operating temperature, the drive automatically reduces its own activity to prevent damage. This behavior is called thermal throttling. In an AI inference or training pipeline, throttling in the storage layer creates a bottleneck that reverberates upstream. The GPU stalls while waiting for data, compute cycles go unused, and the operator pays for capacity that cannot be fully utilized when it is needed most. The business consequence is straightforward. When storage throttles under load, the GPU waits. A GPU that is waiting is not generating inference throughput, not serving customers and not justifying the capital cost of the infrastructure around it. For operators running AI at scale, that idle compute time is not an abstract inefficiency. It limits infrastructure utilization and reduces the performance AI systems can deliver.

Liquid cooling addresses this directly. Cold-plate liquid cooling works by placing a thermally conductive metal block against the SSD enclosure. A coolant, typically a water-glycol mixture, flows through internal channels machined into that block, continuously extracting heat at the source. The warmed fluid circulates through the facility cooling loop, is cooled and returns to repeat the process. Because cooling occurs directly at the device rather than relying on airflow to carry heat away from a densely packed drive bay, drive temperatures stay within their optimal range even under sustained peak workloads. Throttling is eliminated, and the storage layer performs as it was designed to.

At its core, the advantage comes down to physics. Liquids transfer heat far more efficiently than air. The heat capacity and density of a water-glycol coolant means that a much smaller volume of coolant can carry the same amount of thermal energy as a much larger volume of airflow. A small circulation pump moving a modest flow of coolant can accomplish what would otherwise require significant fan infrastructure. That efficiency difference is what makes the data so compelling.

What the efficiency numbers show

Micron’s analysis of a 32-drive NVMe SSD bank, representative of the dense storage configurations increasingly deployed in AI and data center environments, modeled the electrical power required to achieve equivalent cooling using air versus cold-plate liquid cooling. The results illustrate the scale of the difference: Air cooling requires between 37 and 80 watts of electrical power to cool the same bank of drives. Cold-plate liquid cooling achieves equivalent cooling using between 0.42 and 1.35 watts. That is approximately 98 percent less energy to accomplish the same thermal outcome⁵.

This is not a marginal gain. It reflects a fundamental physical property of liquids versus air as a heat transfer medium. The same efficiency advantage that improves cooling at the drive level can also help reduce the energy required to cool and operate the broader data center environment.

Understanding power usage effectiveness

Power usage effectiveness (PUE) is the metric operators and investors use to evaluate how efficiently a data center converts utility power into useful computing work. A PUE of 1.0 would represent perfect efficiency. In practice, data centers have historically run from 1.2 to above 1.5, meaning operators purchase between 20 and 50 percent more electricity than their servers actually use, with the overhead going to cooling systems, power distribution and facility operations⁶.

Liquid cooling is one of the most direct levers available to reduce PUE. When heat is extracted at the device level, the facility’s central air conditioning burden decreases. Fan speeds can be reduced. The energy overhead per compute cycle decreases. For hyperscale operators, even modest improvements in PUE can translate into significant reductions in energy consumption and operating costs.

What this means for an AI inference deployment

Consider an AI inference deployment running high-density storage in a large-scale server environment. Under traditional air cooling, storage drives operating near their thermal limits during peak inference loads begin to throttle, slowing the rate at which data is delivered to the GPU. The GPU, waiting on data, sits partially idle. The operator is paying for compute capacity that cannot be fully utilized during exactly the periods when demand is highest.

With cold-plate liquid cooling applied to the same storage configuration, drives maintain consistent temperatures throughout sustained workloads. Throttling is eliminated. GPU utilization improves. The cooling overhead drops from tens of watts per storage shelf to less than two watts, a reduction that compounds meaningfully across a large deployment running continuously.

The connection is straightforward. Infrastructure that eliminates thermal bottlenecks in the storage layer enables more of the available compute capacity to be put to productive use. Energy efficiency and performance utilization are not competing goals. They reinforce each other.

The path forward

Liquid-cooled SSDs are not a future-looking concept. They are a deployable answer to a problem that is already costing operators compute time, energy and performance. For those building AI infrastructure, they offer a direct path to better GPU utilization, lower cooling overhead and consistent performance under sustained demand.

As AI workloads continue to grow, the infrastructure behind them must deliver data efficiently, maintain performance under load and support the experiences users rely on every day. Liquid-cooled storage is an important part of that evolution.

To learn more about how Micron is engineering memory and storage for the AI data center, visit our data center insights page and read our companion piece on how low-power DRAM is reshaping data center memory.