Two methods gaining steam as data center liquid cooling goes mainstream
Liquid cooled computers for many years have been associated with gaming PC’s and more recently specialized markets like Bitcoin mining. As we enter 2020 market expectations for liquid cooled systems will begin to rise from a “niche” market to main stream. There are several factors that are driving this change.
As system compute densities have increased, it is now typical for a 2U rack server solution to have dual x86 CPUs, up to 6 accelerator cards and 8-12 NVME drives. These rack solutions draw as much as 2 kilowatts or more of power. As compute density increases, this number can go even higher. Several companies are offering liquid cooled solutions which remove heat from systems more efficiently than air cooled systems. These liquid cooled solutions are more efficient, reducing costs by as much as 90% when compared to traditional air cooled systems.
The next potentially disruptive data center cooling technology may be direct liquid cooling. The concept of direct liquid cooling has been around IT for some time in various forms, keeping workstations quiet and cooling mainframes at the heart of many mission critical enterprise applications. Supercomputers also tend to favor liquid cooling, because they pack electronics at such density that airflow becomes a major issue.
Air is still the dominant medium of cooling in datacenters, but this is about to change. After years of development and feedback from facility operators, datacenter-class liquid cooling techniques are now well-honed and easier to integrate than ever before. Today, options go well beyond water cold-plates and the original immersion systems which were either too costly or used unreliable coolants. They now include chassis-level immersion cooling which can also support cold-plate techniques.
Liquid cooling solutions are not only an option for hyper scale data centers but also for on-prem data centers where space, power and air conditioning are limiting factors to how much compute can be hosted locally.
Liquid cooling technology makes it possible not only to cost-effectively build data centers in any geography but also to build supercomputers and standard data centers in hot and humid climates. Liquid cooled solutions are not only for data centers. The technology also makes "edge of network" computing a reality, enabling high performance computing at edge nodes where the environment is prohibitive to thermal extraction via air conditioning.
There are two mainstream methods of liquid cooling:
- Direct-to-chip liquid cooling - Replaces the heatsink with a liquid solution to remove heat externally. This can be accomplished with an air condenser, water or liquid cooler, or be leveraging other heat exchangers. There are different solutions offered from different companies, including single phase and two-phase solutions. The benefit is that these methods require less modifications to be made to the existing systems while capturing 50-80% of the heat.
- Immersive liquid cooling - Like the previous method, there are also multiple approaches to this method of cooling. One approach requires that the chassis is sealed, and the electronics are immersed in liquid. In the second approach, the entire chassis is immersed in tub or open liquid bath. These solutions provide more efficiency than direct-to-chip cooling, resulting in around 95% heat capture. The drawback of these approaches is that they require more space and potentially more system modifications.
Liquids used in these solutions can be categorized in three areas: water-based, hydrocarbon-based oils (dielectric), engineered fluids (dielectric).
Liquid cooling will become more standard in data centers in coming years for three primary reasons:
- More compute for the same amount of power – Peak power for liquid cooling is a much lower than air. Liquid cooling also reduces or eliminates system fans, which also consume energy (5-10% of the load on average). Liquid cooling allows for 25-50% more IT capacity within a given power envelope.
- Higher application performance – Modern server processors have a configurable thermal budget which the customer can turn up or down to match the desired performance and power. Liquid cooling helps maintain processors and accelerators, achieving higher peak performance longer.
- Environmental and social benefits – Research predicts that by 2025, Information and Communication Technologies (ICT) will account for 20% of the world’s electricity consumption and contribute 3.5% of global carbon emissions. Cooling will represent about 40% of this electricity consumed. With data center efficiency standards and regulations coming into place, liquid cooled solutions will enter the mainstream.
There is no one solution that meets all requirements for data centers. Certainly, both methods have benefits and drawbacks to be weighed. IT organizations will have to consider cost of capital, energy, serviceability, rack density, water usage, water availability, environment, air flow, scalability and tradeoffs of different liquid coolants.
Contact the Avnet Integrated team to learn more.