As we speak, we’re unveiling the subsequent Fairwater web site of Azure AI datacenters in Atlanta, Georgia. This purpose-built datacenter is related to our first Fairwater web site in Wisconsin, prior generations of AI supercomputers and the broader Azure international datacenter footprint to create the world’s first planet-scale AI superfactory. By packing computing energy extra densely than ever earlier than, every Fairwater web site is constructed to effectively meet unprecedented demand for AI compute, push the frontiers of mannequin intelligence and empower each individual and group on the planet to attain extra.
To fulfill this demand, we’ve got reinvented how we design AI datacenters and the techniques we run within them. Fairwater is a departure from the standard cloud datacenter mannequin and makes use of a single flat community that may combine a whole lot of hundreds of the most recent NVIDIA GB200 and GB300 GPUs into a large supercomputer. These improvements are a product of a long time of expertise designing datacenters and networks, in addition to learnings from supporting a few of the largest AI coaching jobs on the planet.
Whereas the Fairwater datacenter design is effectively fitted to coaching the subsequent technology of frontier fashions, additionally it is constructed with fungibility in thoughts. Coaching has advanced from a single monolithic job into a spread of workloads with totally different necessities (reminiscent of pre-training, fine-tuning, reinforcement studying and artificial information technology). Microsoft has deployed a devoted AI WAN spine to combine every Fairwater web site right into a broader elastic system that permits dynamic allocation of numerous AI workloads and maximizes GPU utilization of the mixed system.
Under, we stroll by way of a few of the thrilling technical improvements that help Fairwater, from the way in which we construct datacenters to the networking inside and throughout the websites.
Most density of compute
Trendy AI infrastructure is more and more constrained by the legal guidelines of physics. The pace of sunshine is now a key bottleneck in our skill to tightly combine accelerators, compute and storage with performant latency. Fairwater is designed to maximise the density of compute to reduce latency inside and throughout racks and maximize system efficiency.
One of many key levers for driving density is enhancing cooling at scale. AI servers within the Fairwater datacenters are related to a facility-wide cooling system designed for longevity, with a closed-loop strategy that reuses the liquid repeatedly after the preliminary fill with no evaporation. The water used within the preliminary fill is equal to what 20 houses devour in a 12 months and is barely changed if water chemistry signifies it’s wanted (it’s designed for 6-plus years), making it extraordinarily environment friendly and sustainable.
Liquid-based cooling additionally gives a lot greater warmth switch, enabling us to maximise rack and row-level energy (~140kW per rack, 1,360 kW per row) to pack compute as densely as attainable contained in the datacenter. State-of-the-art cooling additionally helps us maximize utilization of this dense compute in steady-state operations, enabling giant coaching jobs to run performantly at excessive scale. After biking by way of a system of chilly plate paths throughout the GPU fleet, warmth is dissipated by one of many largest chiller vegetation on the planet.
One other manner we’re driving compute density is with a two-story datacenter constructing design. Many AI workloads are very delicate to latency, which suggests cable run lengths can meaningfully impression cluster efficiency. Each GPU in Fairwater is related to each different GPU, so the two-story datacenter constructing strategy permits for placement of racks in three dimensions to reduce cable lengths, which in flip improves latency, bandwidth, reliability and price.
Excessive-availability, low-cost energy
We’re pushing the envelope in serving this compute with cost-efficient, dependable energy. The Atlanta web site was chosen with resilient utility energy in thoughts and is able to reaching 4×9 availability at 3×9 price. By securing extremely obtainable grid energy, we will additionally forgo conventional resiliency approaches for the GPU fleet (reminiscent of on-site technology, UPS techniques and dual-corded distribution), driving price financial savings for purchasers and sooner time-to-market for Microsoft.
We’ve additionally labored with our trade companions to codevelop power-management options to mitigate energy oscillations created by giant scale jobs, a rising problem in sustaining grid stability as AI demand scales. This features a software-driven resolution that introduces supplementary workloads during times of decreased exercise, a hardware-driven resolution the place the GPUs implement their very own energy thresholds and an on-site vitality storage resolution to additional masks energy fluctuations with out using extra energy.
Slicing-edge accelerators and networking techniques
Fairwater’s world-class datacenter design is powered by purpose-built servers, cutting-edge AI accelerators and novel networking techniques. Every Fairwater datacenter runs a single, coherent cluster of interconnected NVIDIA Blackwell GPUs, with a complicated community structure that may scale reliably past conventional Clos community limits with current-gen switches (a whole lot of hundreds of GPUs on a single flat community). This required innovation throughout scale-up networking, scale-out networking and networking protocol.
When it comes to scale-up, every rack of AI accelerators homes as much as 72 NVIDIA Blackwell GPUs, related by way of NVLink for ultra-low-latency communication inside the rack. Blackwell accelerators present the best compute density obtainable immediately, with help for low-precision quantity codecs like FP4 to extend whole FLOPS and allow environment friendly reminiscence use. Every rack gives 1.8 TB of GPU-to-GPU bandwidth, with over 14 TB of pooled reminiscence obtainable to every GPU.
These racks then use scale-out networking to create pods and clusters that allow all GPUs to operate as a single supercomputer with minimal hop counts. We obtain this with a two-tier, ethernet-based backend community that helps large cluster sizes with 800 Gbps GPU-to-GPU connectivity. Counting on a broad ethernet ecosystem and SONiC (Software program for Open Community within the Cloud – which is our personal working system for our community switches) additionally helps us keep away from vendor lock-in and handle price, as we will use commodity {hardware} as an alternative of proprietary options.
Enhancements throughout packet trimming, packet spray and high-frequency telemetry are core elements of our optimized AI community. We’re additionally working to allow deeper management and optimization of community routes. Collectively, these applied sciences ship superior congestion management, speedy detection and retransmission and agile load balancing, making certain ultra-reliable, low-latency efficiency for contemporary AI workloads.
Planet scale
Even with these improvements, compute calls for for big coaching jobs (now measured in trillions of parameters) are rapidly outpacing the facility and house constraints of a single facility. To serve these wants, we’ve got constructed a devoted AI WAN optical community to increase Fairwater’s scale-up and scale-out networks. Leveraging our scale and a long time of hyperscale experience, we delivered over 120,000 new fiber miles throughout the US final 12 months — increasing AI community attain and reliability nationwide.
With this high-performance, high-resiliency spine, we will instantly join totally different generations of supercomputers into an AI superfactory that exceeds the capabilities of a single web site throughout geographically numerous areas. This empowers AI builders to faucet our broader community of Azure AI datacenters, segmenting site visitors based mostly on their wants throughout scale-up and scale-out networks inside a web site, in addition to throughout websites by way of the continent spanning AI WAN.
This can be a significant departure from the previous, the place all site visitors needed to trip the scale-out community whatever the necessities of the workload. Not solely does it present clients with fit-for-purpose networking at a extra granular degree, it additionally helps create fungibility to maximise the pliability and utilization of our infrastructure.
Placing all of it collectively
The brand new Fairwater web site in Atlanta represents the subsequent leap within the Azure AI infrastructure and displays our expertise working the most important AI coaching jobs on the planet. It combines breakthrough improvements in compute density, sustainability and networking techniques to effectively serve the large demand for computational energy we’re seeing. It additionally integrates deeply with different AI datacenters and the broader Azure platform to kind the world’s first AI superfactory. Collectively, these improvements present a versatile, fit-for-purpose infrastructure that may serve the complete spectrum of contemporary AI workloads and empower each individual and group on the planet to attain extra. For our clients, this implies simpler integration of AI into each workflow and the power to create progressive AI options that have been beforehand unattainable.
Discover out extra about how Microsoft Azure may help you combine AI to streamline and strengthen growth lifecycles right here.
Scott Guthrie is liable for hyperscale cloud computing options and providers together with Azure, Microsoft’s cloud computing platform, generative AI options, information platforms and data and cybersecurity. These platforms and providers assist organizations worldwide resolve pressing challenges and drive long-term transformation.
Editor’s notice: An replace was made to extra clearly clarify how we optimize our community.
