Google TPU v7 and Azure Cobalt 200 arrived at SC25 as two very different bets on custom silicon: Google’s liquid‑cooled TPU v7 “Ironwood” accelerator and Microsoft’s Azure Cobalt 200 Arm CPU. One is tuned for AI inference at massive scale, the other for general‑purpose cloud compute, but together they show how hyperscalers are moving away from merchant parts toward vertically integrated, workload‑specific hardware. Both chips target the same constraint: fixed power and cooling envelopes in AI data centers that must carry ever more compute.
Underneath the product branding are hard trade‑offs around process nodes, packaging, yields, and perf/W. Google TPU v7 and Azure Cobalt 200 are not one‑off experiments; they are datapoints on a vector where the largest clouds design silicon, racks, and software as a single system.
Why Google TPU v7 and Azure Cobalt 200 Matter for Today’s Data Centers
Google TPU v7 and Azure Cobalt 200 arrive at a moment when hyperscalers are using custom silicon to re‑architect AI data centers around perf/W, density, and total cost of ownership (TCO). Generative AI build‑outs, tight power ceilings, and increasingly competitive cloud pricing all push in the same direction: more work per watt and per rack, not just more peak FLOPs.
Demand for training and serving large language models (LLMs) continues to outpace the cadence of traditional x86 and GPU roadmaps. Even as aggregate data center power budgets grow, the power available per rack remains constrained in many facilities. That makes perf/W and rack‑level density core design variables.
Google positions TPU v7 Ironwood as delivering substantial perf/W gains over its own prior parts, enabled by a move to TSMC’s 3 nm‑class N3P node, HBM3e memory, and direct liquid cooling, as detailed in Google’s disclosures and ServeTheHome’s package analysis. Microsoft’s Azure Cobalt 200, based on Arm Neoverse V3 cores and fabbed on TSMC N3E, is pitched as a way to lift efficiency for mainstream VMs, databases, and microservices compared with previous x86 and Arm‑based Azure instances, according to ServeTheHome’s coverage.
Landing both parts now is not incidental. Nvidia and AMD are mid‑cycle on new accelerator families, Arm server CPUs have matured at AWS and other clouds, and enterprises are replanning AI infrastructure that will carry them through the later 2020s. Hyperscalers want those long‑lived deployments anchored on proprietary, vertically integrated platforms.
Inside Google TPU v7 Ironwood: Architecture and Data Center Role
TPU v7 Ironwood extends Google’s pattern of pairing each new TPU generation with system‑level changes rather than treating the chip in isolation. The company frames it as the workhorse of its latest “AI hypercomputer” generation, co‑designed with refreshed Jupiter networking, storage, and facility‑level cooling.
TPU v7 Architecture, Packaging, and Liquid Cooling Design
At the silicon level, TPU v7 Ironwood moves to TSMC’s advanced N3P node and uses CoWoS advanced packaging, with Broadcom reported as a key partner in assembly and testing based on public commentary and independent die photography. The package centers on a single large compute die flanked by eight HBM3e stacks. Each TPU v7 exposes on the order of 192 GB of HBM3e with multi‑terabyte‑per‑second aggregate bandwidth, placing it alongside top‑end AI accelerators from merchant vendors in memory capacity and bandwidth.
The die itself is dominated by systolic arrays and matrix engines optimized for dense linear algebra, not general‑purpose scalar or vector work. While Google has not published exact die area, analysis of high‑resolution photos suggests a design that pushes close to reticle‑friendly limits. Keeping TPU v7 as a single monolithic die, rather than adopting a chiplet layout, keeps on‑package latency and control logic simple but pushes more yield risk onto each wafer, especially at a 3 nm‑class node where defect densities remain a concern.
Cooling is the other defining element. Ironwood is built around direct‑to‑chip liquid cooling, with each accelerator drawing roughly a kilowatt under sustained AI workloads according to Google commentary and third‑party reporting. This allows Google to raise accelerator density per rack beyond what air‑cooled systems can sustain while keeping junction temperatures within safe bounds. The trade‑off is dependence on dense liquid distribution networks within each data hall and tighter limits on where these systems can be deployed.
Inter‑chip connectivity relies on a dedicated Inter‑Chip Interconnect (ICI) fabric. Each TPU v7 exposes multiple high‑speed ICI links that wire boards into a local 3D torus within a rack‑level “cube” of 64 chips and then out into Jupiter for pod‑scale clustering. The topology is tuned for predictable, high‑throughput collective operations aligned with Google’s internal training and inference frameworks rather than arbitrary multi‑tenant patterns.
Where Google TPU v7 Ironwood is optimized around matrix compute, HBM3e bandwidth, and tightly coupled AI pods, Azure Cobalt 200 instead targets broadly deployed cloud CPUs where Arm cores and large caches drive better perf/W on everyday workloads.
TPU v7 System Scale: Pods, Interconnects, and AI Training Targets
A typical TPU v7 Ironwood board carries four accelerators; 16 such boards form a 64‑chip cube. Google has described configurations that scale to 256‑chip and multi‑thousand‑chip pods, with Jupiter providing a leaf‑spine network to connect multiple pods into larger clusters. At scale, a full TPU v7 pod delivers tens of exaFLOPS of low‑precision compute aimed squarely at large‑scale LLM training and multi‑tenant inference.
Google emphasizes that TPU v7 is not only a training engine. Ironwood is also tuned for inference‑heavy tasks: model serving, retrieval‑augmented generation, and latency‑sensitive ranking. That pushes design effort toward memory bandwidth, interconnect latency, and scheduling for batch‑size‑one scenarios, not just throughput at very large batch sizes.
Within Google’s AI hypercomputer architecture, TPU v7 pods function as modular building blocks. Tight coupling with Jupiter networking and facility‑level liquid cooling means Google can scale pods up for internal workloads like Search and YouTube recommendations or carve them into slices for Google Cloud customers.
TPU v7 Software Stack and AI Workload Focus
The other half of TPU v7’s value resides in software. Google has built a TPU‑centric ecosystem around XLA, TensorFlow, JAX, and, more recently, improved PyTorch support. With Ironwood, that stack extends deeper into popular inference tools such as vLLM and optimized libraries for LLM decoding, quantization, and sparsity.
This co‑design yields high utilization on real workloads. Compiler passes, kernel fusion strategies, and memory planners can be tuned to TPU v7’s systolic arrays and HBM bandwidth profiles. In practice, this means production‑grade models at Google can hit utilization levels that would be difficult to match on a generic accelerator not designed with the same workload traces in mind.
The trade‑off is friction for developers and enterprises steeped in GPU‑first ecosystems. CUDA remains the dominant target for many open‑source frameworks, and adapting complex model pipelines to TPU execution semantics and XLA can require non‑trivial work. For Google’s own products, that investment is amortized across massive internal workloads. For external customers, TPU v7‑backed instances offer attractive economics primarily when workloads are large, relatively stable, and worth the porting effort.
Inside Azure Cobalt 200: Microsoft’s Arm CPU for Cloud Compute
If TPU v7 Ironwood is Google’s dedicated AI engine, Azure Cobalt 200 is Microsoft’s answer to refreshing baseline cloud compute without leaving economics and perf/W entirely in the hands of x86 incumbents. Azure continues to deploy Intel and AMD CPUs broadly; Cobalt 200 is designed to sit beside them, taking share wherever Arm’s efficiency wins.
Azure Cobalt 200 Neoverse V3 Cores: Design Goals and Positioning
Compared with Google TPU v7’s focus on AI acceleration, Azure Cobalt 200 anchors Microsoft’s general‑purpose cloud compute strategy on custom Arm silicon. Cobalt 200 builds around Arm’s Neoverse V3 server core, tuned for cloud workloads. Public descriptions point to a design with roughly 128 Neoverse V3 cores, significant private L2 caches, and a large shared last‑level cache in the hundreds of megabytes, fed by multiple DDR5 memory channels, as outlined in ServeTheHome’s analysis.
The chip is manufactured at TSMC on the N3E process, placing it on a comparable node to leading custom CPUs and accelerators. The design goal is straightforward: maximize perf/W for highly parallel, mostly integer‑heavy cloud workloads such as microservices, web front‑ends, and many database and analytics tasks. Microsoft indicates a thermal envelope similar to modern x86 server CPUs but claims that, for a given performance target, Cobalt 200 can operate at lower power, cutting rack‑level energy use.
In Azure’s SKU stack, Cobalt 200 goes head‑to‑head with Intel Xeon and AMD EPYC in general‑purpose families. The Arm server ecosystem has matured to the point where most mainstream Linux workloads can migrate with minimal changes. Microsoft presents Cobalt‑backed VMs as drop‑in candidates for many containers and PaaS offerings while promising better price‑performance for customers who switch.
Cloud Services and Workloads Optimized for Azure Cobalt 200
Cobalt 200 fits most naturally into general‑purpose VM families, container‑oriented services like Azure Kubernetes Service, and higher‑level PaaS layers where the CPU runs application logic rather than dense numeric kernels. Stateless microservices, API gateways, web servers, and a wide range of online transaction processing applications map well onto its high core counts and cache structure.
Databases and analytics are another focus area. Here, abundant cores and a large shared cache help with query parallelism and in‑memory datasets. Combined with fast networking and storage, Cobalt‑powered instances can offer competitive latency and throughput while staying within tighter power budgets than many legacy x86 configurations.
Cobalt 200 is also a foundational piece of Azure’s heterogeneous architecture. Alongside Cobalt CPUs, Microsoft is deploying Maia accelerators for AI, plus GPUs from Nvidia and AMD. The longer‑term picture is a mix of custom Arm CPUs for control planes and many user workloads, specialized accelerators for AI, and smartNICs or DPUs for offloading networking and storage—all tuned to Azure’s own traffic patterns.
What Azure Cobalt 200 Means for Arm in the Data Center
Azure Cobalt 200 also sends a signal to the broader Arm ecosystem. AWS’s Graviton series demonstrated that Arm server CPUs can sustain large‑scale production traffic; Microsoft’s Neoverse‑based chip shows that Arm has fully entered the mainstream at multiple top clouds.
For Arm Ltd., traction for Neoverse V3 within Azure validates its roadmap for higher‑performance cloud cores and its licensing model for custom silicon. For Intel and AMD, Cobalt 200 sharpens the bar they must clear to retain share of Azure’s general‑purpose compute tiers. Perf/W, SKU flexibility, and deeper customization around cloud needs are now table stakes.
Further down the stack, Cobalt 200 may encourage regional providers and on‑prem operators to revisit Arm servers—either via merchant Neoverse platforms or semi‑custom collaborations. Still, few can match Microsoft’s ability to co‑design chips, boards, firmware, and services around a single CPU line.
Vertical Integration: How Custom Silicon Lets Hyperscalers Control the Stack
Google TPU v7 and Azure Cobalt 200 join Google’s Axion CPU, AWS’s Graviton and Trainium/Inferentia lines, and Meta’s in‑house accelerators in a broader shift: hyperscalers are no longer primarily component buyers but full‑stack platform designers. Together, Google TPU v7 and Azure Cobalt 200 illustrate how custom silicon underpins a vertical integration play where clouds design everything from chips and interconnects to cooling and managed services.
A decade ago, large clouds mostly bought commodity x86 CPUs, third‑party GPUs, and off‑the‑shelf NICs and storage controllers. Over time, they pushed into custom networking gear, smartNICs and DPUs, programmable switches, and specialized storage silicon to squeeze more efficiency from each rack. The final step was to own the compute dies at the center of the system.
Today, that stack spans silicon, boards, racks, power distribution, cooling, compilers, runtimes, and managed cloud solutions. TPU v7 is optimized not just as a chip, but as a tile in a Jupiter‑connected pod fed by specific liquid‑cooling infrastructure. Cobalt 200 is tuned not just as a CPU, but as the default engine behind particular VM families, billing models, and service‑level objectives.
Economic Logic: TCO, Capex Efficiency, and Silicon Supply Security
The economic rationale is clear at hyperscale. Once a cloud provider deploys hundreds of thousands of identical chips, shaving a few percent off power, raising average utilization, or reducing exposure to merchant margins can compound into very large savings.
Custom silicon allows hyperscalers to optimize for their precise workload mix and utilization patterns instead of the industry average. Google can tune TPU v7 for the shapes of its LLM serving traffic, search ranking models, and ads pipelines. Microsoft can shape Cobalt 200 around the instruction mixes and cache behaviors of its own Azure workloads. That kind of specificity is challenging for a merchant CPU or GPU vendor to serve without fragmenting its product line.
Owning critical silicon also yields bargaining leverage and supply resilience. When capacity at leading‑edge foundries or HBM vendors is tight, hyperscalers with large, predictable orders for their own chips can negotiate priority. They can also decide how to allocate scarce HBM stacks or CoWoS packaging slots between in‑house parts and purchased accelerators, potentially prioritizing the former when supply is constrained.
Custom chips further enable differentiated features—security enclaves, confidential computing extensions, tenant isolation, and proprietary compression or data‑movement engines—that competitors cannot easily clone. Over time, those features become attributes of the cloud platform itself. For readers tracking these trends across vendors, an overview of how hyperscalers use in‑house silicon helps contextualize where TPU v7 and Cobalt 200 fit.
How Custom Silicon Like TPU v7 and Cobalt 200 Reshapes AI and Cloud Hardware
As Google, Microsoft, and peers double down on bespoke silicon, the impact cascades through the broader hardware ecosystem and supply chain.
Pressure on Nvidia, AMD, and x86 from TPU v7 and Azure Cobalt 200
For Nvidia and AMD, TPU‑class accelerators and Trainium‑style chips mean a smaller share of hyperscaler capex is available for merchant GPUs. The accelerators that are purchased must clear a higher bar—stronger ecosystems, absolute performance leadership, or unique capabilities—to justify their premium.
Owning silicon gives hyperscalers a credible outside option in negotiations. That can translate into demands for semi‑custom GPU SKUs, specialized firmware, and more aggressive pricing structures. In response, merchant vendors are tightening partnerships with foundries and OSATs, advancing aggressive roadmaps, and pushing deeper into cloud‑native software integration to keep their platforms compelling.
On the CPU side, Azure Cobalt 200 and similar Arm‑based designs erode what was once a near‑monopoly for x86 in general‑purpose compute. Merchant x86 CPUs remain common, but they now compete directly with cloud‑owned Arm designs that set new baselines for perf/W and price‑performance in high‑volume tiers.
Foundry, Packaging, and Cooling as New Strategic Battlegrounds
The gravity of custom silicon pulls strategic competition down into manufacturing, packaging, and facility engineering. TPU v7 depends on access to TSMC’s N3P node, CoWoS packaging, and HBM3e supply; Cobalt 200 leans on N3E capacity and advanced substrate technology, as highlighted in detailed hardware teardowns on sites like ServeTheHome.
Capacity constraints at any of these layers can bottleneck deployments. Hyperscalers increasingly reserve wafer starts and advanced packaging slots years ahead, co‑invest in fab and OSAT expansions, and design SKUs that fit around supply limits—for example, variants that use fewer HBM stacks where possible.
Cooling innovations are part of the same battleground. Direct‑to‑chip liquid loops, rear‑door heat exchangers, and immersion cooling all offer avenues to raise rack densities, but each requires heavy facility capex and operational retraining. The emerging result is a split between accelerator‑dense data centers designed around liquid cooling and more conventional air‑cooled facilities that remain x86‑centric.
Impact on Smaller Clouds, Enterprises, and OEMs
Outside the top hyperscalers, custom silicon can feel like the ground shifting. Hardware standards fragment as each big cloud exposes a different mix of accelerators and CPUs with proprietary performance characteristics. For enterprises consuming cloud services, that complicates portability and raises lock‑in concerns when moving between providers or regions.
For OEMs and white‑box vendors, the challenge is sharper. If the highest‑volume, most profitable workloads move onto in‑house silicon inside hyperscaler facilities, the addressable market for generic server platforms shrinks or migrates to niches such as sovereign clouds, regulated industries, or specialized vertical stacks. Those niches can still sustain healthy businesses, but they require different strategies than selling commodity x86 fleets.
New opportunities emerge in adjacent layers: Arm server platforms tailored to regional providers, DPUs and smartNICs that help smaller clouds emulate hyperscale‑style offload architectures, and software that abstracts away hardware heterogeneity and targets multiple backends, from GPUs to TPUs to custom Arm CPUs.
Strategic Risks and Constraints of Custom Silicon for Hyperscalers
The upside of Google TPU v7 and Azure Cobalt 200 is substantial, but so are the risks inherent in deep silicon bets.
On the engineering side, multi‑year design cycles mean that missing a process node window or mis‑forecasting workload trends can strand a chip behind merchant alternatives. A TPU optimized too tightly for one class of model may underperform if architectures shift. Sustaining robust developer ecosystems for non‑standard hardware is also hard; if tooling or frameworks lag behind GPU‑centric stacks, utilization and customer adoption suffer.
Integration risk is another dimension. Custom chips must align with firmware, host operating systems, compilers, orchestration frameworks, and observability tools. Misalignment anywhere along this chain can appear as fragility or reliability issues at scale. Hyperscalers have the talent and capital to address these challenges, but every new silicon line increases the surface area for potential failures.
Policy and regulatory risks are rising as well. Authorities in multiple regions are scrutinizing hyperscaler market power and may see deep vertical integration—from chips through cloud services and software platforms—as reinforcing that power. Export controls and IP regulations can complicate cross‑border deployments or supply arrangements. Data localization rules may require specific chip types in certain jurisdictions, constraining how providers allocate scarce TPU, GPU, or custom CPU capacity.
What Comes Next for TPU v7, Azure Cobalt 200, and Custom Cloud Silicon
Looking toward the later 2020s, Google TPU v7 and Azure Cobalt 200 are best read as stepping stones toward heterogeneous, workload‑tuned clouds. If current trends hold, future generations of Google TPU and Azure Cobalt will deepen the split between custom accelerators for AI clusters and custom Arm CPUs for scale‑out services, leaving merchant GPUs and x86 CPUs to compete in more specialized or lower‑volume roles.
Cloud architectures are likely to converge on mixes of custom CPUs, AI accelerators, DPUs, and specialized storage and networking silicon. TPU‑class accelerators will anchor AI‑first data center clusters designed around high‑density liquid cooling and HBM‑rich packages. Cobalt‑class Arm CPUs will shoulder a growing share of general‑purpose compute, especially for microservices and data‑plane tasks where perf/W is critical.
For enterprises, this implies that underlying hardware in public clouds will become more differentiated and less interchangeable. Buyers will need to invest in benchmarking and observability to understand how workloads behave on TPU‑, GPU‑, or Arm‑backed instances, and to negotiate SLAs and pricing that reflect real perf/W and utilization rather than nominal core counts or FLOP figures. Concerns about portability will intensify, pushing more organizations toward containers, higher‑level abstractions, and multi‑cloud strategies that can span heterogeneous hardware.
Several signals will show whether the custom silicon trend continues to accelerate: the cadence of new chip announcements and tape‑outs from major clouds; capex breakdowns that reveal how much spend shifts to in‑house silicon; and the degree of standardization around software and interconnects that either lock customers in or make it easier to target multiple vendors’ accelerators.
On balance, the mid‑term forecast is that custom silicon will become the default for top‑tier workloads at the largest clouds, while merchant GPUs and CPUs increasingly compete in the bands around those proprietary cores. TPU v7 Ironwood and Azure Cobalt 200 are early, visible proofs that hyperscalers can deliver credible, high‑volume parts on leading process nodes. As long as they can secure foundry and packaging capacity and AI and cloud demand stay robust, the industry vector points toward deeper vertical integration, more heterogeneous racks, and a data center landscape where differentiation starts at the transistor level.
