TPUv4 Interconnect Cuts Cost, Power

Google employs optical interconnect when building a 4,096-node AI supercomputer featuring its TPUv4 accelerator chip. The result is much higher potential performance coupled with a sharp drop in carbon emissions.
Bryon Moyer
Bryon Moyer

Google’s latest AI tensor-processing chip, TPUv4, quadruples the size of the computing block for both AI training and inference as compared with the prior generation. Optical interconnect enables the larger size as well as higher performance and performance efficiency.

The giant hyperscaler built a supercomputer of 4,096 interconnected TPUv4 nodes. Arranged in a 3D torus, the topology balances the performance needs of older and newer algorithms. Running signals in both directions along a fiber reduces cost, halving the number of fibers in the system. Overall, the interconnect reduces latency and power relative to prior TPU versions.

Although Google deployed the technology in 2020, it’s only now disclosing details in a paper at the 2023 International Symposium on Computer Architecture (ISCA). The paper describes a rack-size 4x4x4 cube of TPUv4 nodes as a building block. Electrical connections serve within the rack; optical links connect nodes from different racks. Optical circuit switches create end-to-end optical links with no need for packet manipulations along the way.

Overall performance of the resulting supercomputer is at least 10x that of TPUv3’s smaller supercomputer. Energy consumption drops by as much as 83%, with an estimated drop in carbon emissions (CO2e) of 95% per node.

Free Newsletter

Get the latest analysis of new developments in semiconductor market and research analysis.

Subscribers can view the full article in the TechInsights Platform.


You must be a subscriber to access the Manufacturing Analysis reports & services.

If you are not a subscriber, you should be! Enter your email below to contact us about access.

The authoritative information platform to the semiconductor industry.

Discover why TechInsights stands as the semiconductor industry's most trusted source for actionable, in-depth intelligence.