Intel’s Gaudi 3 datacenter GPU from late 2024 advertises about 1800 tops in fp8, at 3.1 tops/w. Google’s mid 2025 TPU v7 advertises 4600 tops fp8, at 4.7 tops/w. Which is a difference, but not that dramatic of one. The reason it is so small is that GPUs are basically TPUs already; almost as much die space as is allocated to actual shader units is allocated to matrix accelerators. I have heard anecdotally.
Yes, it works out to a ton of power and money, but on the other hand, 2x the computation could be like a few percent better in results. so it’s often a thing of orders of magnitude, because that’s what is needed for a sufficiently noticeable difference in use.
basing things on theoretical tops is also not particularly equivalent to performance in actual use, it just gives a very general idea of a perfect workload.
Intel’s Gaudi 3 datacenter GPU from late 2024 advertises about 1800 tops in fp8, at 3.1 tops/w. Google’s mid 2025 TPU v7 advertises 4600 tops fp8, at 4.7 tops/w. Which is a difference, but not that dramatic of one. The reason it is so small is that GPUs are basically TPUs already; almost as much die space as is allocated to actual shader units is allocated to matrix accelerators. I have heard anecdotally.
At scale the power efficiency is probably really important though
Yes, it works out to a ton of power and money, but on the other hand, 2x the computation could be like a few percent better in results. so it’s often a thing of orders of magnitude, because that’s what is needed for a sufficiently noticeable difference in use.
basing things on theoretical tops is also not particularly equivalent to performance in actual use, it just gives a very general idea of a perfect workload.