Google open-sources TorchTPU, a native PyTorch back end for TPUs
107 points by ilreb 3 hours ago | 28 comments
fancyfredbot 33 minutes ago
Cool! XLA is pretty good + potentially a better compiler infrastructure than inductor which will lead to improved performance on all hardware.
replyAlso I guess this is the final nail in the coffin of Tensorflow.
pcmoritz 36 minutes ago
For people interested in this: I've been working on an open-source libtpu implementation for https://tenstorrent.com/ accelerators as a side project, it can already run most of torchtpu, see https://github.com/pcmoritz/libtt (based on the tenstorrent software stack). It shows all the things involved in lowering torch to tile based accelerators.
replyyipinwong 2 hours ago
Dumb question from non-AI engineer.
replyIs this somehow Google's answer to today's best DeepSeek open-sourcing inferencing engine?
bitpush 2 hours ago
I dont think big companies like Google react to news like that. Very likely this was meant to go out on Tuesday for weeks now. And it just happened to have coincided with another news.
replyJacobAsmuth 57 minutes ago
Yeah they read the news this morning and whipped up a quick Torch->XLA translation layer and coordinated with four different inference engines (who also all wrote their own blog posts) and shoved this out the door quick to stay on top of the news cycle
replyparthdesai 44 minutes ago
Why be rude on purpose? OP clearly prefaced by saying it was a dumb question
replymathisfun123 54 minutes ago
they're releasing this today because the pytorch conf is this week...
replyTanjreeve 2 hours ago
They do different things and sit in different parts of the stack (training vs inference). TorchTPU seems more like a play to get more people familiar with Googles TPU chips as they want to sell them to people outside of google.
replybeardedetim 33 minutes ago
Yeah I'd assume this is to allow us to train on their TPUs with torch instead of needing to rent a GPU
replynelaggy 2 hours ago
so if the moat isn't the model and the moat isn't the inference layer, i guess the moat is the hardware now? what does this mean for frontier labs?
replyLucasoato 3 hours ago
Are there TPUs outside Google Cloud?
replyboulos 2 hours ago
Yes. Google is famously selling TPUs on prem to big customers (e.g., Anthropic) and Crux AI is now a venture to do just that.
replywesturner 2 hours ago
TPU / neural compute accelerator / Math coprocessor;
reply"OpenTPU: Open-Source Reimplementation of Google Tensor Processing Unit (TPU)" https://news.ycombinator.com/item?id=44111452
From https://news.ycombinator.com/item?id=45848676 :
> How does Cerebras WSE-3 with 44GB of 'L2' on-chip SRAM compare to Google's TPUs, Tesla's TPUs, NorthPole, Groq LPU, Tenstorrent's, and AMD's NPU designs?
There are TPUs made out of silicon carbide with standard fab.
There is in-flight tensor processing with photons, but that's not quite a TPU.
hackernudes 56 minutes ago
I think the commenter was asking if Google sells TPU chip hardware to use outside of Google's own infrastructure and cloud.
reply
https://github.com/aws-neuron/torch-neuronx