Google made TorchTPU available as an open-source project on 20 October, offering PyTorch developers a configuration-only route to its Tensor Processing Units (TPUs) one day after Amazon released a comparable tool for Trainium, TechCrunch reported. The announcement gives developers a way to move existing applications to Google’s chips and cloud without rewriting them.
Key points
- Google’s TorchTPU lets developers move existing PyTorch applications to TPUs through a configuration change.
- Amazon released TorchNeuron on 19 October to bring PyTorch applications to its Trainium chips without rewrites.
- Amazon says more than 30 early customers are using TorchNeuron.
- Google introduced a TPU agent suite and updated LiteRT to support developers using PyTorch.
TorchTPU targets existing PyTorch applications
PyTorch is an open-source toolkit for building, training and running AI models. Meta developed it in 2016 and transferred it to the Linux Foundation in 2022. Many of its features were built to work with Nvidia GPUs, which were already standard hardware for AI research when PyTorch was developed.
Most AI applications are written by default for Nvidia GPUs, according to TechCrunch. TorchTPU addresses the work of moving an existing PyTorch application to different hardware: developers can change its configuration rather than rewrite the application to run on Google’s TPUs. The same move takes the application to Google Cloud.
Google says it worked with Meta and Hugging Face, as well as Inferact, the maker of vLLM, and RadixArk, the maker of SGLang. The company says the work was intended to make TorchTPU’s features and code libraries compatible with those AI development tools. That work sits alongside the configuration change in Google’s effort to make its chips usable with software developers already employ.
Amazon’s TorchNeuron offers a Trainium route
Amazon released TorchNeuron on 19 October, offering developers a way to move PyTorch applications to Trainium and AWS without rewriting them. AI Affairs previously reported Amazon’s no-rewrite pitch for Trainium. Both companies are using PyTorch compatibility to make their own chips an option for developers whose applications were built around the toolkit.
Amazon is also submitting code to the main PyTorch project to separate various features from Nvidia’s chips. Unlike TorchNeuron, which offers a route to Amazon’s own hardware, those proposed changes are aimed at making PyTorch less tied to particular chips by default, TechCrunch reported.
Amazon says more than 30 early customers in robotics, world models and voice AI are using TorchNeuron, Benzinga reported. The customer count describes use of Amazon’s developer tool. Google, for its part, is offering migration assistance alongside TorchTPU rather than relying on the open-source project alone.
Google adds a TPU agent suite and LiteRT support
Google launched a TPU agent suite to help developers move applications to its chips and cloud and address errors during that process. The suite adds a migration tool to the changes TorchTPU makes possible through an application’s configuration.
Google also updated LiteRT, its tool for running AI on phones, laptops and embedded devices, to work better with PyTorch applications.
