Google releases TorchTitan fork for TPU with optimization guide
What happened
Reported October 20, Google released a TorchTitan fork for Google Cloud TPU, along with distributed training recipes and a step-by-step model optimization guide. SimpleFSDP, Graph Trainer and Pallas kernels handle sharding, full-step graph capture and compute optimization. In the FLUX.1 case, a training step fell from 11.49 seconds to 2.78 seconds, reaching about 99% of JAX MaxDiffusion throughput on the same hardware.
Written by AI from the coverage · updated 1 hour ago
Coverage
Follow the reports to see the story from different sides.
- AI HOT · Tips & opinionsGoogle 发布 TorchTitan TPU 分叉及模型优化指南
Google 发布适配 Google Cloud TPU 的 TorchTitan 分叉，提供分布式训练配方及按步骤执行的模型优化指南。SimpleFSDP、Graph Trainer 和 Pallas 内核支持分片、整步图捕获及计算优化，FLUX.1 案例的训练步耗时从 11.49 秒降至 2.78 秒，达到同硬件 JAX MaxDiffusion 吞吐量的约 99%。
Heat over time
Not enough continuous observations to draw a trend yet.