Shipping a Trillion Parameters With a Hub Bucket: Delta Weight Sync in TRL
通过万亿参数与 Hub Bucket 实现增量权重同步:TRL 中的增量权重同步
Hugging Face merged TRL PR 5417 for delta weight sync, sending only changed weights as sparse safetensors via a Hugging Face Bucket; on Qwen3-0.6B, the per-step payload falls from 1.2GB to 20–35MB.
Why it matters: HKR-H/K/R all pass: TRL gets delta weight sync with a concrete sparse-safetensors mechanism and a 1.2GB to 20–35MB example. Scope is training infra, so it stays below must-write.