GLM built its own inference infra on 100k+ Chinese accelerators, tripling throughput in under two weeks

GLM Built Its Own Inference Infrastructure

Zhipu AI disclosed how GLM-5.3-Flash inference was built from scratch on a cluster of over 100,000 Chinese-made AI accelerators. The team faced limited chip memory, low bandwidth, and an immature software ecosystem. Instead of relying solely on human engineers, they deployed an Infra Agent powered by GLM-5.3 that turned sparse end-to-end metrics into fine-grained, attributable feedback—kernel-level correctness checks, microbenchmarks, and execution traces—so the agent could pinpoint bottlenecks. Combined with tensor parallelism, W8A8 quantization, mixed-precision KV cache, and an Encode-Prefill-Decode disaggregated architecture, end-to-end throughput improved roughly 3× over the initial baseline, with per-token cost reaching parity with mainstream NVIDIA GPUs. Within a week of launch under the anonymous name Ox-Alpha, the model processed over 62 trillion tokens and became the most-used model on both OpenCode and OpenRouter.

Why it matters: Zhipu used GLM-5.3 as an agent to debug its own inference stack on 100k+ domestic accelerators — concrete technical path with real numbers (W8A8 quantization), not a PR piece. All three HKR axes hit, but the excerpt cuts off before key performance and stability metrics, so thi...