Hetzner experiments with an LLM inference API
Hetzner is working on LLM Inference
Hetzner launched an experimental, OpenAI-compatible inference API with a single model: Qwen 3.6 35B FP8. The author measured 153 ms median time-to-first-token and 224 tokens/sec output speed—fast, but the model failed simple arithmetic. There is no billing or SLA yet; Hetzner says it wants to learn about demand, scaling, and load. The more interesting angle is Hetzner's potential to turn spare GPU capacity into a low-margin inference commodity, given its cost-efficient hardware operations. The post does not confirm which GPUs power the service; Hetzner's public GPU lineup includes RTX 4000 SFF (20 GB) and RTX PRO 6000 (96 GB), leaving the hardware question open.
Why it matters: Hetzner's experimental OpenAI-compatible inference API serves only Qwen 3.6 35B FP8, with a measured 153 ms median first-token latency and 224 tokens per second, and no billing or SLA.