r/LocalLLaMAJul 23, 2026, 08:04 UTC+8

Multi-node GPU inference at 30 tok/s over a $20 USB-to-Ethernet adapter

FYI You dont need expensive networking for multi-node gpu. 30t/s laguna Q2_K_XL (39.7GB) on 2x4060+1x4060 using a $20 usb->ethernet.

A Reddit user ran the 39.7 GB laguna Q2_K_XL model across two nodes with three RTX 4060 GPUs, connected only by a direct Ethernet cable. At ubatch 768, generation reached 28.28 tok/s on an 11k-token prompt, with network traffic peaking at 30–70 MB/s. The post includes NCCL+RPC build flags and ubatch comparisons, but does not provide a single-machine dual-GPU baseline for the same model.

Why it matters: A concrete, low-cost multi-node inference experiment: three RTX 4060s over a $20 USB Ethernet adapter hitting 28-30 tok/s, directly challenging the assumption that fast networking is mandatory. Hits all three HKR axes, but it's a community validation rather than a product laun...

Read the original ↗Export Markdown

Take

This post caught my eye because it lowers the bar for multi-GPU inference to almost nothing. A Reddit user connected two PCs with three RTX 4060s using a plain Ethernet cable and ran the 39.7 GB laguna Q2_K_XL model. On an 11k-token prompt with ubatch 768, generation hit 28.28 tok/s, and network traffic peaked at just 30–70 MB/s — nowhere near saturating a gigabit link.

I'd discount it a bit: the post doesn't include a single-machine dual-GPU baseline for the same model, so we can't tell how much speed the second node actually costs. Also, three 4060s total 24 GB VRAM while the model is 39.7 GB, so it's clearly spilling into system RAM — the real bottleneck might be PCIe bandwidth, not the network.

Still, the direction is solid. If you've got two old machines lying around, a $20 cable can pool their VRAM to run models that wouldn't fit on either one alone. No switch, no InfiniBand. For the local LLM crowd, that's a genuinely useful reference point.

Source: r/LocalLLaMA · reddit.com
Published: Jul 23, 2026, 08:04 UTC+8 · Jul 23, 2026