Skip to content
r/LocalLLaMA

New MLX LM Server From Apple

A Reddit post says Apple’s MLX LM Server uses continuous batching for concurrent sub-agent requests and supports distributed inference across multiple Macs via Thunderbolt RDMA.

Why it matters: Apple's MLX LM Server adds continuous batching and Thunderbolt-linked distributed inference across Macs, a concrete option for anyone running local models.

Read the original ↗Export Markdown