Benchmarks of 20 Small LLMs on a 6GB RTX 4050
Benchmarks of 20 small LLMs on a 6GB RTX 4050
The author benchmarked 20 small LLMs on a 6GB RTX 4050 using LM Studio’s OpenAI-compatible API, with N=5 speed runs at 1k, 8k, and 32k context; unsloth/lfm2.5-vl-1.6b led throughput at 207 tok/s on 1k context while using 3.0GB VRAM.
Why it matters: A rare speed reference for running 20 small models on a 6GB laptop GPU, tested at 1k, 8k and 32k context.