Skip to content
r/LocalLLaMA

Computer Build Using Intel Optane Persistent Memory Runs a 1T-Parameter Model at Over 4 Tokens/s

Computer build using Intel Optane Persistent Memory - Can run 1 trillion parameter model at over 4 tokens/sec

Reddit user APFrisco ran the 1T-parameter Kimi K2.5 Q2_K_XL quant locally at about 4 tokens/s using 768GB Intel Optane PMem, 192GB DDR4 ECC DRAM, and a 12GB RTX 3060 with llama.cpp hybrid GPU/CPU inference.

Why it matters: A Reddit build pairs 768GB of Intel Optane persistent memory with an RTX 3060 to run a 1-trillion-parameter model locally at over 4 tokens/sec, showing llama.cpp hybrid inference on cheap hardware.

Read the original ↗Export Markdown