Skip to content
r/LocalLLaMA

LLMs on Android: Snapdragon 8 Elite MoE Experience

LLM's on Android (Snapdragon 8 Elite) MOE Experience

A Reddit user tested MoE LLMs on an Honor Magic 7 Pro with Snapdragon 8 Elite and 24GB RAM; under Q4 quantization, LFM2-24b-a2b reached about 24 tokens/s while Gemma reached about 11 tokens/s, and CPU inference was still faster than NPU or GPU in the reported setup.

Why it matters: A Reddit user's real measurements of MoE models on a Snapdragon 8 Elite phone, where CPU inference beat the NPU and LFM2-24b-a2b ran near 24 tok/s at Q4.

Read the original ↗Export Markdown