LLMs on Android: Snapdragon 8 Elite MoE Experience
LLM's on Android (Snapdragon 8 Elite) MOE Experience
A Reddit user tested MoE LLMs on an Honor Magic 7 Pro with Snapdragon 8 Elite and 24GB RAM; under Q4 quantization, LFM2-24b-a2b reached about 24 tokens/s while Gemma reached about 11 tokens/s, and CPU inference was still faster than NPU or GPU in the reported setup.
Why it matters: A Reddit user's real measurements of MoE models on a Snapdragon 8 Elite phone, where CPU inference beat the NPU and LFM2-24b-a2b ran near 24 tok/s at Q4.