Show HN: Needle Distills Gemini Tool Calling into a 26M Model
Show HN: Needle: We Distilled Gemini Tool Calling into a 26M Model
Cactus open-sourced Needle, a 26M-parameter tool-calling model that reaches 6,000 tok/s prefill and 1,200 tok/s decode on consumer devices, with MIT-licensed weights released on Hugging Face.
Why it matters: Cactus distilled Gemini's tool calling into a 26M-parameter model that runs locally on consumer devices at 6,000 tokens per second prefill and 1,200 per second generation.