Skip to content
Hacker News front page

Show HN: Needle Distills Gemini Tool Calling into a 26M Model

Show HN: Needle: We Distilled Gemini Tool Calling into a 26M Model

Cactus open-sourced Needle, a 26M-parameter tool-calling model that reaches 6,000 tok/s prefill and 1,200 tok/s decode on consumer devices, with MIT-licensed weights released on Hugging Face.

Why it matters: Cactus distilled Gemini's tool calling into a 26M-parameter model that runs locally on consumer devices at 6,000 tokens per second prefill and 1,200 per second generation.

Read the original ↗Export Markdown