Skip to content

All news

1 today

May 20Wednesday

AI HOT (Curated Pool)

Gemini 3.5 Released: A New Model Family Combining Intelligence and Action

Google AI Developers announced the Gemini 3.5 model family, saying it combines intelligence with action capabilities; the post does not disclose parameters, benchmarks, pricing, availability, or context window details.

Why it matters: Google's Gemini 3.5 family merges reasoning and execution for business workflows, but the announcement gives no parameters, benchmarks, pricing or launch date.

Hacker News front page

Gemini 3.5 Flash

The title names Gemini 3.5 Flash, while the RSS body only includes a documentation link; the Hacker News item has 196 points and 179 comments, and the post does not disclose parameters, pricing, or context-window details.

Why it matters: Google's Gemini 3.5 series shifts the model from answering questions to carrying out actions, drawing 196 upvotes and 179 comments on Hacker News.

r/LocalLLaMA

KV cache quantization benchmarks: TurboQuant is overrated, q5 deserves attention, q8 may waste VRAM

Anbeeld benchmarked KV cache quantization for Qwen 3.6 27B on one RTX 3090 at 64k and 128k context, reporting q4_0 tail KLD 32% worse than q5_0 and turbo4 running 17% slower than q4_0 with little memory saving.

Why it matters: Anbeeld benchmarks Qwen 3.6 27B KV cache quantization on an RTX 3090, with numbers like q4_0's tail KLD 32% worse than q5_0.

r/LocalLLaMA

Public repository Codegraph claims 94% fewer Claude, Cursor, Codex, and OpenCode tool calls locally

Codegraph uses a pre-indexed knowledge graph for symbol relationships, call graphs, and code structure. In the VS Code test, it reduced tool calls from 52 to 3 and runtime from 1m37s to 17s.

Why it matters: Codegraph indexes symbols and call chains into a code knowledge graph, cutting a VS Code test from 52 tool calls to 3 and 1m37s to 17s.

May 19Tuesday

Hacker News front page

Show HN: Forge takes an 8B model from 53% to 99% on agentic tasks

Forge adds five guardrail layers to self-hosted LLM tool calling, raising Ministral 8B to 99.3% across 18 multi-step agentic scenarios, with the accepted ACM CAIS ’26 paper covering 97 model/backend configurations and 50 runs per scenario.

Why it matters: Forge used five layers of guardrails to lift Ministral 8B from 53% to 99.3% task success across 18 multi-step agent scenarios, covering 97 model and backend combinations.

r/LocalLLaMA

ByteDance released an open-source model that attempts broad multimodal tasks with 3B parameters

ByteDance released Lance, an open-source unified multimodal model with 3B active parameters that supports image and video understanding, generation, and editing, and the post says it was trained from scratch with a staged multi-task recipe under a 128-A100-GPU budget.

Why it matters: ByteDance open-sourced Lance, which puts image and video understanding, generation and editing in one model with 3B active parameters, plus the staged multi-task training recipe and 128-A100 setup.

AI HOT (Curated Pool)

Horizon Open-Sources 400M-Parameter Robot Control Model HoloMotion-1

Horizon Robotics Lab open-sourced HoloMotion-1, a 400M-parameter full-body humanoid control model that uses MoE sparse activation and KV-cache inference to reach about 300 FPS on-device, with code and a technical report released.

Why it matters: Horizon Robotics' lab open-sourced HoloMotion-1, a 400M-parameter whole-body control model for humanoid robots, reaching about 300FPS on-device via MoE sparse activation and KV-cache, well above the usual 50Hz control rate.

AI HOT (Curated Pool)

Cursor releases Composer 2.5, calling it its strongest model yet

Cursor released Composer 2.5, claiming a 10x efficiency gain at comparable capability, with larger training scale, more complex reinforcement-learning environments, and a text-feedback mechanism.

Why it matters: Cursor moved Composer to 2.5, claiming better handling of long tasks and complex instructions and 10x efficiency at equal capability, on an architecture shared with Kimi K2.5.

r/LocalLLaMA

21 GPUs benchmarked running a small TTS model, with 5GB peak VRAM

A Reddit user rented 21 GPUs on vast.ai to benchmark OmniVoice, a small TTS model with about 5GB peak VRAM, using xRT as the audio generation speed metric and averaging 3 voice-cloning runs with reference audio.

Why it matters: A Reddit user benchmarked 21 rented GPUs on a small voice-cloning TTS model, averaging three runs per card, as a buying reference for anyone building speech synthesis.

May 18Monday

AI HOT (Curated Pool)

The Open Agent Leaderboard

IBM Research published the Open Agent Leaderboard on Hugging Face to evaluate agents across language understanding, tool use, and multi-step reasoning tasks; the post does not disclose dataset size, model scores, or the evaluation date.

Why it matters: IBM Research put an open agent leaderboard on Hugging Face that scores whole systems across six scenario benchmarks and reports quality alongside running cost.

r/LocalLLaMA

I Tested 42 LLMs on Their Willingness to Build the Apocalypse

DystopiaBench tested 42 open and closed models across 36 escalating scenarios and 6 dystopia types, using 3 LLM-as-judge scorers and an average over 3 runs; the post says many models catch obvious dangerous requests but fail when risk is hidden behind dual-use framing and normalization.

Why it matters: The author ran 42 models through DystopiaBench's 36 escalating scenarios with three judge models, and many caved when the harmful ask was wrapped as dual-use work.

r/LocalLLaMA

Qwen 3.6 27B on 24GB VRAM: backend comparisons, quant choice, and settings

The author tested Qwen 3.6 27B on an RTX 3090 24GB and kept ik_llama.cpp with Qwen3.6-27B-MTP-IQ4_KS.gguf; at 156k context with q8_0 KV and MTP, a ~5.9k-token prompt plus 1024-token output reached about 1261 tok/s prefill and 72.9 tok/s decode, while vLLM lacked a clean single-card long-context run.

Why it matters: The author benchmarked several backends and quantization schemes for Qwen 3.6 27B on a single 24GB RTX 3090, settling on ik_llama.cpp with IQ4_KS, as a single-card comparison for local deployment.

r/LocalLLaMA

I built a coding agent that gets 87% on benchmarks with a 4B parameter model

SmallCode passes 87 of 100 benchmark tasks with Gemma 4 activating 4B parameters per token. The author attributes the result to compound tools, compile and lint feedback, task decomposition after two repeated failures, and optional escalation to Claude or OpenAI for one task.

Why it matters: The author built a coding agent on a 4B parameter model that solved 87 of 100 tasks by having it compile, run lint and fix errors, splitting a task into smaller subtasks after two failures.

r/LocalLLaMA

Benchmarking vLLM vs SGLang vs llama.cpp on a mixed Blackwell/Ada cluster

The author benchmarked long-context prefill on a 7-GPU mixed Blackwell/Ada cluster; on Qwen3.5-397B-A17B with 75k tokens, vLLM reached 9.8s TTFT and 7,683 t/s, while llama.cpp took 57.2s and 1,319 t/s.

Why it matters: A hands-on test of long-context prefill across seven mixed Blackwell and Ada GPUs, with vLLM hitting 9.8s first-token latency at 75k tokens and 7,683 tok/s prefill, roughly 4 to 6 times llama.cpp.

r/LocalLLaMA

LLMs on Android: Snapdragon 8 Elite MoE Experience

A Reddit user tested MoE LLMs on an Honor Magic 7 Pro with Snapdragon 8 Elite and 24GB RAM; under Q4 quantization, LFM2-24b-a2b reached about 24 tokens/s while Gemma reached about 11 tokens/s, and CPU inference was still faster than NPU or GPU in the reported setup.

Why it matters: A Reddit user's real measurements of MoE models on a Snapdragon 8 Elite phone, where CPU inference beat the NPU and LFM2-24b-a2b ran near 24 tok/s at Q4.

Google DeepMind

Google DeepMind releases Gemini Omni Flash video model

Google DeepMind released Gemini Omni Flash, the first model in the Gemini Omni family. It combines image, audio, video and text inputs to generate high-quality video, and supports multi-turn editing in natural language.

Why it matters: One model now handles image, audio, video and text input plus multi-turn editing, and it ships inside Gemini, Flow and Shorts.

May 17Sunday

r/LocalLLaMA

MiroThinker-1.7 Open-Weight Deep Research Agent Based on Qwen3 MoE

MiroMindAI released the MiroThinker-1.7-deepresearch and mini APIs, with the mini version using 30B total parameters and 3B active parameters, weights on HuggingFace, and context management based on sliding window K=5 plus episode restarts.

Why it matters: MiroMindAI released weights for its deep research agent MiroThinker-1.7 on HuggingFace, with a mini version at 30B total and 3B active parameters to lower the bar for local runs.

r/LocalLLaMA

85 GPU-hours comparing 5 abliteration methods on Qwen3.6-27B

Abliterlitics compared five Qwen3.6-27B abliteration variants against the base model using 85 GPU-hours of benchmarks, HarmBench, KL divergence, and weight forensics; Huihui had the smallest benchmark deltas, Heretic had the lowest KL divergence, and all five variants reached near-complete safety removal.

Why it matters: 85 GPU-hours on Qwen3.6-27B comparing five abliteration methods, with benchmarks, HarmBench safety evals, KL divergence and weight forensics all reported together.

r/LocalLLaMA

DeepSeek V4's 1M Context Window: The Breaking Point

A Reddit user tested DeepSeek V4 on 45k, 180k, and 520k-token codebases and found 150k-250k tokens best for coding work. Past 300k tokens, line-number precision degraded; at 520k, outputs shifted toward architecture summaries and skipped implementation details.

Why it matters: A Reddit user tested DeepSeek V4 on codebases of 45,000, 180,000 and 520,000 tokens, finding coding smoothest between 150,000 and 250,000 tokens and line-number accuracy slipping past 300,000.

r/LocalLLaMA

Same Models Tested Across Strix Halo, RTX 3090, and RTX 5070

C_Coffie published 55 local inference benchmark runs across Strix Halo, RTX 3090, RTX 5070, five backends, and 0.35B to 35B-A3B models; RTX 5070 beats RTX 3090 on models fitting 12GiB, while RTX 3090 leads in the 14–31B band that exceeds 12GiB but fits 24GiB.

Why it matters: A user ran 55 local inference tests across three GPUs and five backends, and reports an RTX 5070 decoding faster than an RTX 3090 within 12GB of VRAM.

AI HOT (Curated Pool)

Latest Open Artifacts #21: Gemma 4, DeepSeek V4, Kimi K2.6, MiMo 2.5, GLM-5.1, and More

Open AI model teams released Gemma 4, DeepSeek V4, Kimi K2.6, MiMo 2.5, GLM-5.1, and other versions this month, and the post says they were tested under CAISI’s V4 evaluation framework, but the RSS snippet does not disclose scores.

Why it matters: Five open models, including DeepSeek V4, Gemma 4 and Kimi K2.6, all shipped new versions in the same month, giving anyone tracking open-model release cadence and version choice a single list.

AI HOT (Curated Pool)

Ring-2.6-1T Open-Sourced and Listed on OpenRouter for Agent Workflows

AntLingAGI open-sourced Ring-2.6-1T and listed it on OpenRouter with a 75% discount through the end of May; the trillion-scale reasoning model targets agent workflows, including planning, tool use, context maintenance, and complex task execution, using Async RL and IcePop training methods.

Why it matters: AntLingAGI open-sourced its trillion-parameter Ring-2.6-1T model and put it on OpenRouter for agent workflows that plan steps, call tools and hold long context, at 25% off on the platform through the end of May.

May 16Saturday

Hacker News front page

SANA-WM, a 2.6B open-source world model for 1-minute 720p video

SANA-WM’s title says the project is a 2.6B open-source world model for 1-minute 720p video; the RSS body only lists the project URL, Hacker News comments URL, 9 points, and 8 comments, and the post does not disclose training data, license terms, inference cost, evaluation setup, or benchmark results.

Why it matters: NVIDIA open-sources SANA-WM, a 2.6B world model that turns one image plus a camera trajectory into a minute of 720p video, with a two-stage pipeline for long-video quality and motion consistency.

r/LocalLLaMA

Qwen3.6-35B-A3B and 9B land on the public Terminal-Bench 2.0 leaderboard

little-coder × Qwen3.6-35B-A3B scored 24.6% ±3.2 on Terminal-Bench 2.0, above Gemini 2.5 Pro on Gemini CLI at 19.6% and Qwen3-Coder-480B on Terminus 2 at 23.9%.

Why it matters: Qwen3.6 in 35B-A3B and 9B sizes lands on the public Terminal-Bench 2.0 leaderboard, giving small models a checkable public score on hard terminal-agent tests.

Google DeepMind

Google DeepMind releases Gemini 3.5 Flash

Google DeepMind released the Gemini 3.5 model family, with the first model, Gemini 3.5 Flash, available the same day in the Gemini app, Google Search AI Mode, Google Antigravity, the Gemini API and Gemini Enterprise.

Why it matters: Gemini 3.5 Flash beats Gemini 3.1 Pro on coding and agent benchmarks and outputs 4x faster than other frontier models, shipping the same day in the Gemini app, Search AI Mode and the Gemini API.

The Verge · AI

AI radio hosts demonstrate why AI can’t be trusted alone

Andon Labs had Claude, ChatGPT, Gemini, and Grok run separate radio stations with $20 in seed money each; the RSS snippet says all failed, but the post does not disclose the full experimental results.

Why it matters: Andon Labs gave Claude, ChatGPT, Gemini and Grok $20 each to run a radio station independently, and all four went off the rails unsupervised, offering a same-conditions sample of where autonomous agent operation breaks down.

May 15Friday

r/LocalLLaMA

Evaluated a RAG Chatbot: The Most Expensive Model Was the Worst Performer

The author evaluated a customer-support RAG bot and raised the quality score from 6.62 to 7.88 while cutting per-session cost from $0.002420 to $0.000509, using retrieval logging, LLM-as-judge scoring, chunk deduplication, stricter grounding, and a five-model sweep.

Why it matters: A builder used a model as judge to tune a customer-service RAG bot, lifting quality from 6.62 to 7.88 and cutting cost per session by nearly 80%, while the priciest model performed worst.

r/LocalLLaMA

Used over a million tokens in three sessions to test Qwen 3.6 35B MTP

A Reddit user tested Qwen3.6-35B-A3B MTP across three million-token-scale sessions, using 300k context and KV Q8_0, and reported about 1.5x the tok/sec of earlier tests.

Why it matters: A Reddit user ran over a million tokens across three sessions to test the MTP build of Qwen3.6-35B-A3B at 300k context and KV Q8_0 quantization, measuring roughly 1.5x the generation speed of the earlier version.

QbitAI · WeChat

Understand LeCun’s JEPA World Model in 160 Lines of Code

A developer released the keon/jepa teaching repository with five JEPA variants implemented as standalone PyTorch files, ranging from 160 to 278 lines, depending only on PyTorch and torchvision; the post reports iJEPA runs on CIFAR-10 for 100 epochs and reaches 52.7% linear-probe accuracy, while V-JEPA, C-JEPA, and LeWorldModel use toy or synthetic datasets.

Why it matters: The open teaching repo keon/jepa breaks LeCun's JEPA world model into five single-file PyTorch implementations, the shortest 160 lines, with iJEPA reaching 52.7% linear probe accuracy on CIFAR-10 after 100 epochs.

AI HOT (Curated Pool)

Granite Embedding Multilingual R2: Open Multilingual Embedding Model with 32K Context

IBM Granite released Granite Embedding Multilingual R2 on Hugging Face under Apache 2.0, with fewer than 100 million parameters, a 32K-token context length, and top same-scale retrieval performance on MTEB according to the post.

Why it matters: IBM open-sourced two Apache 2.0 multilingual embedding models on Hugging Face, with the 97M-parameter version scoring 60.3 on MTEB multilingual retrieval, a new reference point for lightweight multilingual search.

May 14Thursday

AI HOT (Curated Pool)

MiMo V2.5 Pro Places Third on DesignArena

MiMo V2.5 Pro placed third on the DesignArena overall leaderboard; its Thinking version rose 8 spots over MiMo-V2.5 and matched Claude Sonnet 4.6 performance on frontend coding tasks.

Why it matters: Xiaomi's MiMo V2.5 Pro climbed eight places to third on DesignArena, matching Claude Sonnet 4.6 on front-end coding tasks.

Xinzhiyuan · WeChat

Anthropic Overtakes OpenAI in Enterprise AI Adoption After Three Years

Ramp says Anthropic reached 34.4% enterprise adoption, surpassing OpenAI at 32.3% for the first time; the index is based on credit-card and invoice spending from more than 50,000 companies.

Why it matters: Ramp's tally of card and invoice spending across more than 50,000 companies puts Anthropic at 34.4% of enterprise share and OpenAI at 32.3%, the first time OpenAI has lost this lead in three years.

AI HOT (Curated Pool)

OpenSquilla Open-Source Project Uses Smart Routing and Local Retrieval to Cut LLM Costs

OpenSquilla combines local model routing, vector retrieval, incremental sending, and cache hits to reduce transmitted tokens by more than 90%, while routing simple tasks to cheaper models and complex tasks to stronger models without spending tokens on the routing decision.

Why it matters: OpenSquilla runs model routing decisions locally, judging question difficulty without spending tokens, and cuts actual transfer volume below a tenth through incremental sending and cache hits.

AI HOT (Curated Pool)

Anthropic overtakes OpenAI in B2B adoption for the first time, Ramp data shows

Ramp AI Index data shows Anthropic reached 34.4% adoption among U.S. enterprise customers, surpassing OpenAI’s 32.3% for the first time, while its business coverage grew fourfold over one year.

Why it matters: Ramp payment data shows Anthropic's paid adoption among US business customers hit 34.4% in April 2026, passing OpenAI's 32.3%, with its coverage quadrupling in a year.

r/LocalLLaMA

sensenova/SenseNova-U1-A3B-MoT · Hugging Face

SenseNova published SenseNova-U1-A3B-MoT on Hugging Face; the post lists A3B MoT, 8B MoT, and 0.4B LoRA weight links, and says the NEO-unify architecture unifies multimodal understanding, reasoning, and generation in one model family.

Why it matters: SenseTime released the SenseNova-U1-A3B-MoT weights on Hugging Face, using the NEO-unify architecture to fold multimodal understanding, reasoning and generation into one model family, plus 0.4B LoRA weights.

May 13Wednesday

TechCrunch · AI

Anthropic now has more business customers than OpenAI, according to Ramp data

Ramp’s survey of client expense data shows 34.4% of participating businesses pay for Anthropic services, while 32.3% pay for OpenAI; the snippet does not disclose sample size, customer segments, or spend levels.

Why it matters: Ramp counted business spending across its own customers: 34.4% pay Anthropic and 32.3% pay OpenAI, drawn from the same sample so the two numbers compare directly.

Xinzhiyuan · WeChat

Tsinghua-affiliated team open-sources MiniCPM-V 4.6, a 1.3B model tunable on one RTX 4090

ModelBest, Tsinghua University, and OpenBMB open-sourced MiniCPM-V 4.6, a 1.3B multimodal model that supports full fine-tuning on one RTX 4090 and offers 4x/16x visual token compression for accuracy or speed trade-offs.

Why it matters: ModelBest, Tsinghua and OpenBMB open-source MiniCPM-V 4.6 at 1.3B parameters, small enough to fully fine-tune on one RTX 4090, with 4x and 16x visual token compression modes.

r/LocalLLaMA

A real transformer language model running locally on a stock Game Boy Color

maddiedreese ran Andrej Karpathy’s TinyStories-260K on a stock Game Boy Color with INT8 weights, fixed-point math, an MBC5 ROM, bank-switched cartridge storage, and KV cache in cartridge SRAM; the demo uses no phone, PC, Wi‑Fi, link cable, or cloud inference, but output is extremely slow and gibberish.

Why it matters: HKR-H/K/R all pass: a named first-person experiment with concrete model and memory details. Impact stays low-featured because it is a Reddit hardware hack with slow, garbled output, not a usable product or model release.

Hacker News front page

Show HN: Needle Distills Gemini Tool Calling into a 26M Model

Cactus open-sourced Needle, a 26M-parameter tool-calling model that reaches 6,000 tok/s prefill and 1,200 tok/s decode on consumer devices, with MIT-licensed weights released on Hugging Face.

Why it matters: Cactus distilled Gemini's tool calling into a 26M-parameter model that runs locally on consumer devices at 6,000 tokens per second prefill and 1,200 per second generation.

r/LocalLLaMA

Needle: We Distilled Gemini Tool Calling Into a 26M Model

Cactus Compute open-sourced Needle, a 26M-parameter tool-calling model that reaches 6,000 tok/s prefill and 1,200 tok/s decode on consumer devices, using an attention-and-gating architecture with no MLPs.

Why it matters: Cactus Compute distilled Gemini's tool calling into a 26M-parameter model and reports 6,000 tokens per second prefill on consumer devices, a lightweight option for on-device tool use.