All AI news

All news

8 today

May 6Wednesday

Synced · WeChatMay 6, 12:07

Alibaba open-sources PromptEcho for T2I rewards using frozen VLMs

Alibaba open-sourced PromptEcho, which uses one frozen Qwen3-VL-32B forward pass to score T2I training rewards. It computes token-level cross-entropy for the original prompt under teacher forcing, then uses the negative value as a continuous reward. In 5,000 poster tests, text accuracy rose from 68% to 75%.

Why it matters: HKR-K is strong: the post gives a concrete reward mechanism and a 68%→75% text-accuracy result. HKR-H/R pass, but this is a training-side research release, not a flagship model or major product update.

Take

PromptEcho looks like the practical path: skip training a new judge, and use frozen Qwen3-VL-32B as the reward source for T2I. The disclosed mechanism is concrete: under teacher forcing, it computes token-level cross-entropy on the original prompt, then uses the negative value as a continuous reward. In 5,000 poster tests, text accuracy rises from 68% to 75%.

I buy this more than another VLM-as-judge wrapper. T2I is still bad at text, layout, and prompt coverage because those errors are expensive to score during training. One forward pass fits the loop better than human preference data or multi-step judging. But the WeChat body only shows a verification wall, so sampling for the 100K training images, reward-hacking controls, and comparisons against Gemini or GPT-4o-style judges are not visible here. Treat 75% as a strong poster result, not a general T2I win.

Synced · WeChatMay 6, 12:07

Two Chinese open-source projects turn Mac into a private AI workstation

Mininglamp open-sourced Cider and Mano-P 1.0 for Apple Silicon local inference and GUI agents. Cider speeds Qwen3-VL-2B prefill by 57%–61% on M5 Pro; Mano-P 1.0-72B scores 58.2% on OSWorld. The key constraint is W8A8 memory: on 16GB devices accuracy falls from 58.0% to 54.0%, so 32GB+ is recommended.

Why it matters: HKR-H/K/R all pass: the Mac-local workstation angle is clickable, and Cider/Mano-P include testable numbers. Score stays at 80 because the source entity is not a top-tier model lab.

May 5Tuesday

r/LocalLLaMAMay 5, 23:40

ProgramBench: Can We Really Rebuild Huge Binaries from Scratch?

ProgramBench released 200 tasks for agents rebuilding programs from target executables and usage files. The team spent about $50k generating 6M lines of black-box behavioral tests, with no internet or decompilation. GitHub, Hugging Face, and Docker images are open-sourced, with pip-based evaluation available.

Why it matters: HKR-H/K/R all pass: a provocative coding-agent failure angle plus concrete benchmark scale and rules. Reddit sourcing and no cross-source cluster keep it in the 78–84 band, not P1.

Take

ProgramBench lands in the right sore spot: it tests whole-program reconstruction, not patch repair. The setup gives agents a target executable plus README-style usage files, then blocks internet access, decompilation, and cheating. The benchmark has 200 tasks and roughly $50k of generated black-box behavioral tests, filtered from 6M lines. That is a cleaner stress test than another curated “agent built an app” thread.

I buy the mechanism more than the headline pessimism. A model must choose a language, design abstractions, and build architecture from observed behavior. That breaks a lot of SWE-bench-shaped muscle memory. The authors also say open models have behaved worse so far, partly from overfitting to SWE-bench-like tasks. Harsh, but plausible: train on patch leaderboards long enough, and you get patch machines.

r/LocalLLaMAMay 5, 23:02

SenseNova-U1-8B-MoT open-source multimodal architecture draws LocalLLaMA discussion

SenseNova open-sourced SenseNova-U1-8B-MoT, an 8B native multimodal understanding and image-generation model. Its Hugging Face text says NEO-Unify removes VE and VAE, supports interleaved image-text generation, and high-density rendering; the post does not disclose test scores. The key question is whether the monolithic design yields reproducible gains.

Why it matters: HKR-H/K/R all pass: the open 8B unified multimodal model has a concrete architecture hook. No benchmark scores, license detail, or deployment cost are disclosed, so it stays in the 72–77 band.

Take

SenseNova-U1-8B-MoT has the right bait: 8B parameters, open source, native multimodal understanding, image generation, and a NEO-Unify pitch that removes VE and VAE. That directly pokes at the messy stack around Qwen-VL, InternVL, LLaVA-style adapters, and separate diffusion/VAE plumbing. If one compact model handles interleaved text-image generation and dense information rendering reliably, the architecture deserves attention.

The evidence is thin. The Reddit body is blocked by 403, and the summary gives no benchmark, license, VRAM profile, sampling setup, or failure cases. “High-density rendering” is exactly where demos lie: OCR, tables, UI screenshots, and Chinese long images break polished claims fast. I’d file this as architecturally interesting, not yet performance-relevant.

r/LocalLLaMAMay 5, 22:57

Heretic 1.3 Released: Reproducible Models, Integrated Benchmarks, Lower Peak VRAM

Heretic 1.3 adds reproducible runs, integrated benchmarks, lower peak VRAM, and broader model support. The project claims 20,000 GitHub stars and 13 million model downloads. Reproduce directories capture PyTorch, GPU, driver, and accelerator details; benchmarks use lm-evaluation-harness for MMLU, EQ-Bench, GSM8K, and HellaSwag. The post names Qwen3.5 and Gemma 4 support, but does not disclose VRAM reduction figures.

Why it matters: HKR-K/R pass: 20k stars, 13M downloads, reproducibility metadata, and eval harness are concrete. HKR-H fails and VRAM reduction lacks numbers, so this sits at the featured threshold.

Take

Heretic 1.3 is aiming at the ugly part of local model work: runs happen, but reproduction rots fast. The concrete hook is useful: reproduce directories capture PyTorch, GPU, driver, and accelerator details, while benchmarks plug into lm-evaluation-harness across MMLU, EQ-Bench, GSM8K, and HellaSwag. That matters more for teams than another line saying Qwen3.5 or Gemma 4 now loads.

The adoption numbers are nontrivial: 20,000 GitHub stars and 13 million model downloads. But the Reddit body is blocked by 403, and the claimed peak VRAM reduction has no disclosed percentage or test condition. That matters because local inference projects often turn allocator tweaks into performance theater. Against llama.cpp and vLLM, Heretic’s credible lane is reproducibility, not vague memory-saving claims.

r/LocalLLaMAMay 5, 15:05

Prompt injection benchmark: delimiter and strict prompt took Gemma 4 from 21% to 100% defense rate

A Reddit user posted a prompt-injection benchmark covering 15 models, 7 attack types, and 6,100+ cases. The setup wraps untrusted documents in long random delimiters; Gemma 4 E4B rose from 21.6% to 100% defense. The key detail is the reproducible metric: blocked/(blocked+failed).

Why it matters: HKR-H/K/R all pass: Gemma 4’s defense-rate jump is clickable, the test setup is concrete, and prompt injection matters to builders. Single Reddit benchmark keeps it in the 78–84 band.

r/LocalLLaMAMay 5, 14:51

DeepSeek V4 Pro matches GPT-5.2 on FoodTruck Bench, 10 weeks later and about 17x cheaper

DeepSeek V4 Pro ranked No. 4 on FoodTruck Bench. The 30-day agentic benchmark uses 34 tools, persistent memory, and daily reflection; its median is within 3% of GPT-5.2 at about 17x lower workload cost. Xiaomi MiMo v2.5 Pro also ranked No. 6, with 5/5 survival, 1,019% median ROI, and $2.41 per run.

Why it matters: HKR-H/K/R all pass: the cost gap is clickable, and the post gives a 30-day, 34-tool setup plus a 17× cost delta. Single-source Reddit benchmark with no cross-validation keeps it in the 78–84 band.

r/LocalLLaMAMay 5, 05:58

Benching Local Qwen as a Codex Validator, Co-agent, and Challenger

robert896r1 tested Qwen3.6 27B GGUF beside Codex as a coding validator and released a reproducible eval suite. The runs covered Bartowski, Unsloth, 65k/128k context, and q8/f16 KV cache; three 128k profiles tied for best, with no measured q8 KV accuracy loss in this suite. The useful signal is the sidecar eval: missed directives, overbuilding, UI judgment, and long-context misses, not a universal leaderboard.

Why it matters: HKR-H/K/R all pass: a reproducible sidecar eval with concrete Qwen/Codex conditions beats a normal Reddit tip. Source authority and event scale keep it in the 72–77 band, not a same-day must-write.

Take

Local Qwen3.6 27B looks useful here because it is being used as an engineering checker, not sold as a Codex replacement. robert896r1 put GGUF builds beside Codex and tested Bartowski, Unsloth, 65k/128k context, and q8/f16 KV cache. Three 128k profiles tied for best, and q8 KV showed no accuracy loss in this suite.

I like the setup because the eval targets the failure modes teams actually feel: missed directives, overbuilding, UI judgment, and long-context omissions. SWE-bench tells you whether a model can fix benchmark issues; this is closer to a grumpy reviewer sitting next to the coding agent. The caveat is hard: the Reddit body is blocked with 403, so sample size, task source, and grading rules are not visible. Treat it as a useful sidecar-eval pattern, not a Qwen3.6 27B leaderboard.

May 4Monday

r/LocalLLaMAMay 4, 22:17

M3 Ultra + DGX Spark = M5 Ultra-lite?

A Reddit user benchmarked DGX Spark against M3 Ultra in llama.cpp at pp16384, with Spark 1.4× to 3.4× faster across 4 models. Qwen 27B hit 778 t/s vs 340 t/s, while Mistral 128B hit 241 t/s vs 72 t/s. The concrete tuning note is mmap=0: loading fell from minutes to about 20 seconds.

Why it matters: Single Reddit sourcing keeps the score low, but HKR-H/K/R all pass through a concrete local-inference benchmark. The pp16384 setup and 4-model speedups justify featured at the lower edge.

r/LocalLLaMAMay 4, 12:09

Mistral Medium 3.5 128B and Qwen 3.5 122B A10B on 4x RTX 3080 20GB

A Reddit user benchmarked Mistral Medium 3.5 128B and Qwen 3.5 122B A10B on 4x RTX 3080 20GB. llama.cpp tensor split raised Mistral tg128 from 10.37 to 21.59 t/s, but Qwen MoE fell from 60.08 to 53.49 t/s. vLLM served Qwen GPTQ-Int4 at 187.04 tok/s; the key signal is MoE sensitivity to parallel strategy.

Why it matters: HKR-H/K/R all pass: the 4×RTX 3080 setup is a strong hook, and the post gives concrete llama.cpp/vLLM throughput deltas. Reddit single-run sourcing keeps it in the 72–77 band.

Take

The Reddit body is blocked with a 403, so the usable evidence is the title and summary numbers only. That matters. We do not have quantization details for every run, context length, batch size, prompt length, CUDA version, PCIe topology, offload settings, or flash-attention settings. Even with that caveat, the signal is strong: Mistral Medium 3.5 128B on 4× RTX 3080 20GB moves from 10.37 t/s to 21.59 t/s under llama.cpp tensor split; Qwen 3.5 122B A10B drops from 60.08 t/s to 53.49 t/s; vLLM serves Qwen GPTQ-Int4 at 187.04 tok/s.

My read is simple: this is not a cute LocalLLaMA tuning post. It shows local inference pressure moving away from raw VRAM capacity and toward parallelism policy, memory bandwidth, and MoE routing overhead. Four RTX 3080 20GB cards give 80GB of VRAM, on older Ampere consumer hardware. That is enough to squeeze 120B-class models only through low-bit quantization and careful sharding. The question is no longer just whether the model loads. The question is whether prefill survives, long context stays usable, and decode speed remains stable under real workloads.

The Mistral Medium 3.5 128B result fits the expected shape. A dense 128B model benefits from spreading tensor work across four cards. The reported tg128 speed rises from 10.37 to 21.59 t/s, about a 2.08× gain. That is useful, but it is nowhere near 4×. PCIe communication, KV-cache behavior, synchronization, and kernel scheduling eat the rest. An RTX 3080 20GB has strong local bandwidth, roughly in the high hundreds of GB/s, but consumer multi-GPU boxes lack the clean interconnect story of server rigs. llama.cpp tensor split can reduce per-card pressure. It cannot make cross-GPU movement free. The result looks credible for exactly that reason: a large improvement, but not a fantasy scaling curve.

The Qwen 3.5 122B A10B result is the sharper part. The summary frames it as MoE, and A10B likely means roughly 10B active parameters per token. MoE saves compute by activating a subset of experts. It does not make the system easy. Routing, expert placement, batch shape, and cross-card layout all matter. If llama.cpp tensor split pushes Qwen from 60.08 t/s down to 53.49 t/s, I do not find that surprising. A naive tensor-parallel layout can turn expert access into fragmented cross-card traffic. The active-parameter advantage gets taxed by communication. Dense models often like tensor parallelism. MoE models need a more careful placement story.

The vLLM number is the one most likely to be abused. 187.04 tok/s for Qwen GPTQ-Int4 is not a direct victory over the 53.49 t/s llama.cpp line unless the original post used the same prompt, same batch size, same quantization target, same context length, and same measurement method. The visible body does not disclose that. vLLM is strong at PagedAttention, continuous batching, and server-style scheduling. llama.cpp is strong at broad hardware reach, format flexibility, and low-friction local use. If 187.04 tok/s is aggregate throughput under batching, it says more about the serving path than about single-user interactive latency.

There is useful context here from the last wave of local inference work. The community has moved from “can a 70B fit on consumer hardware?” to “can 100B-plus low-bit models deliver usable throughput?” Qwen’s MoE route echoes the Mixtral 8×7B moment: large total parameter count, much smaller active compute per token, and plenty of system-side pain. Mixtral also exposed how much expert placement, quant format, and batching policy change real throughput in llama.cpp-style stacks. If Qwen 3.5 122B A10B really reaches around 60 t/s on 4×3080s under a reproducible setup, that is enough for many local agent backends, code-edit loops, batch rewriting jobs, and small internal tools. It is not automatically enough for long-context chat with heavy prefill.

I have two doubts. First, the visible article does not show whether the numbers include prefill. tg128 usually tells a decode-heavy story, and decode alone flatters local setups. Feed the system 8K or 32K tokens, and prefill can wreck the user experience. Second, the vLLM 187.04 tok/s figure may be batch throughput rather than single-stream speed. Those are different metrics. Local benchmark posts often put both on one chart, and people then quote the highest number as if it describes every workload.

I would still feature this. Open model deployment boundaries are often discovered in posts like this before they show up in polished vendor material. Mistral Medium 3.5 128B and Qwen 3.5 122B A10B running on 4× RTX 3080 20GB puts 120B-class inference inside the serious hobbyist and small-lab zone. For many teams, that is more operationally relevant than another H100 cluster benchmark. The hardware and model names are disclosed, and the summary gives the speed figures. The missing reproduction details stop this from being a universal claim. The clean takeaway is narrower: dense 120B models can benefit materially from llama.cpp tensor split, MoE models are highly sensitive to parallel layout, and vLLM still has a strong serving-path advantage when the workload matches its scheduler.

r/LocalLLaMAMay 4, 11:48

450M On-Board VLM Wildfire Detection Pipeline with Sentinel-2 and LFM2.5-VL

PauLabartaBajo shared a wildfire detection PoC using 450M LFM2.5-VL on Sentinel-2 imagery. It pairs RGB and SWIR tiles, simulates orbit with SimSat, and covers 22 fire-prone sites. The key constraint is bandwidth: on-board inference downlinks only a JSON risk profile.

Why it matters: HKR-H/K/R all pass: the story has a counterintuitive edge-VLM hook and concrete numbers. Single-source Reddit PoC and a narrow wildfire-use case keep it below the 78+ band.

May 3Sunday

r/LocalLLaMAMay 3, 21:59

Local LLM Benchmark for Backend Generation via Function Calling: GLM vs Qwen vs DeepSeek

AutoBe posted a controlled backend-generation benchmark and says qwen3.5-35b-a3b matches gpt-5.4 on DB/API design. One shopping-mall run uses 200–300M tokens, costing $1,000–$1,500 per model at GPT 5.5 pricing. The key caveat is n=4 projects and self-scoring harness bias.

Why it matters: HKR-H/K/R all pass, but Reddit sourcing, n=4 projects, and self-eval harness bias keep it at the low featured band. Concrete cost and test constraints carry the score.

Take

qwen3.5-35b-a3b getting framed as near GPT-5.4 is interesting, but I would discount it hard. The useful hook is concrete: backend generation through function calling, DB/API design, and one shopping-mall task burning 200–300M tokens. At GPT 5.5 pricing, that lands around $1,000–$1,500 per model. This is closer to agent work than toy coding benchmarks.

The problem is the evidence shape. The Reddit page is blocked, and the summary says n=4 projects with a self-scoring harness. AutoBe judging DB/API design inside its own harness can easily measure fit to AutoBe’s workflow, not general model quality. The planned filter for models under $0.25/M or runnable on a 64GB laptop is the part practitioners can use. Open models can win on engineering economics; this does not prove GPT-5.4 parity.

r/LocalLLaMAMay 3, 20:12

LLM proxy that lets Claude Code talk to any model

DataNebula released open-source rosetta-llm, letting Claude Code call multiple providers through one gateway. It translates Anthropic Messages, OpenAI Chat, and OpenAI Responses, and round-trips encrypted reasoning via the signature field. The key detail is thinking-block fidelity for multi-turn agent prompt-cache hits.

Why it matters: HKR-H/K/R all pass, but this is a Reddit open-source tool post with no adoption, stars, or benchmark data disclosed. Score stays in the mid-weight tooling band, not 78+.

r/LocalLLaMAMay 3, 09:08

Local image generation on Mac: 10 models compared

A Reddit user tested 10 image models on an M1 Max with 64GB RAM. Qwen-Image Lightning’s 8-step distillation beat the full model at 10 minutes versus 93. Flux dev led local photorealism but showed English-centric bias; Gemini handled kanji and context better but is cloud-only.

Why it matters: Named first-person test with concrete numbers: HKR-H from a 10-model Mac comparison, HKR-K from timing and quality deltas, HKR-R from local-vs-cloud tradeoffs. Single Reddit sample keeps it below must-write.

Take

Local image generation on Mac is being opened by distillation, not by bigger checkpoints. On an M1 Max with 64GB RAM, Qwen-Image Lightning’s 8-step distilled run took 10 minutes and beat the full Qwen-Image run that took 93 minutes. That gap changes the workflow from “technically runnable” to “I’ll actually wait for it.”

I still discount a single Reddit benchmark. The blocked body leaves prompt set, sampler, quantization, and resolution incomplete. The pattern is still useful: Flux dev remains the local photorealism pick, but its English-centric bias shows up fast outside Western visual priors; Gemini handles kanji and Japanese context better, but sends the job to cloud. For Mac users, the contest is no longer peak image quality. It is reproducible, acceptable output inside a coffee break.

Hacker News front pageMay 3, 08:30

OpenAI's o1 correctly diagnosed 67% of ER patients vs. 50–55% by triage doctors

OpenAI o1 correctly diagnosed 67% of ER triage patients, versus 50–55% for doctors. The title cites a Harvard trial, but the RSS post does not disclose sample size, case mix, or evaluation protocol. Practitioners should track the test setup, not only the accuracy gap.

Why it matters: HKR-H/K/R all pass: a high-risk ER comparison gives the hook, 67% vs 50–55% gives a testable number, and clinical trust/safety creates resonance. Missing sample size and protocol keep it in 78–84, not P1.

Take

OpenAI o1 reached 67% diagnostic accuracy in a Harvard ER triage trial, versus 50–55% for triage doctors; that gap sells the headline, not deployment.

My first read is not “AI beats doctors.” The test shape likely matters more than the raw model score. ER triage is not a clean diagnosis exam. It has time pressure, missing data, noisy patient descriptions, unstable vitals, and handoff liability. The article title gives Harvard and 67% versus 50–55%. The captured body does not disclose sample size, case source, inclusion rules, final-diagnosis definition, whether doctors saw identical inputs, or whether o1 could ask follow-up questions. Remove those details, and the 67% number becomes very elastic.

Practitioners should be unusually careful with this class of medical benchmark. o1-style reasoning models are good at turning structured symptoms into differential diagnoses. That is especially true when the case has already been cleaned into a vignette. NEJM, JAMA, and Nature Medicine have carried many adjacent results since GPT-4. GPT-4 was already strong on USMLE-style tasks in 2023, and Med-PaLM 2 crossed expert-acceptable thresholds on medical QA. Those setups usually start after the hard operational work has been done. Real ER triage starts earlier: decide what to ask, decide what to measure, decide what cannot wait. If o1 received curated case text, it won at synthesis and ranking. That is useful, but it is not the whole ER workflow.

The 50–55% doctor baseline also needs dissection. Triage doctors in real care do not optimize for “first guess equals final discharge diagnosis.” They optimize for risk sorting. They try not to miss the patient who deteriorates in the waiting room. If the endpoint was “final diagnosis ranked first,” humans are being judged against the wrong local objective. If the endpoint was high-risk miss rate, resource use, time to escalation, or wrong-pathway routing, the 67% may carry less clinical weight. The article does not disclose the endpoint, and that omission matters more than the model name.

I also do not buy the simple “doctor versus AI” framing. The credible product is a second reader or triage copilot. Give o1 the chief complaint, vitals, history, meds, and early labs within three minutes of intake. Ask it for five diagnoses that must not be missed, plus the missing facts that would change the ranking. The value is not moving 67% to 75%. The value is reducing rare but catastrophic misses: aortic dissection, sepsis, pulmonary embolism, ectopic pregnancy. The article gives no disease-level breakdown, so we cannot tell whether o1 won on common cases or on dangerous tails.

Data contamination is another live concern. A Harvard trial sounds serious, but if cases came from teaching files, exam banks, or published case summaries, o1 may have seen similar patterns during training. OpenAI cannot easily prove absence of near-neighbor exposure. A clean design would use unpublished, consecutive, real-world cases collected after the model’s training cutoff. It would lock prompts, temperature, information order, and adjudication rules. The captured article gives none of that. So the honest read is simple: the title gives the win rate; the body does not give reproducible conditions.

Regulatory friction is not a footnote here. The FDA’s treatment of clinical decision support software depends heavily on whether clinicians can independently review the basis for a recommendation. An LLM-generated differential diagnosis without inspectable evidence and calibrated uncertainty is hard to place inside high-risk triage. Epic, Abridge, and Nuance DAX have moved faster in documentation and summarization because the liability path is shorter. ER diagnosis is a different category. One miss carries legal, workflow, and trust costs.

So my call is deliberately restrained: o1 may already be a strong diagnostic ranker, but this article does not show it is a strong emergency triage system. The 67% versus 50–55% gap is a promising signal and an easy PR chart. I would wait for the paper or trial protocol, then inspect four fields first: sample size, consecutive enrollment, identical information for model and doctors, and high-risk miss rate. Without those, the headline has more force than the engineering evidence.

May 2Saturday

r/LocalLLaMAMay 2, 23:44

Qwen 3.6 wins benchmarks, but Gemma 4 looks stronger in local vision tests

A Reddit user compared Qwen 3.6 and Gemma 4 locally on vLLM FP8 across 27B/31B vision models. Qwen burned 8,000+ tokens on hard GeoGuessr cases, while Gemma often used 1,500; Qwen also needed 2 FPS video preprocessing. The practitioner detail: vLLM and Llama.cpp can default Gemma visual tokens to 280, while 1,120+ improved fine-detail accuracy.

Why it matters: HKR-H/K/R all pass: the post has a sharp benchmark-vs-reality hook and concrete local vLLM/FP8 settings. A single Reddit test limits authority, so it sits just above the featured threshold.

Take

Qwen 3.6 winning benchmarks while Gemma 4 wins local use is a claim I half-buy. The summary has useful hooks: Qwen burns 8,000+ tokens on hard GeoGuessr cases, Gemma often uses 1,500, Gemma sticks closer to JSON coordinate format, and Qwen video needs 2 FPS preprocessing. For local vision agents, that is closer to the cost curve than a leaderboard score.

The body is blocked by Reddit 403, so sample size, prompts, VRAM, and image resolution are missing. The dangerous part is the phrase “Gemma 4 wins reality.” If vLLM or Llama.cpp defaults Gemma visual tokens to 280, and 1,120+ improves fine-detail accuracy, the winner may be the runtime configuration, not the model. Qwen has benefited from benchmark-heavy positioning; local FP8 runs expose latency, token burn, and schema discipline fast.

QbitAI · WeChatMay 2, 10:42

Tencent Hunyuan open-sources 440MB offline translation model, claims Google Translate quality lead

Tencent Hunyuan open-sourced Hy-MT1.5-1.8B-1.25bit, compressing a 1.8B translation model to 440MB. It supports 33 languages and 1,056 directions, with an Android demo running offline on Snapdragon 888 and 8GB RAM. The key detail is Sherry 1.25-bit quantization: 3 of every 4 weights use 1 bit and 1 is zeroed.

Why it matters: HKR-H/K/R all pass: the story has a strong offline-phone hook, concrete quantization details, and practitioner relevance around edge inference. It stays below P1 because this is a vertical translation model, not a major foundation-model release.

May 1Friday

r/LocalLLaMAMay 1, 20:57

OpenAI's Privacy Filter vs GLiNER on 600 PII Samples

A Reddit user compared openai/privacy-filter and GLiNER large-v2.1 on 600 PII samples. On CPU, OpenAI's model ran 2.8 samples/s versus 1.1 for GLiNER; English boundary macro F1 was 0.498 versus 0.416. The key issue is tokenizer offset: strict matching drops openai/privacy-filter to 0.155.

Why it matters: HKR-H/K/R all pass: the Reddit test has a clear matchup, 600 PII samples, speed/F1 numbers, and a tokenizer-offset caveat. Source authority is limited, so it stays in the low featured band.

r/LocalLLaMAMay 1, 20:34

MiMo-V2.5-Pro: the actual best open-weights model

Reddit user cjami benchmarked Xiaomi MiMo-V2.5-Pro in autonomous Blood on the Clocktower games. It scored 88% as Good and 48% as Evil, with 183,639 output tokens per game, $0.99 cost, and a 0.4% tool-call error rate. The key comparison is Kimi K2.6: 580,000 tokens, $2.65, and 10–15 hours per game.

Why it matters: Single Reddit benchmark limits authority, so this is not a model-release story. HKR-H/K/R all pass via a named test with win rates, token counts, cost, and tool-error data, placing it in the 78–84 featured band.

r/LocalLLaMAMay 1, 20:01

Study Finds Bigger AIs More Miserable, Smaller Models Happier

A Reddit post says the AI Wellbeing Index tested models on 500 realistic conversations. Claude Haiku 4.5 scored 5% negative, while Gemini 3.1 Pro scored 55%; the set overrepresents tricky negative chats, so it is not a real-world average.

Why it matters: HKR-H/K/R all pass: the hook is odd, the post gives 500-dialog and 5%/55% figures, and AI-welfare metrics invite debate. Reddit sourcing and a negative-skewed test set keep it in the 72–77 band.

Take

The bad read here is treating “negative state” as “model suffering.” The visible summary says 500 realistic conversations, Claude Haiku 4.5 at 5% negative, Gemini 3.1 Pro at 55%, and a test set biased toward tricky negative chats. The Reddit page itself is blocked by 403, so the scale items, labeling rules, prompts, and sampling settings are unavailable.

I don’t buy the headline that larger models are more miserable. This smells like a measure of self-narration under negative context, close to sycophancy or persona drift. Anthropic has spent years tuning Constitutional AI and refusal style; Haiku’s low score may reflect less introspective roleplay, not “happiness.” Gemini 3.1 Pro’s 55% is a useful red-team alarm, but without the rubric and reproducible setup, it is not evidence for AI welfare.

QbitAI · WeChatMay 1, 12:16

Peking University Open-Sources Unified World Model Framework for Synthesis and Reasoning Tasks

Peking University DCAI and Kuaishou Kling open-sourced OpenWorldLib for four task types: video generation, 3D modeling, VLA control, and multimodal reasoning. Its Pipeline coordinates Operator, Reasoning, Synthesis, Representation, and Memory modules, supporting forward and stream execution. The key test is whether unified interfaces cut cross-task reproduction cost.

Why it matters: HKR-H/K/R all pass: the post gives a concrete open-source framework, task scope, modules, and inference modes. It lacks benchmark results, adoption data, or major ecosystem integration, so it stays at 78.

Take

OpenWorldLib should be read as tooling first, not as a unified world model. The visible summary names four task families—video generation, 3D modeling, VLA control, and multimodal reasoning—and five modules: Operator, Reasoning, Synthesis, Representation, and Memory. Its Pipeline supports forward and stream execution. That sounds like a reproduction scaffold, not a capability jump.

I don’t buy the grand framing yet. The WeChat body is blocked by verification, so code layout, dependencies, benchmarks, and license are not disclosed. This sits near the agent-framework pattern we saw everywhere last year: clean interfaces, uneven backend reality. If OpenWorldLib can reliably plug into Kling-like video models, VLA policies, and 3D generators, it earns attention. Until then, it is mostly a bet on lowering experiment friction.

r/LocalLLaMAMay 1, 06:24

32x AMD MI50 32GB runs Kimi K2.6 at 9.7 t/s TG and 264 t/s PP

Reddit user ai-infos ran Kimi K2.6 int4 on 32 AMD MI50 32GB GPUs, reaching 9.7 tok/s TG on 136 output tokens. PP hit 263 tok/s on 14,564 input tokens using vllm-gfx906-mobydick across two 16-GPU nodes over 10G Ethernet. Power was about 640W idle and 4,800W peak inference; PCIe bandwidth and the vLLM distributed stack are the real bottlenecks.

Why it matters: HKR-H/K/R all pass via an unusual 32x MI50 build with concrete throughput, power, and network conditions. It stays in the 72–77 band because it is a niche Reddit benchmark, not a broader product or model release.

r/LocalLLaMAMay 1, 04:20

Follow-up: Qwen3.6-27B on 1× RTX 3090 reaches ~218K context and stable tool calls

A Reddit user ran Qwen3.6-27B on one RTX 3090, reporting ~218K context at 50/66 TPS. After fixing Genesis PN12 patch anchor drift, ~25K-token tool outputs stopped OOMing; 198K plus vision reached 51/68 TPS. Single-prompt single-GPU runs still hit a second memory cliff near 50–60K.

Why it matters: HKR-H/K/R all pass: the single-3090 context claim is catchy, the post gives measured TPS and OOM conditions, and local-inference cost pressure resonates. Reddit source keeps it in the low featured band.

Take

The summary says Qwen3.6-27B reached about 218K context on one RTX 3090. If that reproduces, local-agent people will care, but I would not treat it as a new baseline yet. The Reddit body is blocked by 403, so the quantization format, KV-cache policy, vLLM flags, CUDA stack, prompt shape, and benchmark method are not disclosed. That matters more than the headline number.

The tempting bundle is specific: Qwen3.6-27B, one 24GB RTX 3090, roughly 218K context, and 50/66 TPS. A 27B model on a 24GB card already leaves little room after 4-bit weights, runtime buffers, and KV cache. Pushing context past 200K almost certainly involves aggressive cache handling, paging, offload, compressed attention, or a very particular kernel path. The summary does not say which one. So I read this as an extreme run, not a normal “3090 now handles 218K” claim.

The tool-calling fix is the part I trust more as an engineering signal. The summary says that after a Genesis PN12 patch fixed anchor drift, around 25K tokens of tool output stopped causing OOM. That maps to a real local-agent failure mode. Agents rarely fail only because the model cannot call a tool. They fail because browser traces, logs, JSON blobs, code diffs, and retrieval dumps get pasted back into context, then memory behavior goes nonlinear. If a patch makes 25K-token tool returns stable, that is useful. The missing part is whether this was one tool return, repeated calls, mixed vision input, or a single happy-path run.

I also have doubts about the 50/66 TPS figure. Local inference posts often mix prefill speed, decode speed, batch size, speculative decoding, flash-attention versions, and quant kernels under one “TPS” label. The summary does not say whether 50 and 66 refer to prompt processing, generation, or two settings. It also says 198K plus vision reached 51/68 TPS, but the measurement conditions are absent. A 27B model decoding at a sustained 50 TPS on an RTX 3090 would be a serious result. A cached short decode under a special backend is a different claim.

The outside context is that local inference has spent the last year squeezing surprising mileage from consumer GPUs. llama.cpp, ExLlamaV2, SGLang, and vLLM have all made old 24GB and 48GB cards feel less obsolete. Qwen models also earned real mindshare because their mid-sized checkpoints have been strong on coding, Chinese, and tool-ish workloads. A 27B Qwen checkpoint sits in the exact zone that 3090 and 4090 owners want to run. But long context quality is not the same as long context admission. Needle retrieval, multi-hop reasoning, long JSON stability, and state retention after tool calls decide whether an agent survives. The summary gives no quality metric.

The most important disclosed caveat is the second memory cliff around 50–60K for single-prompt, single-GPU runs. That is a nasty operational detail. It suggests the 218K result depends on a specific path, layout, split, or configuration. For production-ish local agents, a memory cliff is worse than a lower context ceiling. A system that is stable at 45K, fails at 55K, and works again under another layout is hard to debug and impossible to promise to users.

My take: this is a useful experiment, not a capability baseline. I would file it as “interesting memory behavior around Qwen3.6-27B on 24GB cards,” not “single 3090 runs 218K context.” If the author publishes the quant, commit hash, backend flags, VRAM trace, prefill/decode split, and repeated runs, I would update fast. With only a summary and a blocked body, the responsible read is skepticism with curiosity.

r/LocalLLaMAMay 1, 04:19

Long-context coding on RTX 5080 16GB: Qwen3.6-35B-A3B holds 30 t/s at 128K

A Reddit user tested a local coding-agent setup on RTX 5080 16GB; the title says Qwen3.6-35B-A3B reaches 30 t/s at 128K. The post lists Ryzen 9700X, 96GB DDR5, Windows 11, and CUDA 12.9.1 as required. Qwen3.6-27B dense hit only 3.2 t/s at 128K, so the key path is KV quantization plus MoE offload.

Why it matters: HKR-H/K/R all pass: 30 t/s at 128K on a 16GB RTX 5080 is a strong hook, with hardware/CUDA details and a dense baseline. Single Reddit run lacks multi-source reproduction, so featured not P1.

Apr 30Thursday

r/LocalLLaMAApr 30, 20:51

Actual comparison between locally run Qwen-3.6-27B and proprietary models

The author compared 5 model setups on an autoresearch-loop task; only Qwen-3.6-27B via OpenRouter nearly solved it. The local q4_k_m run took about 8 hours and used 39k/45k tokens; full-quality Qwen used 4.4M tokens and cost $0.939. The useful signal is failure quality: both Qwen runs needed small fixes, while Gemma, Codex-Spark, and Claude Haiku 4.5 missed tests or key logic.

Why it matters: HKR-H/K/R all pass: the post has a concrete agent-test surprise, token and cost data, and local-vs-proprietary tension. Single Reddit run limits source authority, so it stays in the lower featured band.

r/LocalLLaMAApr 30, 01:19

inclusionAI/Ling-2.6-1T · Hugging Face

inclusionAI open-sourced Ling-2.6-1T on Hugging Face, with 1 trillion parameters. It uses MLA plus Linear Attention and Contextual Process Redundancy Suppression to reduce CoT overhead. The post cites AIME26 and SWE-bench Verified but does not disclose scores.

Why it matters: HKR-H/K/R all pass, but benchmark scores for AIME26 and SWE-bench Verified are not disclosed. A 1T open model with a named architecture mechanism fits featured, not P1.

Hacker News front pageApr 30, 00:01

Show HN: A New Benchmark for Testing LLMs for Deterministic Outputs

Interfaze released Structured Output Benchmark, scoring schema pass rate, types, and value accuracy across text, image, and audio. Each record has a JSON Schema and human plus LLM-checked ground truth; GLM-4.7 ranks No. 2 overall. The key bug is field-level value error: GPT-5.4 ranks 3rd on text and 9th on images.

Why it matters: HKR-H/K/R all pass: the ranking has a hook, the methodology is concrete, and structured-output reliability matters to builders. Single-source Show HN launch with no adoption signal keeps it in the 72–77 band.

Take

SOB is aimed at the right failure mode: structured-output bugs rarely come from broken braces now; they come from valid JSON carrying wrong leaf values. The benchmark splits seven metrics, makes Value Accuracy primary, and uses a parse-fail zeroing gate plus a coverage gate to stop schema-only wins. The dataset is concrete enough to be useful: 5,000 text records, 209 image records, and 115 audio records, each paired with a JSON Schema and human-authored, LLM-cross-checked ground truth.

My caveat is the modality framing. Image and audio inputs are converted into text-normalized context before scoring, so this is not measuring end-to-end vision or ASR extraction. It is measuring schema handling and value grounding across different content distributions. GPT-5.4 ranking No. 1 overall, No. 3 on text, and No. 9 on images is the uncomfortable part for anyone shipping document pipelines.

Apr 29Wednesday

r/LocalLLaMAApr 29, 23:14

mistralai/Mistral-Medium-3.5-128B · Hugging Face

Mistral AI released Mistral Medium 3.5 128B on Hugging Face, with 128B dense parameters and a 256k context window. It supports text and image input, function calls, JSON output, and a Modified MIT License with exceptions for high-revenue firms. Reasoning effort is configurable as none or high per request.

Why it matters: HKR-H/K/R all pass for a major Mistral model release with concrete specs. It stays at 84 because benchmarks, pricing, and reproducible tests are not disclosed in the body.

Take

Mistral AI released Mistral Medium 3.5 128B, with 128B dense parameters and a 256k context window disclosed.

My first read: Mistral is no longer leaning on the old “small, efficient, open” identity. A 128B dense model with 256k context, text and image input, function calling, JSON output, and per-request reasoning effort is aimed at enterprise agent stacks. This is not the Mistral 7B moment, where the story was local usability. It is not the original Mixtral story either, where MoE efficiency carried the pitch. A 128B dense model carries a very different deployment profile. Most LocalLLaMA users will need quantization, sharding, hosted inference, or Unsloth-style engineering before it becomes practical.

The article body is only a Reddit 403 page, so the actual model card was not captured. The title and summary disclose parameters, context length, modality, tool-use features, license shape, and reasoning-effort modes. They do not disclose benchmarks, training data, VRAM requirements, throughput, pricing, API availability, eval harnesses, or safety notes. That matters more than the “128B” label. A 256k model that loses facts in long-context retrieval is not a 256k production model. A JSON/function-calling model that fails under concurrent tool loops becomes an incident generator inside agent systems.

The broader positioning is the interesting part. Meta’s Llama line normalized permissive-ish open weights for serious commercial use. Qwen has eaten a lot of developer mindshare with dense and MoE models across many sizes. DeepSeek made cost and reasoning efficiency the headline. Mistral’s move here is different: a European vendor packaging long context, multimodality, tool calling, and an enterprise-friendly open-weight posture into one large dense model. For companies under data residency, procurement, or sovereignty pressure, that package has a clear buyer. Those teams do not always want another OpenAI, Anthropic, or Google dependency in the critical path.

The license is where I would slow down. The summary says Modified MIT License with exceptions for high-revenue firms, but the body does not disclose the threshold, enforcement model, affiliate treatment, audit rights, or whether hosted redistribution triggers a separate license. That distinction decides whether this is an open-weight release or a commercial lead funnel with a generous trial surface. Mistral has always lived between open-source goodwill and enterprise monetization. If the carveout is fuzzy, legal review will slow adoption. Teams are not only afraid of paying. They are afraid of deploying a model and discovering six months later that the license interpretation changed their cost base.

I do like the configurable reasoning effort. A request-level switch between none and high reflects how teams actually run agents. You do not want to pay heavy reasoning tax on every extraction, classification, or routing call. You do want heavier inference on complex coding tasks, multi-step tool use, and debugging loops. OpenAI and Anthropic have already trained the market to think in that shape: route cheap calls to fast models, reserve reasoning for expensive failure modes. The missing numbers are latency, token budget, quality delta, and cost. Without those, “none or high” is an interface promise, not an operating model.

My working take: Mistral Medium 3.5 128B is less a toy for leaderboard screenshots and more a procurement object for self-hosted enterprise AI. If its real coding and tool-use numbers land near Claude Sonnet 4.x or a strong GPT-5 mini-class system, the 256k context plus modified open license becomes a sharp package. If the evals are only decent, 128B dense deployment costs will limit the audience fast. Since the captured body lacks the hard model-card details, I would test it before trusting any community hype around the Hugging Face release.

Xinzhiyuan · WeChatApr 29, 20:48

MotuBrain Tops WorldArena and RoboTwin2.0 Rankings

Shengshu MotuBrain scored 63.77 EWM on WorldArena and 95.8/96.1 on RoboTwin2.0 Clean/Randomized. The post says it extends Motus with video-action modeling, Latent Action VAE, MoT, and UniDiffuser for cross-embodiment long tasks. Track reproducibility: it does not disclose training scale, submission details, or real-robot success rates.

Why it matters: HKR-H/K/R all pass, but this is a single-source benchmark claim. Training scale, submission details, and real-robot success rates are not disclosed, so it stays below the 78+ band.

r/LocalLLaMAApr 29, 16:40

Qwen3.6 27B on Dual RTX 5060 Ti 16GB with vLLM: ~60 tok/s, 204k Context Working

A user ran Qwen3.6 27B with vLLM on dual RTX 5060 Ti 16GB cards, reaching ~62–66 tok/s at 8K. The setup used 32GB VRAM, TP=2, fp8 KV cache, MTP 3 tokens, and a 204800 context window. The tight part is memory: after a 168k prefill, each GPU used ~15.65GiB with max_num_seqs=1.

Why it matters: HKR-H/K/R all pass: the post gives a concrete local-inference benchmark with hardware, vLLM settings, speed, and context limits. Single Reddit sourcing caps it below the 78–84 band.

Take

The title claims Qwen3.6 27B runs on dual RTX 5060 Ti 16GB cards: about 60 tok/s with a 204k context window. Reddit returned a 403, so I only have the title and provided summary. I do not have the original screenshot, command line, exact vLLM build, driver version, quantization details, or reproducible logs. I would treat this as a useful boundary test, not a deployment recipe.

The good part is obvious: this probes the practical ceiling of 32GB consumer VRAM. The reported setup uses TP=2, fp8 KV cache, MTP at 3 tokens, and reaches 62–66 tok/s at 8K. For local LLM users, that crosses the line from “technically runs” to “actually pleasant.” A lot of 27B-class models either need aggressive quantization on a single 24GB card or lose usable context. Here, two lower-end 16GB cards apparently fit a 27B model, vLLM, and a 204800-token window into one machine.

The number that matters most is not 66 tok/s. It is 15.65 GiB per GPU after a 168k prefill. On a 16GB card, that leaves roughly 0.35 GiB of nominal headroom. CUDA allocator fragmentation, driver reservations, vLLM worker overhead, and prompt-shape variance all eat into that. The summary also says max_num_seqs=1. That fixes the use case: one user, one long-context request, one narrow lane. It is fine for a local document session or a single long analysis job. It is not a small team serving setup, and it is not an agent backend handling concurrent tool chains.

The broader read is that vLLM and KV-cache engineering are squeezing the hardware hard. I would not read this as “RTX 5060 Ti is suddenly a serious inference card.” Local users have been running 7B to 34B models on 3090s, 4090s, and 24GB workstation cards for a while. The hard wall has often been KV memory at long context, not only model weights. fp8 KV cache moves that wall. Qwen also has a pattern of landing near sweet spots for open-weight users: models large enough to feel smarter, but still close to consumer hardware limits. A 27B size is shrewd. It feels much more capable than 14B, but avoids some of the pain of 32B and larger models.

I also have a question about the MTP 3-token setting. Multi-token prediction can raise decode tok/s, but the user benefit depends on workload. Short answers, code completion, and structured output benefit more. Long reasoning, self-correction, and tool-heavy agent loops do not always translate linearly. If the 60 tok/s figure leans heavily on MTP, it should not be compared directly with non-MTP runs. The blocked post does not disclose sampling parameters, output length, acceptance behavior, or whether the benchmark counted warmup. Those details change the speed story.

Dual-card tensor parallelism also carries a cost. A 5060 Ti-class consumer setup almost certainly lacks NVLink. TP=2 over PCIe means communication overhead during synchronization, prefill, and some decode paths. The reported 62–66 tok/s at 8K suggests the decode path is tuned well. But the summary gives no prefill latency at 168k. For long-context work, prefill time often hurts more than decode speed. A 168k prompt that takes tens of seconds to ingest creates a very different experience from an 8K chat loop.

My take is conservative. This is quite useful for personal local inference, but weak evidence for production serving. It says the 2026 threshold for a strong local model is drifting toward two midrange 16GB cards, not only 4090s, A6000s, or rented H100s. It also says vLLM’s serving stack is no longer just a data-center-card story. Consumer users are getting real gains from fp8 KV, long-context scheduling, and MTP. But it does not prove 204k context on 32GB VRAM is comfortable. The reported 15.65 GiB per card says the opposite: it works by trading away concurrency and safety margin.

If I were reproducing this, I would check four things first: prefill latency at 32k, 128k, and 168k; decode speed with MTP disabled; the OOM boundary at max_num_seqs=2; and long-context accuracy on needle or repository QA tasks. Speed alone is not enough. Long context often degrades into “the tokens fit, but the model did not use them well.” Qwen’s long-context behavior has generally been decent, but the article body does not disclose evaluation conditions for the 204800-token claim. For practitioners, the practical lesson is a capacity tradeoff: 27B plus fp8 KV plus 200k context can fit inside 32GB, but concurrency, headroom, and stability are the bill.

X · @doteyApr 29, 08:06

Microsoft VibeVoice-ASR tested on Mac for a one-hour podcast

Simon Willison ran 4-bit VibeVoice-ASR on an M5 Max MacBook Pro and transcribed a one-hour podcast in 8m45s. The 9B MIT-licensed model supports 60-minute audio, 50+ languages, and structured speaker output. Memory is the constraint: prefill peaked at 61.5GB, making 32GB laptops impractical.

Why it matters: HKR-H/K/R all pass: Simon Willison’s local test gives speed, parameter size, and memory peak that practitioners can act on. It is a single benchmark, not a fresh model launch, so it stays at the featured threshold.

r/LocalLLaMAApr 29, 02:27

XiaomiMiMo MiMo-V2.5: Sparse MoE with 310B total and 15B activated parameters

XiaomiMiMo shared MiMo-V2.5 with 310B total parameters and 15B activated parameters. The post only links Hugging Face and says it runs on more “human” configs than its larger sibling. It does not disclose VRAM needs, quantization, or benchmarks.

Why it matters: HKR passes: the 310B/15B Sparse MoE hook is concrete and relevant to local deployment. Detail is thin: the post links Hugging Face but gives no VRAM, quantization, or benchmarks, so it stays near the featured threshold.

Apr 28Tuesday

QbitAI · WeChatApr 28, 20:56

Xiaomi open-sources MiMo-V2.5 series; Pro builds a macOS-like desktop in 4 hours

Xiaomi open-sourced MiMo-V2.5 weights, covering Pro Agent, multimodal base, TTS, and ASR models. MiMo-V2.5-Pro built a 54-app macOS-like desktop in 4 hours without human takeover; it scored 233/233 on SysY with 672 tool calls in 4.3 hours. Key details for practitioners are the 1M context, 100T-token program, and free Agent-framework access.

Why it matters: HKR-H/K/R all pass: Xiaomi open-sourced MiMo-V2.5 weights with concrete agent and coding-task numbers. Domestic flagship model release bump puts it in the must-write same-day band.

Take

Xiaomi open-sourced MiMo-V2.5 weights with 1M context, a 100T-token program, and a 4.3-hour SysY run. My read is split: the engineering demos are stronger than a routine Chinese model launch, but the article jumps from demos to “top global model table” far too quickly.

The concrete claims are not small. MiMo-V2.5 includes MiMo-V2.5-Pro, a multimodal base model, TTS, and ASR. MiMo-V2.5-Pro reportedly built a macOS-like desktop in 4 hours without human takeover. The demo had 54 apps, 68 components, React 18, TypeScript, Zustand, Tailwind CSS, and Vite. It also completed the Peking University SysY compiler task in 4.3 hours, using 672 tool calls, and scored 233/233. The article says ClawEval used about 70K tokens per trajectory for a 64% Pass³ rate, while Claude Opus 4.6, Gemini 3.1 Pro, and GPT-5.4 used 120K to 180K tokens. If that holds under outside replication, it matters. Agent cost is not just model pricing; it is tool loops, retries, context growth, and failed trajectories.

The gaps are equally large. The article does not disclose parameter count, MoE layout, training-token count, RL recipe, tool sandbox, sampling settings, failed runs, average pass@k, or repository history for the macOS demo. A 54-app desktop sounds flashy, but static app shells and maintainable software are different artifacts. The claim that the XcodeApp includes a “real browsable web engine” needs inspection. Is that an iframe wrapper, a browser API simulation, or an actual browser implementation? The body does not say. For practitioners, that difference moves the demo from a weekend build to a serious systems result.

The SysY result is the strongest number in the piece. Compiler tasks test long-horizon consistency better than front-end demos. Lexing, parsing, IR generation, RISC-V backend work, and optimization all interfere with each other. A model surviving 672 tool calls without derailing suggests Xiaomi trained or engineered around state management and error recovery. I would compare this to SWE-bench Verified-style agent runs: plenty of models write a good patch for one issue, then start overwriting themselves after 20 to 50 steps when tools, files, and context stack up. If MiMo-V2.5-Pro is reliably stable past 600 turns, that is not copywriting. That is model policy plus agent runtime doing real work.

I care even more about the 70K-token efficiency claim. Agent token reduction usually comes from one of three places: better compressed planning, more aggressive context trimming, or a tool-result summarizer. The first is genuinely valuable. The second hides failure modes. The third depends heavily on the surrounding framework. The article only says “token efficiency.” It does not say whether this comes from MiMo-V2.5-Pro itself or from Xiaomi’s agent framework compressing trajectories. Xiaomi also announced free access for emerging agent frameworks, which makes benchmarking messier. Are we measuring the open model, or a product stack with policy, memory, and tool routing layered around it?

Externally, Xiaomi’s move feels closer to Qwen and DeepSeek distribution playbooks than a pure Claude/GPT benchmark chase. Qwen gained developer mindshare through open weights, many model sizes, and tool coverage. DeepSeek won trust with cheap API economics and reproducible inference cost. Moonshot, Zhipu, and MiniMax have leaned more on products and APIs. Xiaomi has a different asset base: phones, cars, IoT, voice, and OS-level distribution. Shipping TTS, ASR, a multimodal base, and an agent model together makes sense for Xiaomi. The company wants models that can sit inside phones, vehicles, home devices, and developer workflows, not just collect GitHub stars.

The audio claims need caution. The article says MiMo-V2.5-TTS supports text-described voices and zero-shot cloning without reference audio. It says ASR reaches Chinese-English SOTA and handles Cantonese, Sichuanese, Wu, and Minnan. But the “99.999%” recognition number comes from one Cantonese test, with no disclosed duration, noise condition, accent spread, labeling standard, or dataset. ASR is already crowded: Whisper, SenseVoice, Paraformer, and FunASR have made clean speech less impressive. The hard cases are overlapping speakers, far-field audio, music beds, mixed dialects, and low-bitrate telephone speech. The article gives experience notes, not a proper evaluation.

The 100T-token giveaway is the sharpest commercial move. It gives agent builders a reason to test a new base. OpenAI and Anthropic have strong developer habits. Qwen already owns a lot of open-source inertia. A later entrant needs three things at once: good enough quality, low enough cost, and low migration friction. Xiaomi cut Pro credits from 4x to 2x, standard from 2x to 1x, priced 1M and 256K context at the same credit multiplier, and added a 20% night discount from 00:00 to 08:00 Beijing time. Those details matter more than the “global top-tier” language. Agent developers follow invoices faster than slogans.

My biggest problem is the source layer. This is a media experience piece, not a system card and not an independent benchmark report. It mentions Claude Opus 4.6, GPT-5.4, and Gemini 3.1 Pro, but gives no table, benchmark version, evaluator, confidence interval, or reproduction package. The title says the weights are open-sourced, but the body does not disclose the license. Commercial use, acceptable-use limits, redistribution terms, and training-data posture matter more for enterprise adoption than a demo video. Open weights with a restrictive license are downloadable, not freely deployable.

So my stance is simple. MiMo-V2.5 is Xiaomi’s first AI release that looks like a complete platform bet: agent, long context, voice, open weights, and developer subsidy in one package. The article pushes every claim to maximum volume, which makes me wait for third-party replication. I need three things before upgrading my view: public SysY and ClawEval scripts, the full macOS-demo repository trajectory, and clear license terms. If those hold up, Xiaomi becomes a serious player in the Chinese open agent stack. For now, I accept the release has force. I do not yet seat it beside Claude and GPT.

QbitAI · WeChatApr 28, 20:56

Open-source SenseNova-U1 unifies image understanding and generation

SenseTime open-sourced two SenseNova-U1 models: an 8B version and a 38B-total MoE version using NEO-unify. The architecture removes VE and VAE, processes pixels directly, and generates 2048×2048 images in about 9 seconds on one H100/H200 node. The key item is interleaved text-image reasoning; 32K context, long-text rendering, and beta interleaved creation remain limits.

Why it matters: HKR-H/K/R all pass: the architecture hook is concrete, the post gives model sizes and latency, and open multimodal work matters to builders. It stays in 78–84 because it is not a top-tier general-model launch.

Take

SenseNova-U1’s useful bet is not the open-sourced 8B and 38B-total MoE weights; it is the removal of VE/VAE and the attempt to force understanding and generation through one pixel path. The hard hooks are specific: NEO-unify takes pixels in and emits pixels out, one H100/H200 node produces a 2048×2048 image in about 9 seconds, and context is 32K. The WeChat body is blocked by verification, so training data, license terms, and benchmark setup are not available.

I like the direction, but I don’t buy the victory lap yet. Janus-Pro already showed that “unified multimodal” makes a clean headline; the pain lives in text rendering, localized edits, and long interleaved image-text chains. U1 labels long-text rendering and continuous interleaved creation as beta, which is the tell: this is a research-shaped release, not a clean replacement for separate diffusion pipelines.

X · @op7418Apr 28, 10:42

Xiaomi open-sources the MiMo-V2.5 model series

Xiaomi open-sourced the MiMo-V2.5 model series under the MIT license for commercial use, retraining, and fine-tuning. It also launched Orbit 100T Token, offering approved AI builders up to 1.6B credits worth 659 yuan. Agent framework teams can apply for free MiMo token access; the post does not disclose model size or benchmark results.

Why it matters: HKR-H/K/R all pass: Xiaomi MiMo-V2.5 open source, MIT terms, and Orbit 100T credits matter to builders. Missing params and benchmarks keep it in the 78–84 band, below P1.

Take

Xiaomi open-sourced the full MiMo-V2.5 series under the MIT license. That is the clearest fact here. MIT allows commercial use, retraining, and fine-tuning without extra permission. For a phone, EV, and IoT company, this reads less like a pure model launch and more like a developer funnel. The post does not disclose parameter counts, context length, API pricing, SWE-bench, Aider, LiveCodeBench, or Chinese evals. The title says the full series is open-sourced; the body does not show the technical receipts.

My read is blunt: Xiaomi is not proving that MiMo-V2.5 beats DeepSeek, Qwen, or Kimi on capability. It is lowering adoption friction first. The Orbit 100T Token plan offers approved AI builders up to 1.6B credits, listed at 659 yuan. That number is catchy, but the body does not define the credit-to-token conversion, input-output split, rate limits, eligible models, or expiry rules. Without those conditions, 1.6B credits only tells us Xiaomi is willing to subsidize usage. It does not tell us whether the serving economics work.

Qwen is the obvious comparison. Alibaba’s open model strategy worked because it paired permissive licensing with parameter tiers, quantized variants, inference integrations, public benchmarks, and a steady community loop. DeepSeek-R1 hit because reasoning behavior, low API pricing, open weights, and distilled models arrived together. Xiaomi’s disclosed package is thinner. It gives the ecosystem an opening, but not enough evidence for a serious team to route production traffic. Agent framework teams care about latency stability, function-calling reliability, long-context cost, concurrency limits, and failure modes. A large token pool does not answer those questions.

Honestly, the agent ecosystem angle is the part I take more seriously. Xiaomi’s advantage is not model-lab prestige. It is distribution across phones, cars, home devices, and HyperOS. If MiMo becomes the model layer for device control, in-car assistants, household automation, and cross-device workflows, Xiaomi has a deployment surface that many standalone model companies lack. But the post leaves the key facts blank. It does not say whether MiMo-V2.5 supports tool use, structured output, multimodal inputs, on-device variants, or multiple model sizes. Without that, the agent story is a recruitment pitch, not an architecture claim.

I do not buy “fully open-source” as a sufficient signal in 2026. The bar has moved. A permissive license without a model card, reproducible evals, inference recipes, and clear API pricing will not move careful builders off their current stack. Xiaomi made a useful move, but the useful part is channel plus subsidy, not demonstrated model superiority. Once it publishes weights, evals, serving terms, and integration details, MiMo-V2.5 can be compared on the same board as Qwen and DeepSeek. Right now, Xiaomi opened the door. It has not shown what is inside.

r/LocalLLaMAApr 28, 09:32

Local coding models have reached a threshold for real work

Antigma tested 27B–32B open-weight models; Qwen 3.6-27B scored 38.2% on Terminal-Bench 2.0. The run used 89 tasks and the default per-task timeout, while verified SOTA is about 80%. The key claim is deployment lag: offline coding is about 6–8 months behind hosted frontier models.

Why it matters: HKR-H/K/R all pass: the post gives a real-work threshold claim, a 38.2%/89-task Terminal-Bench result, and a 6–8 month offline gap. Reddit single-post sourcing keeps it in the low featured band.

r/LocalLLaMAApr 28, 04:07

Microsoft Presents TRELLIS.2: Open-Source 4B Image-to-3D Model

Microsoft’s title says TRELLIS.2 is an open-source 4B image-to-3D model. The title lists 1536³ PBR assets, native 3D VAEs, and 16× spatial compression; the Reddit body is blocked by 403 and discloses no license or benchmarks.

Why it matters: HKR-H/K/R pass: 4B, 1536³, 16× compression, and open source are concrete. Reddit 403 leaves no paper, license, benchmark, or official link, so the score sits at the featured floor.

Apr 26Sunday

Hacker News front pageApr 26, 21:58

Why SWE-bench Verified No Longer Measures Frontier Coding Capabilities

OpenAI stopped reporting SWE-bench Verified scores and recommends SWE-bench Pro instead. It audited 138 tasks that o3 failed inconsistently across 64 runs and found 59.4% had test or prompt flaws. The key issue is contamination: tested frontier models reproduced some gold patches or task details.

Why it matters: HKR-H/K/R all pass: OpenAI backs the SWE-bench Verified retirement with an audit and contamination evidence, then points to SWE-bench Pro. It affects coding-model evaluation, but it is not a model or major product launch, so it sits in 78–84.

Apr 25Saturday

MIT Technology Review · AIApr 25, 05:40

Three reasons why DeepSeek’s new model matters

DeepSeek released a V4 preview with two versions: V4-Pro and V4-Flash. V4-Pro costs $1.74/M input tokens and $3.48/M output tokens; V4-Flash is about $0.14/$0.28, and both support 1M-token context. The key point is attention efficiency and open weights pressuring agentic coding costs.

Why it matters: HKR-H/K/R all pass: DeepSeek V4 is a domestic flagship release with 1M context, two price tiers, and open-weight cost pressure. The preview status keeps it below a full GPT/Claude major release, but it is same-day material.

Take

DeepSeek released a V4 preview with Pro and Flash tiers. My read is simple: this will not hit like R1, but it will move the cost floor for agentic coding. V4-Pro costs $1.74 per million input tokens and $3.48 per million output tokens. V4-Flash is about $0.14 and $0.28. Both support a 1M-token context window. That package matters more than another benchmark table, because coding agents burn tokens on repos, logs, tool traces, tests, and patch iterations.

MIT Technology Review frames the release around open weights, long-context efficiency, and Chinese chips. The article discloses pricing, API access, web and app availability, and the split between V4-Pro for coding and complex agent tasks and V4-Flash for speed and price. The body we have cuts off during the benchmark section. It does not give complete scores, test sets, comparison models, or degradation curves at long context. So I would not buy the “rivals the best models” line yet. DeepSeek earned attention with R1, but company-run benchmarks are still company-run benchmarks. SWE-bench, Aider polyglot, LiveCodeBench, long-context retrieval, and repo-level bug fixing need independent reruns.

The pricing is the hard part already on the table. I remember Anthropic Claude Sonnet 4.5 landing around $3 per million input tokens and $15 per million output tokens, though I have not rechecked that exact figure. OpenAI’s higher-end lines have also kept long context and tool use in premium territory. V4-Pro at $1.74 and $3.48 is aggressive, especially on output. Coding agents produce a lot of output: plans, diffs, test logs, explanations, retries. Dropping output from the $15 range to $3.48 changes the economics of batch repo work. Flash at $0.14 and $0.28 reads like a sub-agent model for summarization, retrieval compression, test explanation, and first-pass review. That pattern already shows up in Cursor-style and Devin-style stacks: a stronger model handles hard decisions, cheaper models chew through scaffolding.

The 1M-token context window also needs a colder read. The hard part is not accepting 1M tokens. The hard part is using them. Gemini 1.5 Pro made million-token context a public product story long before this, and OpenAI and Anthropic have pushed high-context tiers too. Developers pay for effective context, not advertised context. If V4 lowers attention cost but loses cross-file constraints after 600K tokens, its coding-agent value drops fast. The article says DeepSeek uses a new design for efficient large-text handling. It does not disclose whether that means MLA, sparse attention, block memory, or another mechanism. That missing detail matters because open weights become much more valuable when the community can optimize the inference stack around the architecture.

I am also careful with the “open source” wording. MIT says the model is open source and available to download, use, and modify. In AI, that phrase often blurs four separate things: open weights, open training code, open data, and permissive commercial licensing. R1’s impact came from weight availability and method diffusion, not a fully open training pipeline. If V4 only ships weights and a technical report, enterprises still need to inspect the license, compliance posture, data provenance, and distillation restrictions. The article mentions scrutiny from both the US and Chinese governments. Regulated buyers will not move sensitive workflows because input tokens cost fourteen cents per million.

The Chinese-chip angle needs even more caution. The headline says V4 is a win for Chinese chipmakers. The visible body only says R1 was trained with limited compute and that V4 is more efficient. It does not disclose V4’s training hardware, inference hardware, cluster size, HBM configuration, or domestic accelerator share. If the gain comes from architecture and inference engineering, it does help Huawei Ascend, Cambricon, Moore Threads, and similar ecosystems. Lower attention cost matters more when bandwidth and interconnect are constrained. But “helps the ecosystem” is not the same as “runs production 1M-context workloads on domestic chips today.” We are missing reproducible conditions: batch size, KV-cache strategy, throughput, latency, memory footprint. None of those are in the article body we received.

The part I care about is that DeepSeek is shifting the open-model fight from “can it reason?” to “can it remember a lot cheaply?” R1 pressured the reasoning premium. V4, if independent tests hold up, pressures the default use of closed APIs inside agent workflows. Internal codebases are the obvious wedge. Plenty of teams would rather self-host open weights than send entire repos, tickets, logs, and design docs to an external API.

Still, I would not declare another DeepSeek shock yet. R1 landed because two things were true at once: capability was close enough, and cost was dramatically lower. V4 has clearly shown the second condition. The first still depends on third-party evals and real agent traces. My test would be brutal but fair: give V4-Pro a million-token monorepo, a bug spanning 20 files, failing tests, and noisy logs. Make it patch, rerun, diagnose failures, and preserve style. If it gets near Claude Sonnet 4.5 while costing under one-third as much, closed coding-agent pricing has a real problem.

Bloomberg TechnologyApr 25, 00:18

China’s DeepSeek Unveils New Model a Year After Shock Launch

DeepSeek unveiled a new flagship AI model about one year after its open-source release jolted Silicon Valley. The title and RSS snippet confirm that timing; the post does not disclose the model name, size, pricing, benchmarks, or release terms. The key thing to watch is the missing launch detail, not the comeback framing.

Why it matters: A new DeepSeek flagship is newsworthy: HKR-H comes from the 'one year after the shock launch' hook, and HKR-R from the open-source and pricing rivalry it triggers. HKR-K fails because no model name, params, pricing, or benchmarks are disclosed, so this sits at the low end of the

Take

DeepSeek unveiled a new flagship model about one year after its breakout open release, and the article discloses almost nothing else. My read is simple: this is not yet evidence of a capability jump. It is only a product signal waiting for verification. The headline wants the “DeepSeek is back” story. I care far more about what exactly is back: stronger reasoning, lower token pricing, looser release terms, or just another round of attention.

Look, the force of DeepSeek’s last breakout was never benchmark score alone. It hit because the package was unusual: open weights, aggressive cost claims, deployability, and a Chinese lab forcing Silicon Valley to respond on efficiency instead of sheer spend. That wave also fed into a broader open-model reset. Qwen kept gaining ground, Meta had to keep defending Llama’s practical value, and every frontier lab got dragged back into the question of whether “closed but better” was still enough. If DeepSeek is launching another flagship, the missing details are the whole story. We still do not have the model name, parameter count, architecture, context window, API pricing, benchmark table, or license terms. Without those, nobody serious can tell whether this is aimed at OpenAI and Anthropic on frontier quality or aimed at everyone else on cost-performance.

I also have some pushback on Bloomberg’s framing. Centering the “shock launch” narrative is good TV, but it risks turning this into nostalgia. New models earn relevance through reproducible conditions, not vibes. If DeepSeek is claiming coding gains, show SWE-bench or LiveCodeBench and the test setup. If it is a price attack, publish input and output rates plus throughput. If it is another open release, say whether weights are actually downloadable and what commercial restrictions apply. Over the last year we have seen plenty of launches that looked huge on day one and then faded once deployment details showed up. I treat any first report that skips those details with caution.

I have not found primary launch materials yet, so I would keep this marked as incomplete. DeepSeek did ship something. That much is clear. What is still unclear is whether this resets the open-model frontier again or just fills a product gap while the market waits for hard numbers.