Skip to content

#开源/仓库

2 today

May 9Saturday

AI HOT (Curated Pool)

Redis founder uses a C inference engine to run a large model on a personal computer

Antirez open-sourced ds4, a native inference engine for DeepSeek V4 Flash that uses a few thousand lines of C to run a 1M-context model on a 128GB MacBook Pro at a reported 27 tok/s.

Why it matters: Antirez's ds4 runs DeepSeek V4 Flash's million-token context on a 128GB MacBook Pro in a few thousand lines of C, with 2-bit MoE quantization and KV cache spilled to SSD.

Xinzhiyuan · WeChat

CUHK Open-Sources ArbiterOS Agent Governance Kernel With 92.95% High-Risk Interception

CUHK CURE Lab open-sourced ArbiterOS, an agent runtime governance kernel that intercepts, parses, governs, and observes actions before execution, raising high-risk step interception on OpenClaw tasks from 6.17% to 92.95%.

Why it matters: CUHK's CURE lab open-sourced ArbiterOS, a runtime governance layer between model output and system execution, lifting the block rate on high-risk OpenClaw steps from 6.17% to 92.95%.

Synced · WeChat

StarVLA Open-Sources a Unified VLA Framework from HKUST and the Community

HKUST and the open-source community released StarVLA, a unified Vision-Language-Action framework that integrates backbones, action heads, training strategies, and evaluation interfaces; the repository has 2.2k GitHub stars and supports benchmarks including LIBERO, SimplerEnv, RoboTwin 2.0, RoboCasa-GR1, and BEHAVIOR-1K.

Why it matters: HKUST and the community open-sourced StarVLA, making VLA backbones, action heads and training interfaces pluggable instead of rebuilt per paper.

r/LocalLLaMA

MTP + TurboQuant Running: Qwen3.6-27B Hits 80+ t/s on a Single RTX 4090

indrasmirror ran Qwen3.6-27B-Heretic-v2 on a single RTX 4090 with 262K context, TBQ4_0 KV cache, and MTP draft 3, improving throughput from about 43 t/s to 80-87 t/s with roughly 73% MTP draft acceptance.

Why it matters: indrasmirror pushed Qwen3.6-27B to 262K context on one RTX 4090, using MTP speculative decoding to lift generation from about 43 t/s to 80-87 t/s at a 73% draft acceptance rate.

May 8Friday

Hacker News front page

Show HN: Git for AI Agents

regent-vcs released the open-source re_gent project for AI-agent version control, currently supporting Claude Code, with workflows for tracking why an agent changed files, rewinding sessions, and bisecting agent actions; the post does not disclose the license, storage format, or installation details.

Why it matters: regent-vcs's open-source re_gent adds Git-like version control to Claude Code, recording why the AI changed files, supporting conversation rollback, and using bisection to find which step caused a problem.

AI HOT (Curated Pool)

Donating the Open-Source Alignment Tool Petri

Anthropic transferred the open-source alignment testing tool Petri to Meridian Labs to preserve independence and credibility. Petri 3.0 separates auditor and target models, adds Dish for real prompts and deployment settings, and integrates Bloom.

Why it matters: Anthropic handed its internal alignment tool Petri, built to catch model lying and sycophancy, to nonprofit evaluator Meridian Labs, putting the tool outside the developer and giving evaluation neutrality organizational backing.

AI HOT (Curated Pool)

DeepSeek 4: Flash Local Inference Engine for Metal

DeepSeek 4 Flash is open-sourced on GitHub for offline inference on Apple Silicon Macs. The post says it uses Metal Performance Shaders to reduce latency and memory use, but discloses no benchmark numbers. The key item is the Metal local inference stack, not another model wrapper.

Why it matters: antirez, the creator of Redis, shipped ds4 to run DeepSeek 4 Flash locally on a Mac, calling Apple's Metal Performance Shaders directly to cut latency and memory use.

May 7Thursday

AI HOT (Curated Pool)

SenseNova-U1 Open-Sources 8-Step Distilled LoRA, Speeds Diffusion Inference by 11x

SenseNova-U1 open-sourced an 8-step distilled LoRA that cuts diffusion generation from 100 steps to 8. GPU inference time drops from 23 seconds to 2 seconds, with ComfyUI workflows for text-to-image, image editing, and interleaved generation. The key signal is distillation for latency, not parameter scale.

Why it matters: An open 8-step LoRA cuts diffusion sampling from 100 steps to 8, dropping GPU inference from 23 seconds to 2, with a ComfyUI workflow included.

r/LocalLLaMA

Running Qwen3.5/Qwen3.6 with NextN MTP in llama.cpp on one RTX 3090 Ti

A Reddit user posted a llama.cpp guide for Qwen3.5/3.6 with NextN MTP on one RTX 3090 Ti. It requires two unmerged PRs, #22400 and #22673; Qwen3.6-35B-A3B-MTP reaches 157 tok/s at 350W, 1700MHz, with q8 KV. The key reproducible detail is nextn=q8_0 quant override; missing it yields “////” output.

Why it matters: A Reddit user posts a hands-on llama.cpp guide for running Qwen3.5 and 3.6 on a single RTX 3090 Ti, built on two unmerged PRs and covering NextN MTP speculative decoding setup.

AI HOT (Curated Pool)

Open Slide lets AI write PPT code

Open Slide builds PPTs with React, using a workflow designed for AI agents. It integrates SVGL with 1,500+ brand logos, supports manual edits, and lets AI read user comments for revisions.

Why it matters: Open Slide turns decks into React components, so AI can read comments and edit the file directly, with over 1,500 brand logos wired in.

r/LocalLLaMA

GB10 inference engine Atlas is open source, with Qwen3.6-35B-FP8 over 100 tok/s

Avarok open-sourced Atlas, an inference engine running Qwen3.5-35B at ~111 tok/s sustained on one DGX Spark. It uses Rust+CUDA, a ~2.5GB image, and sub-2-minute cold start; the author claims 3.0–3.3x vLLM in tests. The key details are Blackwell SM120/121 kernels, NVFP4/FP8, and MTP decoding.

Why it matters: Avarok open-sourced Atlas, a Rust and CUDA inference engine, hitting about 111 tok/s for Qwen3.5-35B on a DGX Spark, which the author puts at 3.0 to 3.3 times vLLM.

May 6Wednesday

r/LocalLLaMA

CopilotKit (MIT): Open-source building blocks for agent apps and generative UI

CopilotKit offers MIT-licensed React components and claims 30k GitHub stars. It covers chat, streaming, tool calls, HITL, and generative UI, with AG-UI support for LangGraph, CrewAI, LlamaIndex, and other backends. The key point is decoupling the UI layer from agent frameworks.

Why it matters: CopilotKit ships chat, streaming, tool calls and human-in-the-loop as MIT-licensed React components, reusable blocks for building agent app frontends.

r/LocalLLaMA

2.5x Faster Inference with Qwen 3.6 27B Using MTP on 48GB

A llama.cpp PR adds MTP support for Qwen 3.6 27B, with a reported 2.5x inference speedup. The author measured 28 tok/s on a Mac M2 Max 96GB and shared GGUF builds, compile steps, and a 262144-context server command. The key detail is turbo4 4.25-bit KV cache: a 48GB Mac runs Q5_K_M at 262K context.

Why it matters: A llama.cpp PR adds MTP support for Qwen 3.6 27B, measured at 2.5x faster inference with 262k context on 48GB, a usable setup for local agentic coding.

Synced · WeChat

Alibaba open-sources PromptEcho for T2I rewards using frozen VLMs

Alibaba open-sourced PromptEcho, which uses one frozen Qwen3-VL-32B forward pass to score T2I training rewards. It computes token-level cross-entropy for the original prompt under teacher forcing, then uses the negative value as a continuous reward. In 5,000 poster tests, text accuracy rose from 68% to 75%.

Why it matters: Alibaba open-sources PromptEcho, which uses a frozen Qwen3-VL-32B to generate continuous reward signals without training a separate scorer, lifting text accuracy from 68% to 75% on 5,000 posters.

Synced · WeChat

DeepSeek Version of Claude Code Tops Trending Chart With 8,700 Stars

DeepSeek TUI topped GitHub trending with over 8,700 stars. Hunter Bown built it in Rust for local terminal use with DeepSeek V4, supporting chat, file edits, shell commands, and task management. The key detail is RLM mode: up to 16 V4 Flash subtasks, plus a 1M-token context window and approval gates.

Why it matters: Developer Hunter Bown's Rust-written DeepSeek TUI has over 8,700 GitHub stars, letting users chat, edit files and run shell commands against DeepSeek V4 from the local terminal.

Synced · WeChat

Two Chinese open-source projects turn Mac into a private AI workstation

Mininglamp open-sourced Cider and Mano-P 1.0 for Apple Silicon local inference and GUI agents. Cider speeds Qwen3-VL-2B prefill by 57%–61% on M5 Pro; Mano-P 1.0-72B scores 58.2% on OSWorld. The key constraint is W8A8 memory: on 16GB devices accuracy falls from 58.0% to 54.0%, so 32GB+ is recommended.

Why it matters: Minglue open-sources Cider and Mano-P 1.0, with Cider cutting Qwen3-VL-2B first-token latency about 60% on M5 Pro and the 72B Mano-P scoring 58.2 on OSWorld.

May 5Tuesday

r/LocalLLaMA

SenseNova-U1-8B-MoT open-source multimodal architecture draws LocalLLaMA discussion

SenseNova open-sourced SenseNova-U1-8B-MoT, an 8B native multimodal understanding and image-generation model. Its Hugging Face text says NEO-Unify removes VE and VAE, supports interleaved image-text generation, and high-density rendering; the post does not disclose test scores. The key question is whether the monolithic design yields reproducible gains.

Why it matters: SenseTime's open 8B multimodal model drops the vision encoder and VAE, putting image understanding and generation in one model, a different trade-off in multimodal architecture.

r/LocalLLaMA

vibevoice.cpp: Microsoft VibeVoice ported to ggml/C++ with no Python at inference

LocalAI released vibevoice.cpp, a ggml/C++ port of Microsoft VibeVoice for CPU, CUDA, Metal, and Vulkan inference. TTS uses a 30s reference clip for 24kHz cloned speech; ASR uses a 7B model with diarized JSON and was tested on 17min audio. The key constraint is memory: 17min CPU Q8_0 peaks near 26GB, with no streaming output yet.

Why it matters: LocalAI rewrote Microsoft VibeVoice's speech synthesis and long-audio transcription as a pure C++ version that runs on CPU, CUDA, Metal and Vulkan, so local deployment no longer needs a Python environment.

r/LocalLLaMA

FastDMS: 6.4X KV-cache compression running faster than vLLM BF16/FP8

FastDMS released an MIT implementation that cuts KV memory to 1/5–1/8 of vLLM BF16 at 8K context. A Llama-3.2-1B replication reports PPL 9.200 with 6.4x compression; Qwen3-8B c=1 drops KV from 1.406 GiB to 0.184 GiB. The key detail is physical reclamation of evicted slots, not just nominal KV-byte reduction.

Why it matters: FastDMS open-sourced an MIT-licensed KV cache compression implementation that cuts Llama-3.2-1B's cache to one-fifth to one-eighth at 8K context, with concrete perplexity and memory figures.

May 4Monday

r/LocalLLaMA

Deep research report with Hermes Agent and qwen3.6-35b-a3b Q6_K

A Reddit user used Hermes Agent and qwen3.6-35b-a3b Q6_K to produce a 21-page research report. The run took 6 loops and over 5 hours on an RTX 4060, at about 28 tokens/s. The repo includes prompts, scripts, intermediate artifacts, and the final report.

Why it matters: A Reddit user ran Hermes Agent with a quantized qwen3.6-35b on one RTX 4060 through six loops to produce a 21-page report, with prompts and scripts in the repo.

QbitAI · WeChat

DeepSeek-TUI, a “DeepSeek Claude Code,” reaches 2.3k GitHub stars

DeepSeek-TUI reached 2.3k GitHub stars; the Rust project is MIT-licensed. It targets DeepSeek V4 with a 1M-token context, RLM up to 16 V4 Flash subtasks, MCP, Shell, Git, and three control modes. Watch cache misses: uncached tokens cost 10x cached tokens.

Why it matters: DeepSeek-TUI is a Rust terminal coding assistant wired to DeepSeek V4 with 1M-token context, MIT-licensed, for developers studying command-line AI coding tools.

r/LocalLLaMA

450M On-Board VLM Wildfire Detection Pipeline with Sentinel-2 and LFM2.5-VL

PauLabartaBajo shared a wildfire detection PoC using 450M LFM2.5-VL on Sentinel-2 imagery. It pairs RGB and SWIR tiles, simulates orbit with SimSat, and covers 22 fire-prone sites. The key constraint is bandwidth: on-board inference downlinks only a JSON risk profile.

Why it matters: The author deployed a 450M vision-language model on the satellite itself for wildfire inference and downlinks only JSON risk assessments, sidestepping bandwidth limits on multispectral imagery.

May 3Sunday

r/LocalLLaMA

LLM proxy that lets Claude Code talk to any model

DataNebula released open-source rosetta-llm, letting Claude Code call multiple providers through one gateway. It translates Anthropic Messages, OpenAI Chat, and OpenAI Responses, and round-trips encrypted reasoning via the signature field. The key detail is thinking-block fidelity for multi-turn agent prompt-cache hits.

Why it matters: DataNebula's open-source rosetta-llm acts as a translation gateway so Claude Code can call models from other vendors, one route to mixing models.

r/LocalLLaMA

Upskill: skill registry your agent consults before it starts, with 10k+ indexed skills

Autoloops released Upskill, an open-source skill registry with 10k+ indexed skills for agents. Search combines Postgres full-text search, 1024-dim embeddings, and reranking by stars, installs, and feedback. LLM adversarial review blocked hundreds of skills at index time.

Why it matters: Autoloops open-sourced the Upskill registry, indexing over 10,000 skills an agent can search before starting a task, ranked by full-text, vector match and install feedback.

QbitAI · WeChat

GS-Playground Embodied AI Simulation Framework Open-Sourced with High-Throughput 3DGS Rendering

Tsinghua AIR DISCOVER Lab and partners open-sourced GS-Playground, accepted by RSS 2026. On an RTX 4090, it reports 10,000 FPS at 640×480 and 2,048 parallel scenes; a 50-humanoid benchmark reaches 1,015 FPS. The key point is coupling batch 3DGS rendering with parallel physics.

Why it matters: A single RTX 4090 hitting 10,000 fps while simulating 2,048 scenes at once sets a new bar for robot-learning environments.

r/LocalLLaMA

Built a C++17 transformer from scratch with 0.83M params and CPU training

Reddit user Suspicious_Gap1121 released Quadtrix.cpp, a C++17 GPT-style model with 0.83M parameters. It uses 4 layers, 4 heads, 200d width, and a 128-character context; one CPU core trained on 31.4M characters for 76.2 minutes to 1.6371 nats val loss. The key detail is handwritten backprop for LayerNorm, attention, Q/K/V, dropout, and AdamW without PyTorch, BLAS, or autograd.

Why it matters: A from-scratch C++17 transformer with hand-written backprop shows what a small GPT actually costs to train without PyTorch or BLAS.

May 2Saturday

r/LocalLLaMA

I built Semvec: A constant-cost semantic memory for LLMs, looking for testers

A developer released Semvec, replacing unbounded chat history with fixed-size semantic state. Its 48-turn benchmark claims about 76% token reduction, with identical input footprint at turn 10 and 10,000. It supports OpenAI-compatible LLMs, MCP, Claude Code, Cursor, and multi-agent shared state.

Why it matters: Swapping chat history for fixed-size semantic vectors cuts token use about 76% and keeps input cost flat as a conversation grows.

r/LocalLLaMA

Qwen3.6-27B hits 72 tok/s on RTX 3090 with native vLLM on Windows

Reddit user One_Slip1455 released a native Windows vLLM launcher for Qwen3.6-27B, reaching 72 tok/s on an RTX 3090. It reports 64.5 tok/s at ~25k tokens, 53.4 tok/s at 127k ctx on one GPU, and 160k ctx with PP=2 on 2×3090. The key detail is no WSL or Docker, an OpenAI-compatible endpoint, and an INT4 quant path.

Why it matters: A native Windows vLLM launcher runs Qwen3.6-27B on one RTX 3090 without WSL or Docker, hitting 72 tok/s at short context.

QbitAI · WeChat

Tencent Hunyuan open-sources 440MB offline translation model, claims Google Translate quality lead

Tencent Hunyuan open-sourced Hy-MT1.5-1.8B-1.25bit, compressing a 1.8B translation model to 440MB. It supports 33 languages and 1,056 directions, with an Android demo running offline on Snapdragon 888 and 8GB RAM. The key detail is Sherry 1.25-bit quantization: 3 of every 4 weights use 1 bit and 1 is zeroed.

Why it matters: Tencent open-sourced a 1.8B translation model that quantizes to 1.25 bits and 440MB, runs offline on a Snapdragon 888 phone, and covers 33 languages and 1,056 translation directions.

May 1Friday

r/LocalLLaMA

PFlash: 10x prefill speedup over llama.cpp at 128K on an RTX 3090

PFlash cuts Qwen3.6-27B Q4_K_M 128K TTFT to 24.8s on an RTX 3090, versus 248.4s cold for llama.cpp. It uses a Qwen3-0.6B drafter to score token importance, keeps 5% of spans, and runs C++/CUDA without Python, Triton, or PyTorch. The quality caveat is clear: only NIAH single-needle passes from 32K to 128K; RULER and multi-needle results are not disclosed.

Why it matters: PFlash uses Qwen3-0.6B to pick the 5% of chunks that matter, cutting first-token latency on a 128K-token RTX 3090 run from 248.4s to 24.8s.

Xinzhiyuan · WeChat

Developer Builds WorldX, an AI World Generator, During a 10-Day Wedding Leave

An independent developer built WorldX in 10 days, generating a full AI world from one sentence in about 5 minutes. The system uses a 6-step map pipeline, about 30k–180k tokens per world, Tick loops, layered memory, and two-axis emotion. The key mechanism is overlay labeling plus color-difference localization for deterministic coordinates.

Why it matters: A developer built WorldX during a 10-day marriage leave; one sentence generates a full AI world in about 5 minutes, and the post details a 6-step map pipeline, token ranges and a deterministic coordinate scheme.

QbitAI · WeChat

Peking University Open-Sources Unified World Model Framework for Synthesis and Reasoning Tasks

Peking University DCAI and Kuaishou Kling open-sourced OpenWorldLib for four task types: video generation, 3D modeling, VLA control, and multimodal reasoning. Its Pipeline coordinates Operator, Reasoning, Synthesis, Representation, and Memory modules, supporting forward and stream execution. The key test is whether unified interfaces cut cross-task reproduction cost.

Why it matters: Peking University DCAI and Kuaishou Kling open-sourced OpenWorldLib, putting video generation, 3D modeling, VLA robot control and multimodal reasoning into one pipeline, split into operator, reasoning, synthesis, representation and memory modules.

Hacker News front page

Show HN: Pu.sh – a full coding-agent harness in 400 lines of shell

Pu.sh ships a coding-agent harness in about 400 lines of shell, using only sh, curl, and awk. It supports Anthropic and OpenAI, 7 tools, REPL, auto-compaction, checkpoint/resume, pipe mode, and 90 no-API tests. It excludes TUI, streaming, images, OAuth, and Windows.

Why it matters: Pu.sh builds a full coding-agent harness in about 400 lines of shell, covering tool calls, context compaction and resumable runs, a minimal readable reference for anyone who wants the core agent loop without heavy dependencies.

Apr 30Thursday

r/LocalLLaMA

inclusionAI/Ling-2.6-1T · Hugging Face

inclusionAI open-sourced Ling-2.6-1T on Hugging Face, with 1 trillion parameters. It uses MLA plus Linear Attention and Contextual Process Redundancy Suppression to reduce CoT overhead. The post cites AIME26 and SWE-bench Verified but does not disclose scores.

Why it matters: inclusionAI open-sourced the trillion-parameter Ling-2.6-1T on Hugging Face, using MLA and linear attention plus context process redundancy suppression, an architecture reference for anyone working on large-model inference optimization.

X · @dotey

Inside Hermes Agent's Memory System and How It Avoids OpenClaw's Pitfalls

Hermes Agent splits memory into 4 layers: prompt files, SQLite session search, skills, and optional Honcho. MEMORY.md is capped at 2,200 chars, USER.md at 1,375; writes apply after a new session or compression. The key design is cache-first: keep system prompts stable and retrieve long-tail history via tools.

Why it matters: Hermes Agent's memory stack has four layers: prompt memory, SQLite session history, skill management and Honcho user modeling, and the piece breaks down how its memory design choices differ from OpenClaw's.

Apr 29Wednesday

QbitAI · WeChat

Avenir-Web Open-Sources Web Agent Harness With 53.7% on ONLINE-MIND2WEB

UCL, Princeton, and Edinburgh open-sourced Avenir-Web, reaching 53.7% success on ONLINE-MIND2WEB. The training-free harness uses EIP, MoGE, checklists, and adaptive memory across 136 sites and 300 live tasks. The key signal: with Gemini 3 Pro, it beats Claude Computer Use 3.7 at 47.3%.

Why it matters: Avenir-Web from UCL, Princeton and Edinburgh plugs into models without extra training and hits 53.7% success on ONLINE-MIND2WEB, 136 sites and 300 real tasks.

r/LocalLLaMA

SenseNova-U1: Unifying Multimodal Understanding and Generation with NEO-Unify Architecture

SenseNova released SenseNova-U1 with 4 MoT multimodal models. The post lists 8B and A3B variants with GitHub and HuggingFace weight links. The key claim is a monolithic architecture instead of adapters; benchmarks are not disclosed.

Why it matters: SenseNova-U1 handles text and images inside the model with no external image adapter, and releases 8B and A3B MoT weights, with A3B smaller and suited to local runs.

X · @dotey

Microsoft VibeVoice-ASR tested on Mac for a one-hour podcast

Simon Willison ran 4-bit VibeVoice-ASR on an M5 Max MacBook Pro and transcribed a one-hour podcast in 8m45s. The 9B MIT-licensed model supports 60-minute audio, 50+ languages, and structured speaker output. Memory is the constraint: prefill peaked at 61.5GB, making 32GB laptops impractical.

Why it matters: Simon Willison ran Microsoft's open-source VibeVoice-ASR on an M5 Max with 128GB of memory: 8 minutes 45 seconds to transcribe a one-hour podcast, with prefill memory peaking at 61.5GB.

r/LocalLLaMA

XiaomiMiMo MiMo-V2.5: Sparse MoE with 310B total and 15B activated parameters

XiaomiMiMo shared MiMo-V2.5 with 310B total parameters and 15B activated parameters. The post only links Hugging Face and says it runs on more “human” configs than its larger sibling. It does not disclose VRAM needs, quantization, or benchmarks.

Why it matters: Xiaomi released the non-Pro MiMo-V2.5 on Hugging Face, a sparse MoE with 310B total parameters and 15B activated per inference, calling on large capacity at lower compute.

X · @dotey

AI terminal tool Warp open-sources client code with OpenAI as founding sponsor

Warp open-sourced its client code under AGPL; only the client is open, while server code stays closed. The Rust terminal has 700,000+ developers, and its Oz cloud AI handles coding, planning, and tests. The key signal is its AI-first contribution workflow.

Why it matters: Warp open-sourced its Rust client under AGPL while the server stays closed, with OpenAI as founding sponsor, giving developers a look at terminal tools plus AI command lines.