Skip to content

Open source

Open models, frameworks and repositories: open weights, community hits and the balance between open and closed.

Latest picks

321–340 of 388

May 5Tuesday

r/LocalLLaMA

FastDMS: 6.4X KV-cache compression running faster than vLLM BF16/FP8

FastDMS released an MIT implementation that cuts KV memory to 1/5–1/8 of vLLM BF16 at 8K context. A Llama-3.2-1B replication reports PPL 9.200 with 6.4x compression; Qwen3-8B c=1 drops KV from 1.406 GiB to 0.184 GiB. The key detail is physical reclamation of evicted slots, not just nominal KV-byte reduction.

Why it matters: FastDMS open-sourced an MIT-licensed KV cache compression implementation that cuts Llama-3.2-1B's cache to one-fifth to one-eighth at 8K context, with concrete perplexity and memory figures.

May 4Monday

r/LocalLLaMA

Deep research report with Hermes Agent and qwen3.6-35b-a3b Q6_K

A Reddit user used Hermes Agent and qwen3.6-35b-a3b Q6_K to produce a 21-page research report. The run took 6 loops and over 5 hours on an RTX 4060, at about 28 tokens/s. The repo includes prompts, scripts, intermediate artifacts, and the final report.

Why it matters: A Reddit user ran Hermes Agent with a quantized qwen3.6-35b on one RTX 4060 through six loops to produce a 21-page report, with prompts and scripts in the repo.

QbitAI · WeChat

DeepSeek-TUI, a “DeepSeek Claude Code,” reaches 2.3k GitHub stars

DeepSeek-TUI reached 2.3k GitHub stars; the Rust project is MIT-licensed. It targets DeepSeek V4 with a 1M-token context, RLM up to 16 V4 Flash subtasks, MCP, Shell, Git, and three control modes. Watch cache misses: uncached tokens cost 10x cached tokens.

Why it matters: DeepSeek-TUI is a Rust terminal coding assistant wired to DeepSeek V4 with 1M-token context, MIT-licensed, for developers studying command-line AI coding tools.

r/LocalLLaMA

450M On-Board VLM Wildfire Detection Pipeline with Sentinel-2 and LFM2.5-VL

PauLabartaBajo shared a wildfire detection PoC using 450M LFM2.5-VL on Sentinel-2 imagery. It pairs RGB and SWIR tiles, simulates orbit with SimSat, and covers 22 fire-prone sites. The key constraint is bandwidth: on-board inference downlinks only a JSON risk profile.

Why it matters: The author deployed a 450M vision-language model on the satellite itself for wildfire inference and downlinks only JSON risk assessments, sidestepping bandwidth limits on multispectral imagery.

May 3Sunday

r/LocalLLaMA

LLM proxy that lets Claude Code talk to any model

DataNebula released open-source rosetta-llm, letting Claude Code call multiple providers through one gateway. It translates Anthropic Messages, OpenAI Chat, and OpenAI Responses, and round-trips encrypted reasoning via the signature field. The key detail is thinking-block fidelity for multi-turn agent prompt-cache hits.

Why it matters: DataNebula's open-source rosetta-llm acts as a translation gateway so Claude Code can call models from other vendors, one route to mixing models.

r/LocalLLaMA

Upskill: skill registry your agent consults before it starts, with 10k+ indexed skills

Autoloops released Upskill, an open-source skill registry with 10k+ indexed skills for agents. Search combines Postgres full-text search, 1024-dim embeddings, and reranking by stars, installs, and feedback. LLM adversarial review blocked hundreds of skills at index time.

Why it matters: Autoloops open-sourced the Upskill registry, indexing over 10,000 skills an agent can search before starting a task, ranked by full-text, vector match and install feedback.

QbitAI · WeChat

GS-Playground Embodied AI Simulation Framework Open-Sourced with High-Throughput 3DGS Rendering

Tsinghua AIR DISCOVER Lab and partners open-sourced GS-Playground, accepted by RSS 2026. On an RTX 4090, it reports 10,000 FPS at 640×480 and 2,048 parallel scenes; a 50-humanoid benchmark reaches 1,015 FPS. The key point is coupling batch 3DGS rendering with parallel physics.

Why it matters: A single RTX 4090 hitting 10,000 fps while simulating 2,048 scenes at once sets a new bar for robot-learning environments.

r/LocalLLaMA

Built a C++17 transformer from scratch with 0.83M params and CPU training

Reddit user Suspicious_Gap1121 released Quadtrix.cpp, a C++17 GPT-style model with 0.83M parameters. It uses 4 layers, 4 heads, 200d width, and a 128-character context; one CPU core trained on 31.4M characters for 76.2 minutes to 1.6371 nats val loss. The key detail is handwritten backprop for LayerNorm, attention, Q/K/V, dropout, and AdamW without PyTorch, BLAS, or autograd.

Why it matters: A from-scratch C++17 transformer with hand-written backprop shows what a small GPT actually costs to train without PyTorch or BLAS.

May 2Saturday

r/LocalLLaMA

I built Semvec: A constant-cost semantic memory for LLMs, looking for testers

A developer released Semvec, replacing unbounded chat history with fixed-size semantic state. Its 48-turn benchmark claims about 76% token reduction, with identical input footprint at turn 10 and 10,000. It supports OpenAI-compatible LLMs, MCP, Claude Code, Cursor, and multi-agent shared state.

Why it matters: Swapping chat history for fixed-size semantic vectors cuts token use about 76% and keeps input cost flat as a conversation grows.

r/LocalLLaMA

Qwen3.6-27B hits 72 tok/s on RTX 3090 with native vLLM on Windows

Reddit user One_Slip1455 released a native Windows vLLM launcher for Qwen3.6-27B, reaching 72 tok/s on an RTX 3090. It reports 64.5 tok/s at ~25k tokens, 53.4 tok/s at 127k ctx on one GPU, and 160k ctx with PP=2 on 2×3090. The key detail is no WSL or Docker, an OpenAI-compatible endpoint, and an INT4 quant path.

Why it matters: A native Windows vLLM launcher runs Qwen3.6-27B on one RTX 3090 without WSL or Docker, hitting 72 tok/s at short context.

QbitAI · WeChat

Tencent Hunyuan open-sources 440MB offline translation model, claims Google Translate quality lead

Tencent Hunyuan open-sourced Hy-MT1.5-1.8B-1.25bit, compressing a 1.8B translation model to 440MB. It supports 33 languages and 1,056 directions, with an Android demo running offline on Snapdragon 888 and 8GB RAM. The key detail is Sherry 1.25-bit quantization: 3 of every 4 weights use 1 bit and 1 is zeroed.

Why it matters: Tencent open-sourced a 1.8B translation model that quantizes to 1.25 bits and 440MB, runs offline on a Snapdragon 888 phone, and covers 33 languages and 1,056 translation directions.

May 1Friday

r/LocalLLaMA

PFlash: 10x prefill speedup over llama.cpp at 128K on an RTX 3090

PFlash cuts Qwen3.6-27B Q4_K_M 128K TTFT to 24.8s on an RTX 3090, versus 248.4s cold for llama.cpp. It uses a Qwen3-0.6B drafter to score token importance, keeps 5% of spans, and runs C++/CUDA without Python, Triton, or PyTorch. The quality caveat is clear: only NIAH single-needle passes from 32K to 128K; RULER and multi-needle results are not disclosed.

Why it matters: PFlash uses Qwen3-0.6B to pick the 5% of chunks that matter, cutting first-token latency on a 128K-token RTX 3090 run from 248.4s to 24.8s.

Xinzhiyuan · WeChat

Developer Builds WorldX, an AI World Generator, During a 10-Day Wedding Leave

An independent developer built WorldX in 10 days, generating a full AI world from one sentence in about 5 minutes. The system uses a 6-step map pipeline, about 30k–180k tokens per world, Tick loops, layered memory, and two-axis emotion. The key mechanism is overlay labeling plus color-difference localization for deterministic coordinates.

Why it matters: A developer built WorldX during a 10-day marriage leave; one sentence generates a full AI world in about 5 minutes, and the post details a 6-step map pipeline, token ranges and a deterministic coordinate scheme.

QbitAI · WeChat

Peking University Open-Sources Unified World Model Framework for Synthesis and Reasoning Tasks

Peking University DCAI and Kuaishou Kling open-sourced OpenWorldLib for four task types: video generation, 3D modeling, VLA control, and multimodal reasoning. Its Pipeline coordinates Operator, Reasoning, Synthesis, Representation, and Memory modules, supporting forward and stream execution. The key test is whether unified interfaces cut cross-task reproduction cost.

Why it matters: Peking University DCAI and Kuaishou Kling open-sourced OpenWorldLib, putting video generation, 3D modeling, VLA robot control and multimodal reasoning into one pipeline, split into operator, reasoning, synthesis, representation and memory modules.

Hacker News front page

Show HN: Pu.sh – a full coding-agent harness in 400 lines of shell

Pu.sh ships a coding-agent harness in about 400 lines of shell, using only sh, curl, and awk. It supports Anthropic and OpenAI, 7 tools, REPL, auto-compaction, checkpoint/resume, pipe mode, and 90 no-API tests. It excludes TUI, streaming, images, OAuth, and Windows.

Why it matters: Pu.sh builds a full coding-agent harness in about 400 lines of shell, covering tool calls, context compaction and resumable runs, a minimal readable reference for anyone who wants the core agent loop without heavy dependencies.

Apr 30Thursday

r/LocalLLaMA

inclusionAI/Ling-2.6-1T · Hugging Face

inclusionAI open-sourced Ling-2.6-1T on Hugging Face, with 1 trillion parameters. It uses MLA plus Linear Attention and Contextual Process Redundancy Suppression to reduce CoT overhead. The post cites AIME26 and SWE-bench Verified but does not disclose scores.

Why it matters: inclusionAI open-sourced the trillion-parameter Ling-2.6-1T on Hugging Face, using MLA and linear attention plus context process redundancy suppression, an architecture reference for anyone working on large-model inference optimization.

X · @dotey

Inside Hermes Agent's Memory System and How It Avoids OpenClaw's Pitfalls

Hermes Agent splits memory into 4 layers: prompt files, SQLite session search, skills, and optional Honcho. MEMORY.md is capped at 2,200 chars, USER.md at 1,375; writes apply after a new session or compression. The key design is cache-first: keep system prompts stable and retrieve long-tail history via tools.

Why it matters: Hermes Agent's memory stack has four layers: prompt memory, SQLite session history, skill management and Honcho user modeling, and the piece breaks down how its memory design choices differ from OpenClaw's.

Apr 29Wednesday

QbitAI · WeChat

Avenir-Web Open-Sources Web Agent Harness With 53.7% on ONLINE-MIND2WEB

UCL, Princeton, and Edinburgh open-sourced Avenir-Web, reaching 53.7% success on ONLINE-MIND2WEB. The training-free harness uses EIP, MoGE, checklists, and adaptive memory across 136 sites and 300 live tasks. The key signal: with Gemini 3 Pro, it beats Claude Computer Use 3.7 at 47.3%.

Why it matters: Avenir-Web from UCL, Princeton and Edinburgh plugs into models without extra training and hits 53.7% success on ONLINE-MIND2WEB, 136 sites and 300 real tasks.

r/LocalLLaMA

SenseNova-U1: Unifying Multimodal Understanding and Generation with NEO-Unify Architecture

SenseNova released SenseNova-U1 with 4 MoT multimodal models. The post lists 8B and A3B variants with GitHub and HuggingFace weight links. The key claim is a monolithic architecture instead of adapters; benchmarks are not disclosed.

Why it matters: SenseNova-U1 handles text and images inside the model with no external image adapter, and releases 8B and A3B MoT weights, with A3B smaller and suited to local runs.

X · @dotey

Microsoft VibeVoice-ASR tested on Mac for a one-hour podcast

Simon Willison ran 4-bit VibeVoice-ASR on an M5 Max MacBook Pro and transcribed a one-hour podcast in 8m45s. The 9B MIT-licensed model supports 60-minute audio, 50+ languages, and structured speaker output. Memory is the constraint: prefill peaked at 61.5GB, making 32GB laptops impractical.

Why it matters: Simon Willison ran Microsoft's open-source VibeVoice-ASR on an M5 Max with 128GB of memory: 8 minutes 45 seconds to transcribe a one-hour podcast, with prefill memory peaking at 61.5GB.