Skip to content

#部署/工程

11 today

May 14Thursday

Bloomberg Technology

AI Chipmaker Cerebras Raises $5.55 Billion in Year’s Biggest IPO

Cerebras Systems raised $5.55 billion in a US IPO, and the title calls it the year’s biggest IPO; the RSS snippet does not disclose the offering price or valuation.

Why it matters: Cerebras raised $5.55 billion in an IPO priced at $185 a share, money earmarked for expanding production of its giant AI training chips and customer growth.

AI HOT (Curated Pool)

Meta AI chief announces Incognito Chat for WhatsApp and Meta AI

Meta’s AI chief announced Incognito Chat for WhatsApp and Meta AI, with conversation inference running inside the phone’s hardware secure enclave, no server logs generated, and session data permanently deleted after the chat ends.

Why it matters: Meta launched Incognito Chat on WhatsApp, where inference runs inside the phone's secure enclave, servers keep no logs and sessions are deleted on exit, unlike temporary chats that only clear history.

May 13Wednesday

QbitAI · WeChat

ByteDance Proposes Generative Refinement Networks as a Third Route for Visual Generation

ByteDance’s commercial technology team proposed GRN, a visual generation architecture using HBQ, global refinement, and complexity-aware sampling to address quantization loss, error accumulation, and fixed-step inference; on a 130M model, adaptive sampling reduced inference from 50 steps to an average of 24, while gFID changed from 3.56 to 3.79.

Why it matters: ByteDance's GRN drafts a coarse image then refines it, with HBQ layered binary quantization cutting loss in discrete codes, a concrete third path beyond diffusion and autoregression.

r/LocalLLaMA

The Trillion-Parameter Dilemma: MiMo-V2.5-Pro Open-Sourced at 1.02T Parameters

Xiaomi open-sourced MiMo-V2.5-Pro with 1.02T parameters, 42B active parameters, a 1M context window, and an MIT license; the author ran 125 Claude Code sessions through the API, spending $70.12 for 387,380,436 tokens with a 96.3% cache hit rate.

Why it matters: A concrete cost benchmark for self-hosting versus API calls, built from 125 Claude Code tasks and 387M tokens.

New York Times Chinese

Jensen Huang Gets Last-Minute Invitation to Join Trump’s China Trip

Trump called Jensen Huang on Tuesday morning to invite him to join the China trip; the White House’s Monday list of 16 CEOs did not include him, while Nvidia is still seeking approval to sell AI chips to China.

Why it matters: Nvidia spent a year lobbying both Washington and Beijing on AI chip export licenses, and its CEO now joins the White House group.

New York Times Chinese

China Seeks AI Technology Self-Reliance, Weakening Washington’s Leverage Over Beijing

DeepSeek optimized its latest model for inference on Huawei chips for the first time, while two semiconductor sources said training still relies on Nvidia chips; Huawei says it plans to release a training chip this year, but matching current Nvidia performance will take another year.

Why it matters: DeepSeek's new model runs inference on Huawei chips while training still depends on Nvidia, and Huawei's training chips are a year behind, which shows where China's AI hardware decoupling stands and where it stalls.

r/LocalLLaMA

A real transformer language model running locally on a stock Game Boy Color

maddiedreese ran Andrej Karpathy’s TinyStories-260K on a stock Game Boy Color with INT8 weights, fixed-point math, an MBC5 ROM, bank-switched cartridge storage, and KV cache in cartridge SRAM; the demo uses no phone, PC, Wi‑Fi, link cable, or cloud inference, but output is extremely slow and gibberish.

Why it matters: HKR-H/K/R all pass: a named first-person experiment with concrete model and memory details. Impact stays low-featured because it is a Reddit hardware hack with slow, garbled output, not a usable product or model release.

Bloomberg Technology

CME Plans Computing Power Futures Market

CME Group and Silicon Data plan to create a futures market for computing power; the RSS snippet says Bloomberg Tech discusses the rationale and mechanics, but the post does not disclose contract specifications, launch timing, or pricing methodology.

Why it matters: CME and Silicon Data are working on a compute futures market that would let AI companies lock in GPU rental costs in advance, the way oil buyers hedge against price swings.

AI HOT (Curated Pool)

Claude Opus 4.7 Fast Mode Opens Research Preview

Claude Opus 4.7 Fast Mode is now available as a research preview in the API and Claude Code. The post does not disclose model parameters, pricing, rate limits, or a general availability date.

Why it matters: Anthropic opened a fast mode of Claude Opus 4.7 as a research preview for the API and Claude Code, which may cut waiting time on coding and tool calls.

Hacker News front page

Show HN: Needle Distills Gemini Tool Calling into a 26M Model

Cactus open-sourced Needle, a 26M-parameter tool-calling model that reaches 6,000 tok/s prefill and 1,200 tok/s decode on consumer devices, with MIT-licensed weights released on Hugging Face.

Why it matters: Cactus distilled Gemini's tool calling into a 26M-parameter model that runs locally on consumer devices at 6,000 tokens per second prefill and 1,200 per second generation.

r/LocalLLaMA

Needle: We Distilled Gemini Tool Calling Into a 26M Model

Cactus Compute open-sourced Needle, a 26M-parameter tool-calling model that reaches 6,000 tok/s prefill and 1,200 tok/s decode on consumer devices, using an attention-and-gating architecture with no MLPs.

Why it matters: Cactus Compute distilled Gemini's tool calling into a 26M-parameter model and reports 6,000 tokens per second prefill on consumer devices, a lightweight option for on-device tool use.

TechCrunch · AI

Report: Google and SpaceX in Talks to Put Data Centers Into Orbit

Google and SpaceX are discussing orbital data centers for AI compute, while the post says current space costs remain higher than ground infrastructure; the RSS snippet does not disclose cost gaps, deployment scale, or a timeline.

Why it matters: Google and SpaceX are weighing orbital data centers for AI compute, and TechCrunch's math shows launch and satellite costs still make space far pricier than the ground.

Financial Times · Technology

CME plans to launch futures market for AI computing power

CME plans to launch futures contracts tied to GPU rental prices, allowing traders and companies to bet on or hedge future costs; the RSS snippet does not disclose contract specifications, launch timing, or the reference index.

Why it matters: CME plans futures contracts tied to GPU rental prices, letting compute costs be hedged like oil, a step toward compute as a tradable financial asset.

May 12Tuesday

r/LocalLLaMA

Local LLM Autocomplete and Agentic Coding on a Single 16GB GPU + 64GB RAM

Reddit user grumd runs Qwen2.5-Coder-7B Q6 for autocomplete and Qwen3.6-35B-A3B Q8 for agentic coding on one RTX 5080 with RAM offloading; the post reports about 145k context, 56GB RAM used with other apps open, and Qwen3.6-35B-A3B speed of tg128 at 35.29 tokens/s.

Why it matters: One user runs both a code-completion model and an agentic coding model on a single 16GB GPU with 64GB RAM, and lists the models, quantization and context lengths.

Synced · WeChat

ByteDance Open-Sources DreamLite for Offline Mobile Image Generation and Editing

ByteDance open-sourced DreamLite, a 0.39B-parameter unified diffusion model that generates or edits a 1024×1024 image on an iPhone 17 Pro in about 3 seconds, using 4-step DMD2 distillation and on-device offline inference without cloud dependency.

Why it matters: ByteDance open-sourced DreamLite, a 390M-parameter unified diffusion model that generates or edits a 1024×1024 image locally on an iPhone 17 Pro in about 3 seconds, using DMD2 distillation to cut inference to 4 steps.

AI HOT (Curated Pool)

What Parameter Golf Taught Us About AI-Assisted Research

OpenAI’s Parameter Golf brought together over 1,000 participants and more than 2,000 submissions to test AI-assisted machine learning research, coding agents, model quantization, and model design under strict parameter constraints.

Why it matters: OpenAI ran a Parameter Golf challenge capping model plus code at 16MB and training at 10 minutes, drew over 1,000 entrants and 2,000-plus submissions, and recaps how participants used AI assistants to compress models.

r/LocalLLaMA

Prompt caching for RL training: 7.5x speedup on long-prompt, short-response workloads

The author proposes prompt caching for RL training. On Qwen3.5-4B, it reports a 7.5x speedup with 16k-token prompts and 64-token outputs, and the G=8 example with 1000-token prompts and 100-token responses reduces 8800 processed tokens to 1800 unique tokens.

Why it matters: The author moved prompt caching from inference into RL training, reusing repeated token computation for long-prompt short-response tasks, reaching a 7.5x training speedup on Qwen3.5-4B with 16k prompts and 64-token responses.

r/LocalLLaMA

Computer Build Using Intel Optane Persistent Memory Runs a 1T-Parameter Model at Over 4 Tokens/s

Reddit user APFrisco ran the 1T-parameter Kimi K2.5 Q2_K_XL quant locally at about 4 tokens/s using 768GB Intel Optane PMem, 192GB DDR4 ECC DRAM, and a 12GB RTX 3060 with llama.cpp hybrid GPU/CPU inference.

Why it matters: A Reddit build pairs 768GB of Intel Optane persistent memory with an RTX 3060 to run a 1-trillion-parameter model locally at over 4 tokens/sec, showing llama.cpp hybrid inference on cheap hardware.

May 11Monday

r/LocalLLaMA

ExLlamaV3 Major Updates

ExLlamaV3 added DFlash in v0.0.31, raising Coding throughput from 59.21 t/s to 177.67 t/s; v0.0.32 optimized five models, with Trinity-Nano gaining 72.4% on 6000 Pro², while v0.0.33 adds DFlash model quantization plus bug fixes and efficiency work.

Why it matters: ExLlamaV3 v0.0.31 adds DFlash, lifting coding throughput from 59.21 to 177.67 tokens per second, and v0.0.32 tunes five models, with Trinity-Nano up 72.4% on the 6000 Pro².

Synced · WeChat

ICML 2026: PRISM Brings Efficient Test-Time Scaling to dLLMs

PRISM raises LLaDA-8B-Instruct on GSM8K from 67.58% to 85.30% by combining hierarchical trajectory search, partial remasking, and self-verified feedback, reducing dLLM test-time scaling cost from O(NT) toward O(N+KT) under a final candidate width K.

Why it matters: An ICML 2026 paper proposes PRISM, a three-layer inference-time search using parallel generation, local rollback rewriting and self-scored selection, raising LLaDA-8B-Instruct accuracy on GSM8K from 67.58% to 85.30%.

QbitAI · WeChat

OpenAI backs Cerebras as the Nvidia challenger targets a $35B IPO valuation

Cerebras raised its IPO price range to $150-$160 per share, targeting about a $35 billion valuation at the top end, after OpenAI signed a 750-megawatt AI compute purchase agreement with deliveries through 2028.

Why it matters: After signing a 750 MW compute purchase deal with OpenAI, Cerebras raised its IPO price range to $150–160 per share, about $35 billion at the top end, with deliveries running into 2028.

QbitAI · WeChat

SpaceXAI Takes Shape as Elon Musk Files Trademark Applications

SpaceX filed two SpaceXAI trademark applications covering satellite-based data centers, orbital computing, AI SaaS, cloud storage, telecom hardware, and social networking; the post says xAI became a SpaceX subsidiary through an all-stock deal and cites a $250 billion xAI valuation.

Why it matters: SpaceX filed a SpaceXAI trademark covering satellite data centers, orbital computing and AI software services, and xAI has already become its subsidiary through an all-stock deal.

AI HOT (Curated Pool)

Cerebras IPO reportedly over 20 times oversubscribed, with pricing set to rise nearly 30%

Cerebras received more than 20 times oversubscription for its IPO and plans to raise the share count from 28 million to 30 million while increasing the price range to $150-$160.

Why it matters: Cerebras drew over 20x oversubscription for its IPO, plans to raise its price nearly 30% and sell more shares, targeting up to $4.8 billion, a concrete number for AI chip demand.

AI HOT (Curated Pool)

Local models handle half of daily tasks and respond faster than cloud models

A five-week experiment tested about 1,400 daily work tasks, where local 35B models such as Qwen 3.6 35B handled about 50% and averaged 2.8-second responses, 2.1 times faster than Claude Opus 4.5, while the cloud model still led complex reasoning by about 20%.

Why it matters: A five-week test of a local 35B model across about 1,400 daily requests completed half the tasks at 2.8 seconds average, 2.1x faster than cloud Claude Opus 4.5 at 5.8 seconds.

r/LocalLLaMA

MTP benchmark results: task type determines speculative inference speedups or slowdowns

A Reddit LocalLLaMA user ran 300+ tests on Qwen 3.6 27B MTP quants, finding coding draft acceptance at 79-89% and F16 coding speed up 171%, while Q4_K_M creative writing slowed down 9%.

Why it matters: More than 300 runs on a quantized Qwen 3.6 27B show speculative decoding's speedup depends on the task, not the setup.

May 10Sunday

r/LocalLLaMA

I have DeepSeek V4 Pro at home

Reddit user fairydreaming ran DeepSeek V4 Pro Q4_K_M with a modified llama.cpp CUDA repo on one RTX PRO 6000 Blackwell Max-Q workstation GPU, using an 859GB model file; the shared log reports a 1M context window and 8.6 tokens per second generation speed.

Why it matters: A Reddit user ran the 859GB DeepSeek V4 Pro on a single RTX PRO 6000 workstation card with a modified llama.cpp, a concrete reference for local inference hardware limits.

r/LocalLLaMA

NVIDIA AI Releases Star Elastic: One Checkpoint Contains 30B, 23B, and 12B Reasoning Models

NVIDIA AI released Star Elastic, a single checkpoint that can zero-shot slice 30B, 23B, and 12B reasoning models in BF16, FP8, and NVFP4; when the 23B submodel handles thinking and the 30B model handles final answers, reported accuracy rises 16% and latency drops 1.9× on AIME-2025, GPQA, LiveCodeBench v5, and MMLU-Pro.

Why it matters: NVIDIA's Star Elastic slices 30B, 23B and 12B reasoning models from one checkpoint with no extra training, the 23B thinking and the 30B answering.

r/LocalLLaMA

BeeLlama.cpp: DFlash and TurboQuant with reasoning and vision support

Anbeeld released BeeLlama.cpp, a llama.cpp fork that runs Qwen 3.6 27B Q5 with 200k context and vision on a single RTX 3090 or 4090; the title claims 2–3x faster than baseline and a 135 tps peak.

Why it matters: Anbeeld's llama.cpp fork BeeLlama.cpp adds DFlash memory-saving attention and TurboQuant quantization, letting Qwen 3.6 27B run 200k context on a single 3090 at 2 to 3 times baseline speed.

May 9Saturday

AI HOT (Curated Pool)

Redis founder uses a C inference engine to run a large model on a personal computer

Antirez open-sourced ds4, a native inference engine for DeepSeek V4 Flash that uses a few thousand lines of C to run a 1M-context model on a 128GB MacBook Pro at a reported 27 tok/s.

Why it matters: Antirez's ds4 runs DeepSeek V4 Flash's million-token context on a 128GB MacBook Pro in a few thousand lines of C, with 2-bit MoE quantization and KV cache spilled to SSD.

r/LocalLLaMA

80 tok/sec and 128K context on 12GB VRAM with Qwen3.6 35B A3B and llama.cpp MTP

Reddit user janvitos ran Qwen3.6-35B-A3B-MTP-GGUF with a llama.cpp MTP PR on an RTX 4070 Super. The posted benchmark shows 69.2-81.9 tok/s, 0.694-0.947 draft acceptance, 131072 context, and a -fitt 1536 setting that reserves 1536 MB for the draft model and KV cache.

Why it matters: Reddit user janvitos runs Qwen3.6 35B A3B on a 12GB RTX 4070 Super and posts measured settings for 69.2 to 81.9 tok/s at 128K context.

AI HOT (Curated Pool)

Baidu releases ERNIE 5.1 with compressed parameters and training cost

Baidu released ERNIE 5.1 with total parameters reduced to about one third of the original scale, active parameters to about one half, and pretraining cost to about 6% of same-scale models; the model is available on the ERNIE platform and Baidu AI Studio.

Why it matters: Baidu's ERNIE 5.1 cuts total parameters to about a third and pretraining cost to 6% of comparable models, with head-to-head numbers against DeepSeek-V4-Pro and Gemini 3.1 Pro.

Xinzhiyuan · WeChat

NVIDIA, AMD, and Intel Back RadixArk’s $100M Seed Round

RadixArk announced a $100 million seed round on May 5 at a $400 million post-money valuation, led by Accel and co-led by Spark Capital, with participation from NVentures, AMD, MediaTek, Databricks, and other investors tied to AI infrastructure.

Why it matters: RadixArk closed a $100 million seed at a $400 million post-money valuation, with Nvidia, AMD and Intel all on the cap table.

AI HOT (Curated Pool)

DeepSeek Raises $7 Billion at Record Scale, Founder Personally Invests $3 Billion

DeepSeek is raising up to $7 billion at a $50 billion valuation, with founder Liang Wenfeng personally contributing $3 billion, or 40% of the round, while the company says the funding will target large-scale compute, V4.1 model releases, enterprise products, and a path toward positive revenue.

Why it matters: DeepSeek is raising $7 billion at a $50 billion valuation, with founder Liang Wenfeng putting in $3 billion of his own money and keeping 90% of the shares.

r/LocalLLaMA

MTP + TurboQuant Running: Qwen3.6-27B Hits 80+ t/s on a Single RTX 4090

indrasmirror ran Qwen3.6-27B-Heretic-v2 on a single RTX 4090 with 262K context, TBQ4_0 KV cache, and MTP draft 3, improving throughput from about 43 t/s to 80-87 t/s with roughly 73% MTP draft acceptance.

Why it matters: indrasmirror pushed Qwen3.6-27B to 262K context on one RTX 4090, using MTP speculative decoding to lift generation from about 43 t/s to 80-87 t/s at a 73% draft acceptance rate.

The Verge · AI

All the Latest Updates on AI Data Centers

The Verge tracks AI data center disputes with specific updates: 43% of Americans blame data centers for rising power bills, a 40,000-acre Utah project won approval despite local opposition, and Anthropic says it will invest $50 billion in US AI data centers.

Why it matters: The Verge rounds up conflicts over AI data centers worldwide, including a poll where 43% of Americans blame data centers for higher power bills and a 40,000-acre Utah project approved despite community opposition.

Bloomberg Technology

Anthropic Inks $1.8 Billion Computing Deal With Akamai

Anthropic signed a $1.8 billion computing deal with Akamai to meet rising demand for its AI software; the post does not disclose capacity, contract duration, or deployment regions.

Why it matters: Anthropic signed an $1.8 billion computing deal with Akamai to meet growing demand for its AI software, putting Akamai into the role of AI compute supplier.

AI HOT (Curated Pool)

EMO: Expert Mixture Models for Emergent Modular Pretraining

AllenAI introduced EMO, a mixture-of-experts model with 14B total parameters and 1B active parameters, trained on 1 trillion tokens and able to use only 12.5% of its experts for specific tasks while retaining near-full-model performance.

Why it matters: EMO lets experts emerge from 1 trillion tokens instead of assigning them by subject, and activates only 12.5% of them at run time while nearing full-model quality.

May 8Friday

r/LocalLLaMA

Gemma 4 26B Hits 600 Tok/s on One RTX 5090

chain-77 benchmarked Gemma 4 26B with vLLM 0.19.2rc1, and DFlash raised output throughput on one RTX 5090 from 228 tok/s to 578 tok/s under 256 input tokens, 1024 output tokens, concurrency 1, and num_speculative_tokens=13.

Why it matters: chain-77 ran Gemma 4 26B on a single RTX 5090 with vLLM 0.19.2rc1, and DFlash speculative decoding lifted output throughput from 228 to 578 tokens per second, with reproducible concurrency and draft-token settings.

Synced · WeChat

SGLang Team Launches RadixArk With $100M Seed Round

RadixArk announced a $100 million seed round on May 5 at a $400 million post-money valuation, while its SGLang inference project has 27K+ GitHub stars and deployments across 400K+ GPUs.

Why it matters: The SGLang team's new company raised a $100M seed at a $400M post-money valuation, with 27,000 GitHub stars and 400,000+ GPUs running its open inference framework.

Xinzhiyuan · WeChat

Token-Level Length Control: 3B Model Beats GPT 5.4 and Claude

UC Santa Barbara and Apple researchers introduced LenVM, which models remaining generation length as a token-level value function; Qwen2.5-3B with a 1.5B LenVM scored 62.6 on LIFEBench length control, above GPT-5.4 at 37.4 and Claude-Opus-4-6 at 35.5.

Why it matters: Modeling remaining generation length as a token-level value function lets a 3B model beat GPT-5.4 and Claude on length control.