Skip to content

#数据/训练

6 today

May 15Friday

r/LocalLLaMA

MOOSE-Star (ICML 2026): 7B Model and 108K-Paper Dataset for Scientific Hypothesis Discovery

MiroMind researchers released the MOOSE-Star collection with three 7B models and TOMATO-Star, a dataset of 108,717 NCBI papers. MS-IR-7B reaches 54.37% inspiration-retrieval accuracy, uses DeepSeek-R1-Distill-Qwen-7B as its base, runs at about 14GB fp16, and supports llama.cpp, vLLM, and SGLang.

Why it matters: MiroMind released three 7B MOOSE-Star models and the 108,717-paper TOMATO-Star dataset, with weights and data available for reuse in scientific hypothesis discovery.

May 14Thursday

r/LocalLLaMA

Automated AI researcher running locally with llama.cpp

Hugging Face’s ml-intern added local-model support through llama.cpp and ollama; the post says Qwen3.6-35B-A3B can orchestrate CPU/GPU sandboxes and Hub jobs to run an end-to-end SFT workflow.

Why it matters: A community setup wires HuggingFace's ml-intern to llama.cpp and ollama, running a full SFT loop on local Qwen3.6-35B-A3B, a path others can copy.

Xinzhiyuan · WeChat

Yuandong Tian and Seven Co-Founders Launch Recursive Superintelligence at $4.65B Valuation

Recursive Superintelligence, founded by Yuandong Tian and seven other AI researchers, has a 25-person team, $650 million in funding, and a $4.65 billion valuation, with a stated goal to automate evaluation, data filtering, training, post-training, and research-direction selection.

Why it matters: A 25-person team raising $650M at a $4.65B valuation to hand the whole research loop, from evaluation to topic choice, over to AI.

Synced · WeChat

China in Focus: PsiBot Uses 100,000 Hours of Human Data for Embodied AI

PsiBot says it uses 100,000 hours of human operation data to train robot policies, with the W0 world model acting only as a training-time transfer module while deployment runs R2 alone.

Why it matters: Lingchu claims 100,000 hours of human operation data trained its robot policy, with the W0 world model used only in training; the data collection and task results can't be confirmed.

Synced · WeChat

ACL 2026: Alibaba DAMO I²B-LPO Improves RLVR Exploration

Alibaba DAMO Academy introduced I²B-LPO, an RLVR post-training framework that branches rollouts at high-entropy nodes and filters them with an information-bottleneck self-reward, reporting up to 5.3% accuracy gains and 7.4% semantic-diversity gains on math benchmarks using Qwen2.5-7B and Qwen3-14B.

Why it matters: Alibaba DAMO Academy's I²B-LPO branches sampling where a model hesitates on a problem and uses an information bottleneck self-reward to pick out different solutions, easing the convergence of reasoning paths in RL fine-tuning.

May 13Wednesday

AI HOT (Curated Pool)

SenseNova-U1 Technical Report Released: Guide to Native Multimodal Model Building

SenseTime released the SenseNova-U1 technical report, covering six-stage training, RL post-training, and distillation; the open-source SenseNova-U1-A3B-MoT uses an MoE architecture and activates only 3 billion parameters.

Why it matters: SenseTime's SenseNova-U1 technical report lays out the six-step pipeline for a native multimodal model, from unified modeling to RL post-training and distillation, and open-sources a 3B-active-parameter MoE version.

Xinzhiyuan · WeChat

Tsinghua-affiliated team open-sources MiniCPM-V 4.6, a 1.3B model tunable on one RTX 4090

ModelBest, Tsinghua University, and OpenBMB open-sourced MiniCPM-V 4.6, a 1.3B multimodal model that supports full fine-tuning on one RTX 4090 and offers 4x/16x visual token compression for accuracy or speed trade-offs.

Why it matters: ModelBest, Tsinghua and OpenBMB open-source MiniCPM-V 4.6 at 1.3B parameters, small enough to fully fine-tune on one RTX 4090, with 4x and 16x visual token compression modes.

May 12Tuesday

AI HOT (Curated Pool)

How Open Model Ecosystems Compound

China’s open AI model community forms a self-reinforcing loop, with domestic open model downloads rising by more than 200% quarter over quarter.

Why it matters: The author argues trial-and-error compute can be 80% of frontier model costs, and Chinese labs share what failed in technical reports, so an open ecosystem compounds cheaper future R&D.

Google DeepMind

Google DeepMind publishes Co-Scientist multi-agent research system

Google DeepMind published Co-Scientist research in Nature, introducing a Gemini-based multi-agent AI system that iteratively generates, debates and evolves new hypotheses for complex scientific problems.

Why it matters: Google DeepMind publishes Co-Scientist in Nature, a system of Gemini-driven specialized agents that generate, debate and evolve hypotheses, already applied to topics such as antimicrobial resistance.

QbitAI · WeChat

Shanghai AI Lab Study: SFT Generalizes Under Three Conditions

Shanghai AI Lab, Shanghai Jiao Tong University, and USTC tested Long-CoT SFT on Qwen3-14B-Base and found that cross-domain performance recovered and improved after 8 epochs, with generalization conditioned on optimization depth, data quality and structure, and base-model capability.

Why it matters: Shanghai AI Lab and other teams fine-tuned Qwen3-14B-Base on long chains of thought, observed cross-domain performance dropping first and then rising, and summarized three conditions under which the generalization holds.

r/LocalLLaMA

Prompt caching for RL training: 7.5x speedup on long-prompt, short-response workloads

The author proposes prompt caching for RL training. On Qwen3.5-4B, it reports a 7.5x speedup with 16k-token prompts and 64-token outputs, and the G=8 example with 1000-token prompts and 100-token responses reduces 8800 processed tokens to 1800 unique tokens.

Why it matters: The author moved prompt caching from inference into RL training, reusing repeated token computation for long-prompt short-response tasks, reaching a 7.5x training speedup on Qwen3.5-4B with 16k prompts and 64-token responses.

May 10Sunday

Synced · WeChat

Turing Award Winner Sutton Uses a 1967 Formula to Improve Streaming Reinforcement Learning

Richard Sutton and coauthors proposed Intentional Updates, which derive the step size from the desired output change; Intentional AC approached SAC on MuJoCo under batch=1 streaming training without replay, while each update used about 1/140 of SAC’s FLOPs.

Why it matters: Sutton's group derives a learning-rate rule from the 1967 Kalman filter, cutting drift in streaming RL that learns one sample at a time with no replay buffer.

Xinzhiyuan · WeChat

Next-ToBE Targets Short-Sighted Next-Token Prediction in LLMs at ICLR 2026

East China Normal University and Fudan University researchers proposed Next-ToBE, a training objective that keeps standard autoregressive inference while adding a soft target over future-token windows, and the article reports the method ranked best in 35 of 36 experiments across Qwen2.5-Math-1.5B, Qwen2.5-Math-7B, and Llama3.1-8B-Instruct.

Why it matters: ECNU and Fudan researchers propose Next-ToBE, which trains against soft targets from the next few words while still generating token by token, winning 35 of 36 tests.

May 9Saturday

Xinzhiyuan · WeChat

NVIDIA, AMD, and Intel Back RadixArk’s $100M Seed Round

RadixArk announced a $100 million seed round on May 5 at a $400 million post-money valuation, led by Accel and co-led by Spark Capital, with participation from NVentures, AMD, MediaTek, Databricks, and other investors tied to AI infrastructure.

Why it matters: RadixArk closed a $100 million seed at a $400 million post-money valuation, with Nvidia, AMD and Intel all on the cap table.

May 8Friday

Synced · WeChat

SGLang Team Launches RadixArk With $100M Seed Round

RadixArk announced a $100 million seed round on May 5 at a $400 million post-money valuation, while its SGLang inference project has 27K+ GitHub stars and deployments across 400K+ GPUs.

Why it matters: The SGLang team's new company raised a $100M seed at a $400M post-money valuation, with 27,000 GitHub stars and 400,000+ GPUs running its open inference framework.

QbitAI · WeChat

All Labs Watch ByteDance, Everyone Praises DeepSeek: A U.S. Researcher’s 36-Hour China AI Trip

Ai2 researcher Nathan Lambert visited Zhipu, Moonshot AI, Tsinghua, Meituan, Xiaomi, and 01.AI within 36 hours, and said Chinese labs closely watch ByteDance and respect DeepSeek, while student participation in core work, open source habits, and in-house control of the technical stack mark key differences.

Why it matters: Ai2 researcher Nathan Lambert spent 36 hours visiting Zhipu, Moonshot AI, Tsinghua, Meituan, Xiaomi and 01.AI, and recorded how Chinese labs watch ByteDance closely and respect DeepSeek.

May 6Wednesday

QbitAI · WeChat

Claude Team Tests New Training Method on Qwen

Anthropic proposed MSM training between pretraining and alignment fine-tuning. Tests on Qwen2.5-32B and Qwen3-32B cut misalignment from 68% and 54% to 5% and 7%. The key point is MSM complements AFT rather than replacing it.

Why it matters: Anthropic tested an MSM stage between pretraining and alignment fine-tuning on Qwen2.5-32B and Qwen3-32B, cutting misalignment from 68% and 54% to 5% and 7%.

Financial Times · Technology

Meta and Zuckerberg Sued by Publishers Over ‘Massive’ Copyright Infringement

Five major publishing groups sued Meta and Zuckerberg over copyrighted works allegedly used to train Llama AI models. The RSS snippet does not disclose work counts, damages, court venue, or training-data mechanism.

Why it matters: Five publishing groups sue Meta and Zuckerberg, claiming Llama models were trained on copyrighted works without authorization, putting training-corpus licensing on the table.

May 3Sunday

r/LocalLLaMA

Built a C++17 transformer from scratch with 0.83M params and CPU training

Reddit user Suspicious_Gap1121 released Quadtrix.cpp, a C++17 GPT-style model with 0.83M parameters. It uses 4 layers, 4 heads, 200d width, and a 128-character context; one CPU core trained on 31.4M characters for 76.2 minutes to 1.6371 nats val loss. The key detail is handwritten backprop for LayerNorm, attention, Q/K/V, dropout, and AdamW without PyTorch, BLAS, or autograd.

Why it matters: A from-scratch C++17 transformer with hand-written backprop shows what a small GPT actually costs to train without PyTorch or BLAS.

May 2Saturday

MIT Technology Review · AI

Musk v. Altman week 1: Musk says xAI distills OpenAI models

Elon Musk testified in week 1 of Musk v. Altman, saying he gave OpenAI $38 million in funding. He asks the court to remove Sam Altman and Greg Brockman and unwind OpenAI’s for-profit restructuring. The sharp detail: Musk said xAI partly distills OpenAI models, while OpenAI previously accused DeepSeek of similar conduct.

Why it matters: Musk admitted in court that xAI distilled OpenAI's models, while OpenAI had previously leveled the same accusation at DeepSeek, putting the boundaries of distillation into judicial review.

May 1Friday

Synced · WeChat

The Evolution of RL: From PPO to MaxRL in LLM Reasoning Training

Jiqizhixin translated Alexander Weers' article on RL algorithms for LLM reasoning from 2024 to 2026. It covers REINFORCE, PPO, GRPO, RLOO, Dr. GRPO, DAPO, CISPO, MaxRL, DPPO, and ScaleRL, comparing critic removal, clipping, normalization, and pass@k goals. The key signal is mechanism choice, not algorithm names.

Why it matters: The article traces reasoning RL algorithm evolution for large models from 2024 to 2026, PPO to GRPO, RLOO and MaxRL, focusing on how each cuts training cost by dropping the value model.

The Verge · AI

Elon Musk confirms xAI used OpenAI’s models to train Grok

Elon Musk testified Thursday in a California federal court that xAI used OpenAI models to improve Grok. The mechanism described is model distillation: a larger teacher model transfers knowledge to a smaller student model. The post does not disclose which OpenAI models, data volume, or training runs.

Why it matters: Musk testified in California federal court that xAI partly used OpenAI models to improve Grok through distillation, laying out how the two firms' models relate.

TechCrunch · AI

Elon Musk testifies that xAI trained Grok on OpenAI models

Elon Musk testified that xAI trained Grok on OpenAI models. The post only says distillation concerns frontier labs; it does not disclose scale, model versions, or case context.

Why it matters: Musk's testimony in California federal court that xAI trained Grok partly on OpenAI models moves distillation from rumor into the court record.

Apr 30Thursday

MIT Technology Review · AI

Goodfire releases Silico, a mechanistic interpretability tool for debugging LLMs

Goodfire released Silico, letting engineers inspect and adjust LLM parameters during training. It maps neurons and pathways; one Qwen 3 neuron triggered trolley-problem-style outputs. Pricing is case-by-case, and the post does not disclose rates.

Why it matters: Silico lets engineers inspect and adjust model parameters during training and map neurons and circuits.

Synced · WeChat

After Generalist, Jianlan Luo’s Team Releases LWD for Embodied AI Training

Jianlan Luo’s team and Agibot released LWD, tested on 16 Agibot G1 robots in real settings. LWD Online scored 0.95 across 8 tasks and 0.91 on long-horizon tasks. Its offline-to-online RL uses failures as data; failed trajectories were 34.8% of a 652.5-hour pool.

Why it matters: Luo Jianlan's team and AgiBot built the LWD training framework that learns from failure data, and on 16 real Agibot G1 robots it hit 0.95 average success across 8 tasks.

QbitAI · WeChat

OpenAI Explains Why GPT-5.5 Keeps Saying “Goblin”

OpenAI says GPT-5.5’s “goblin” habit came from Nerd-persona rewards and training transfer. After GPT-5.1, ChatGPT’s “goblin” use rose 175%; Nerd replies were 2.5% of all replies but 66.7% of goblin mentions. The key issue is reward bias spreading through RL, rollouts, and SFT.

Why it matters: OpenAI traced why GPT-5.5 kept saying goblin, showing the reward signal behind a nerd persona spread through reinforcement learning, lifting use of the word 175% even though such replies were only 2.5% of output.

Apr 28Tuesday

Synced · WeChat

ACL 2026: Huawei Taylor Lab Proposes SHAPE, Adding a Reasoning Tax to LLM Inference

Huawei Taylor Lab, Peking University, and Shanghai University of Finance and Economics proposed SHAPE, accepted by ACL 2026, with about 3% average accuracy gain. It uses entropy segmentation, short rollouts for potential estimation, dynamic length discounts, and token-level credit assignment, cutting token use by about 30%. The key mechanism is a reasoning tax: long high-potential late-stage segments are penalized to reduce verbose confirmation loops.

Why it matters: Huawei's Taylor Lab and others propose SHAPE, which segments by entropy and probes short spans to estimate potential, then discounts long late sections so models answer math problems more accurately and more briefly.

Synced · WeChat

Open-source medical video understanding system uAI-NEXUS-MedVLM released

United Imaging Intelligence released uAI-NEXUS-MedVLM for medical video understanding, with a CVPR 2026 paper. MedVidBench has 532k video-instruction pairs across 8 medical sources and 8 tasks. Qwen2.5-VL-7B SFT reached 89.4% CVS accuracy; GPT-5.4 scored 16.4%.

Why it matters: United Imaging Intelligence open-sourced uAI-NEXUS-MedVLM, a large model for ultrasound, endoscopy and other medical video, plus MedVidBench, an evaluation set of 532,000 video instruction pairs.

X · @op7418

Xiaomi open-sources the MiMo-V2.5 model series

Xiaomi open-sourced the MiMo-V2.5 model series under the MIT license for commercial use, retraining, and fine-tuning. It also launched Orbit 100T Token, offering approved AI builders up to 1.6B credits worth 659 yuan. Agent framework teams can apply for free MiMo token access; the post does not disclose model size or benchmark results.

Why it matters: Xiaomi open-sourced its MiMo-V2.5 series under MIT, allowing commercial use, retraining and fine-tuning without extra permission, with an Orbit 100T Token program offering up to 1.6 billion credits.

Apr 26Sunday

Hacker News front page

DeepSeek-V4 on Day 0: From Fast Inference to Verified RL with SGLang and Miles

SGLang and Miles added day-0 inference and RL support for DeepSeek-V4, covering 1.6T Pro and 284B Flash. The post cites a 1M-token context, FP4 MoE expert weights, 128-token SWA, and 4:1 or 128:1 KV compression. The key systems detail is ShadowRadix coherence across three KV pools and two compression-state pools.

Why it matters: SGLang and Miles shipped inference and RL training support on DeepSeek-V4's launch day, covering the 1.6T-parameter Pro and 284B-parameter Flash versions, and disclosed deployment details including hybrid sparse attention and FP4 expert weights.

Apr 22Wednesday

r/LocalLLaMA

ServiceNow-AI/SuperApriel-15B-Instruct · Hugging Face

ServiceNow released SuperApriel-15B-Instruct, a single-checkpoint 15B model with 8 deployment presets spanning 1.0× to 10.7× decode throughput at 32K sequence length. It has 48 decoder layers with 4 mixer variants per layer and up to 262K context positions depending on runtime; the key point is that speed-quality tradeoffs and speculative decoding are exposed from the same weights.

Why it matters: ServiceNow released SuperApriel-15B-Instruct, a 15B instruction model whose single weight set offers eight speed tiers, up to 10x faster decode throughput, so latency and quality trade off inside one model.

Apr 20Monday

r/LocalLLaMA

Training LoRA adapters for Apple's on-device 3B model on a free Colab T4 and a Mac

The author built a QLoRA pipeline for Apple’s on-device 3B model, cutting training needs from about 24GB to about 1GB RAM and 5GB GPU, enough for a free Colab T4 or a 24GB Mac. The post says A100 LoRA, T4 QLoRA, and Mac QLoRA adapters perform about the same, raising accuracy from about 40% to 75%, or 86% with retrieval; it also reports a confirmed Apple bug that writes a hidden ~160MB cache copy per CLI call, reaching 269GB over ~300 runs.

Why it matters: A walkthrough of fine-tuning Apple's on-device 3B model with QLoRA, cutting memory needs to 1GB RAM plus 5GB VRAM so a free Colab T4 or a 24GB Mac can run it.

r/LocalLLaMA

Actually put Gemma 4 26B to work on something real: extract trading signals from 2,400 earnings calls

A Reddit user fine-tuned Gemma 4 26B on 800 labeled earnings-call transcripts and ran inference on 2,400 transcripts over 3 years on one RTX 4090 in about 14 hours. On 600 out-of-sample transcripts, one signal linked vaguer CFO guidance to about 1.8% sector-relative underperformance over 5 days with IC 0.04. A stronger signal showed 0.85 correlation with sector returns after checks and was discarded as a ghost factor; the key point is factor sanity checks, not the profit claim.

Why it matters: A Reddit user fine-tuned Gemma 4 26B on 800 labeled earnings calls and ran 2,400 records on one RTX 4090, finding one signal where vaguer CFO guidance tracked about 1.8% underperformance.

Apr 12Sunday

最佳拍档 (BestPartners)

Breaking RLHF scaling bottlenecks: DeepMind raises data efficiency 10x with information-directed exploration

A Google DeepMind team reports that online RLHF plus information-directed exploration on Gemma 9B reaches about 55% win rate with under 20k preference labels, versus about 200k for offline RLHF. The post describes four algorithms—offline, periodic, online, and information-directed exploration; online training uses batches of 64 prompts and 16 sampled responses per prompt, while the ENN head adds under 5% parameters. The key point is methodological, not that RLHF failed; the post also says results use Gemini 1.5 Pro simulated feedback, and the 1000x gain is an extrapolation toward 1M labels.

Why it matters: DeepMind compares four RLHF algorithms on Gemma 9B: online RLHF with information-directed exploration reaches the offline method's result with under 20,000 preference labels versus about 200,000, a tenfold gap in data efficiency.

Apr 8Wednesday

QbitAI · WeChat

Free open-source 2B Chinese speech model reproduces Mangzhuang Ren with high-speed tonguetwisters

ModelBest, OpenBMB, and Tsinghua University released VoxCPM 2, a 2B open speech model that supports 9 Chinese dialects, 30 foreign languages, and 48kHz audio. The post says generation often finishes within 1 second, recommends reference audio of at least 5 seconds, and supports denoising, LoRA, and full fine-tuning; the key detail is its tokenizer-free diffusion autoregressive continuous representation design.

Why it matters: ModelBest and Tsinghua's OpenBMB released a 2B-parameter speech model that clones a voice from a 5-second reference clip.

Mar 18Wednesday

MIT Technology Review · AI

The Download: The Pentagon's new AI plans, and next-gen nuclear reactors

The Pentagon plans to create secure environments so generative AI companies can train military-specific models on classified data. The post says Anthropic Claude is already used in classified settings, including analyzing targets in Iran; training on surveillance and battlefield reports would embed sensitive intelligence in the models. It also flags waste challenges from next-gen nuclear reactors, but the post does not disclose reactor designs or disposal parameters.

Why it matters: The Pentagon wants a secure setup where generative AI firms train military-specific models on classified data, and Anthropic's Claude has already been used on classified targets in Iran.

MIT Technology Review · AI

The Pentagon plans to let AI companies train models on classified data, defense official says

The Pentagon is discussing secure facilities where AI firms can train military-specific models on classified data. The post says training would follow tests on nonclassified data; the DoD keeps data ownership, and company staff would access it only rarely with clearance. The key issue is leakage: one shared model may resurface classified information across groups with different access levels.

Why it matters: The Pentagon plans secure environments for OpenAI, xAI and others to train military models on classified data owned by the Defense Department, with company staff access limited to rare cases.

Mistral AI

Mistral AI launches Forge, an enterprise model-training system

Mistral AI launched Forge, a system for enterprises to build frontier-class AI models on their own proprietary knowledge. It covers pre-training, post-training and reinforcement learning, supports dense and MoE architectures, and handles multimodal input. Models can be trained and governed on in-house infrastructure. Mistral AI has worked with ASML, Ericsson, the European Space Agency and Singapore's DSO National Laboratories to train models on their proprietary data.

Why it matters: Mistral's Forge trains models on a company's own data, and ASML and Ericsson are already using it.

Mar 17Tuesday

NVIDIA Blog

GTC spotlights NVIDIA RTX PCs and DGX Spark running latest open models and AI agents locally

NVIDIA used GTC to showcase RTX PCs and DGX Spark for running local AI agents, and announced Nemotron 3 Nano 4B, Nemotron 3 Super 120B, and the open source NemoClaw stack. The post says DGX Spark has 128GB unified memory for models above 120B parameters; Nemotron 3 Super scored 85.6% on PinchBench, and Qwen 3.5 supports a 262,000-token context window. The key signal is local inference for privacy and zero token cost, while the full “latest open models” lineup and pricing are not disclosed in the post.

Why it matters: At GTC, NVIDIA showed AI assistants running locally on RTX PCs and DGX Spark, whose 128GB unified memory holds models above 120B parameters, with privacy and token cost as the two payoffs.

Mar 12Thursday

NVIDIA Blog

NVIDIA Nemotron 3 Super delivers 5x higher throughput for agentic AI

NVIDIA launched Nemotron 3 Super, a 120B open model with 12B active parameters, and says it delivers up to 5x higher throughput for agentic AI. It has a 1M-token context window and uses hybrid MoE, latent MoE, and multi-token prediction; the post says Blackwell NVFP4 gives up to 4x faster inference than Hopper FP8, with over 10T training tokens disclosed. What matters is that NVIDIA is releasing open weights, training recipes, and RL environments for reproduction and fine-tuning.

Why it matters: NVIDIA open-sourced Nemotron 3 Super: 120B total parameters, 12B active per inference, 1M token context, and 5x throughput on multi-step agent tasks.