Research & technical reports

All topics

Official research posts and technical reports from labs: architectures, training methods, measurement and safety research. Purely academic papers are not collected.

262 picksRelated topicsReasoning Safety & alignment Data & training

Latest picks

21–40 of 262

Jun 5Friday

Synced · WeChatJun 5, 12:07

Do Models Need Sleep? CMU Paper Lets LLMs Consolidate Memory During “Sleep”

CMU and the University of Maryland propose Language Models Need Sleep: when each L-token context window fills, the model runs N offline recurrent forward passes and updates SSM fast weights before evicting the KV cache. On GSM-Infinite, Jet-Nemotron 2B with 6 sleep loops improves 6-step arithmetic accuracy from 0.742 to 0.812.

Why it matters: HKR-H/K/R all pass: the hook is strong, and the post gives a testable mechanism plus Jet-Nemotron 2B numbers. It is still a single early paper, not an industry-level release, so it stays just above the featured threshold.

Take

The useful claim here is not that “models need sleep”; it is that long context is a bad proxy for memory. The mechanism is concrete: every L tokens, before evicting KV cache, the model runs N offline recurrent forward passes and updates SSM fast weights. On GSM-Infinite, Jet-Nemotron 2B with 6 sleep loops moves 6-step accuracy from 0.742 to 0.812. The 8-step case only rises from 0.351 to 0.388, so the headline gain is selective.

I’m cautious on the framing. It preserves awake-time prediction latency, but it pays N extra passes during consolidation, and training also gets deeper backprop. This sits in the same escape-from-KV-cache family as Mamba-style memory and recurrent compression work, but with a cleaner “offline internalization” hook. Synthetic tasks and 1.4B/2B models are a start; production agent memory is a much harsher test.

Hacker News front pageJun 5, 07:11

Do Transformers Need Three Projections? Systematic Study of QKV Variants

Ali Kayyam and coauthors evaluate three QKV projection-sharing variants across synthetic, vision, and language-modeling settings, including 300M and 1.2B parameter models trained on 10B tokens; Q-K=V halves the KV cache with a 3.1% perplexity degradation, while Q-K=V plus MQA reduces cache use by 96.9%.

Why it matters: HKR-H/K/R all pass: the title challenges a core architecture default, the paper gives testable 300M/1.2B and 10B-token results, and KV-cache cuts map to inference cost. It remains an arXiv architecture study, so 78–84 fits.

Hacker News front pageJun 5, 00:20

When AI Builds Itself: Our Progress Toward Recursive Self-Improvement

Anthropic published a post on recursive self-improvement under the title “When AI Builds Itself,” while the RSS body only discloses 95 Hacker News points and 106 comments, with no experimental setup, model details, or timeline disclosed.

Why it matters: HKR-H and HKR-R pass: an Anthropic post on recursive self-improvement has a strong hook and practitioner resonance. HKR-K fails because the feed discloses no mechanism or model details.

Jun 4Thursday

QbitAI · WeChatJun 4, 16:26

Beyond TurboQuant: Together AI Brings 2-bit KV Cache to Real Serving

Together AI, the University of Sydney, and UIUC introduced OSCAR, a 2-bit KV Cache quantization method that uses about 2.28 effective bits per KV element and scores 71.86 on Qwen3-4B-Thinking, 40.1 points above TurboQuant.

Why it matters: HKR-H/K/R all pass: OSCAR links 2-bit KV cache to serving and provides concrete scores. The topic is still low-level inference optimization, so it lands in featured rather than same-day must-write.

QbitAI · WeChatJun 4, 16:26

CVPR 2026: NVIDIA, Tesla, and Waymo hear Xpeng present physical AI

Xpeng presented its world-model stack at CVPR 2026, covering X-World, X-Foresight, and X-Cache; the article says X-Cache cuts about 70% of repeated computation, the second-generation VLA used over 4 trillion training tokens, and the in-car stack reduced inference latency to 80 ms.

Why it matters: HKR-H comes from the CVPR stage contrast, HKR-K has X-Cache, 4T+ tokens, and 80 ms latency, and HKR-R fits autonomy competition. It is still a company tech showcase, below the 85 must-write band.

Take

Xpeng’s physical-AI story has substance, but the article oversells the stagecraft; the engineering numbers carry the piece. Its second-generation VLA used over 4 trillion training tokens, X-Cache cuts roughly 70% of repeated compute, and the in-car stack moves from 800 ms on a general chip plus open model to 80 ms on second-gen VLA plus Xpeng’s Turing chip. That is a stronger moat claim than “NVIDIA, Tesla, and Waymo listened.”

I’m less sold on the 1010% single-GPU training-efficiency gain and 4360% single-task gain, because the baseline task and measurement method are not given. “First tier” also needs a comparable yardstick. Compared with Tesla’s FSD narrative, Xpeng is exposing more of the stack—world model, VLA, cache, chip co-design—which is useful for practitioners. It still lacks public closed-loop reliability metrics for the L4 jump.

Jun 3Wednesday

NVIDIA BlogJun 3, 23:00

NVIDIA Research Presents Grasping, Autonomous Driving and Agent Training Work at CVPR

NVIDIA Research presented three physical AI papers at CVPR: GraspGen-X was trained on 2 billion simulated grasps, LCDrive cuts reasoning tokens by about half versus text-based reasoning, and NitroGen trains embodied agents across more than 1,000 games and 40,000 hours of interaction.

Why it matters: HKR-H/K/R all pass: NVIDIA’s CVPR bundle gives concrete mechanisms and scale numbers. It stays in the low 78–84 band because it is a vendor research roundup, not a major model or product launch.

Take

NVIDIA’s CVPR bundle lands because it turns robotics problems into simulation-scale data problems again. GraspGen-X uses 2 billion simulated grasps across thousands of objects and synthetic grippers. LCDrive cuts reasoning tokens by about half versus text reasoning. NitroGen trains across 1,000-plus games and 40,000 hours of interaction. Those numbers all point to the same play: make generalization a distribution problem, then own the factory that produces the distribution.

I don’t fully buy “the first foundation model for zero-shot grasping”; robotics has recycled that promise for years. The harder part is NVIDIA’s stack around it: Isaac, GR00T, cuRoboV2, Alpamayo, and embedded inference targets. That gives the papers a path into tools, not just PDFs. The gap is also obvious: the post gives no real-robot success rate, no vehicle latency, and no clean cross-game generalization benchmark.

Synced · WeChatJun 3, 15:30

Understanding SFT Mechanisms in LLMs: Resolving Practice Disputes and Avoiding Wasted Compute

Junpeng Zhang and coauthors argue that SFT on highly homogeneous data has an effective window of only hundreds to about 1,000 training steps, and their interaction-based warning signal detects overfitting before loss gaps appear, saving roughly 30%–50% of training compute.

Why it matters: HKR-H/K/R all pass: the paper gives testable SFT windows, earlier overfitting warnings, and 30%-50% compute savings. It is strong research, not a major model or product release, so it stays below 85.

Synced · WeChatJun 3, 15:30

RSS 2026: Ant Lingbo Proposes Autoregressive Causal World Model for Robot Manipulation with 50 Demos

Ant Lingbo and HKUST introduced LingBot-VA, an autoregressive video-action world model that unifies visual dynamics prediction and action inference, and the paper reports fine-tuning with 50 real-world demonstrations per task plus 92.0% and 91.1% success on RoboTwin 2.0 Easy and Hard settings.

Why it matters: HKR-H/K/R all pass: the hook is 50-demo robot control, with a concrete video-action world-model mechanism. Single-source coverage lacks code, benchmark detail, and deployment evidence, so it lands at 78.

Take

LingBot-VA’s strongest claim is not “first causal world model”; it is the ablation tying video prediction to control. The article gives a rare clean hook: removing video prediction drops success from 92.93% to 48.31%, and replacing causal modeling with bidirectional attention drops it to 81.46%. That makes the result harder to dismiss as VLA trajectory memorization.

The title still oversells “general manipulation.” LingBot-VA needs 50 real demonstrations per task for fine-tuning, while the 10-demo claim only says it beats baselines. Inference is 0.5 seconds per closed-loop step on one RTX 5880 Ada, so roughly 2Hz. That fits folding clothes or unpacking boxes. It does not fit fast, contact-heavy assembly. Beating π0.5 and Genie-Envisioner matters, but the article does not list per-task real-world success rates for the six physical tasks.

QbitAI · WeChatJun 3, 13:10

Daxiao Robot and NTU Release PhysX-Omni for Unified Physical 3D Generation

Daxiao Robot and NTU introduced PhysX-Omni, a unified simulation-ready physical 3D generation framework for rigid, deformable, and articulated objects, with PhysXVerse covering 8.7K assets across 2.9K categories and PhysX-Bench evaluating six dimensions including geometry, scale, material, affordance, kinematics, and description.

Why it matters: HKR-H/K/R all pass: unified physical 3D generation is a clear hook, the dataset and benchmark numbers add substance, and robotics simulation data is a real practitioner pain. No open-source or product adoption is disclosed, so it stays at 78.

Computing Life · Share · YageJun 3, 08:00

Microsoft AI's MAI-Thinking-1: Getting Models to Think Is Easy, Sustained Thinking Is Hard

Microsoft AI says MAI-Thinking-1 uses three mechanisms—thermostat, circuit breaker, and self-distillation—to keep RL training stable for several thousand steps; the RSS snippet contrasts MAI’s discipline with DeepSeek’s efficiency and GLM’s endurance.

Why it matters: HKR-H/K/R all pass: the hook is training persistence, the new facts are three stability mechanisms and thousand-step RL runs, and the audience cares about reasoning-model stability. Not a major model launch, so it stays below 85.

Take

MAI-Thinking-1 makes RL stability the product, and that is a better angle than another “reasoning model” claim. The snippet names three mechanisms—thermostat, circuit breaker, and self-distillation—to keep training stable for several thousand steps; it does not give model size, benchmarks, data mix, or release plan. Thin evidence, but the framing tracks the field: reasoning failures are no longer just wrong answers, but policy drift, reward hacking, and collapse after long RL runs. After DeepSeek-R1 turned cheap reasoning into the reference point, Microsoft has little room to win on score theater alone. A disciplined training loop is the kind of boring engineering story a large lab should own. My issue is the phrase “several thousand steps”: without the step definition, task mix, or crash threshold, it reads like a claim, not a reproducible result.

Computing Life · Share · YageJun 3, 08:00

After vibe coding: the industrialization of AI programming

MAI filtered 265,000 trainable tasks from 4.87 million open-source PRs and built a three-layer judging system. The key change after vibe coding is the industrialization of training infrastructure.

Why it matters: HKR-H/K/R all pass via the post-vibe-coding angle, 4.87M PR corpus, 265K tasks, and code-agent infra stakes. No model scores, open-source scope, or product access are disclosed, so it stays below P1.

Take

MAI is betting on the data factory, not the IDE surface. It filtered 265,000 trainable tasks from 4.87 million open-source PRs, roughly 5.4% retention. That rejection rate says the coding-agent bottleneck has moved from autocomplete quality to task manufacturing, judging, and feedback loops.

The three-layer judging setup is the tell, though the snippet does not disclose the layers. SWE-bench pushed the field toward real issues. Cursor pulled users into natural-language code editing. MAI is describing the missing middle: infrastructure that converts repository history into verifiable training work. I’m cautious about the “industrialization” label, but this smells less like a benchmark launch and more like a claim that coding agents will be won by whoever can mass-produce reliable evaluation fuel.

AI HOT (Curated Pool)Jun 3, 04:51

Microsoft releases MAI-Thinking-1 model

Microsoft released MAI-Thinking-1, an MoE model with 35B active parameters and 1T total parameters, pretrained from scratch on 30T tokens without third-party model distillation.

Why it matters: HKR-H/K/R all pass: Microsoft released MAI-Thinking-1 with concrete MoE scale and training-token figures. Benchmarks, access, and pricing are not disclosed, so it stays in the 78–84 band rather than P1.

Jun 2Tuesday

Synced · WeChatJun 2, 13:30

DataMaster: When AI Becomes Its Own Data Engineer

DataMaster searches, cleans, and combines data while keeping the model and training algorithm fixed; on MLE-Bench Lite, it raised the medal rate from 35.91% to 68.18%.

Why it matters: HKR-H/K/R all pass: DataMaster changes the data pipeline under fixed model and training code, lifting MLE-Bench Lite medal rate from 35.91% to 68.18%. This is still a single research release without production validation, so it lands at 78 featured.

Synced · WeChatJun 2, 13:30

Turing Award Winner Sutton’s New Paper Argues AI Should Move Toward Enactive Cognition

Banafsheh Rafiee and Richard S. Sutton propose an enactive cognition framework for AI, naming four pillars: experience, perception-action inseparability, autonomy, and embodiment.

Why it matters: HKR-H/K/R all pass, but the article centers on a conceptual framework and does not disclose experiments, code, or reproducible tests. Sutton’s name and the four pillars put it in the 78–84 research-commentary band.

Financial Times · TechnologyJun 2, 12:00

Top AI Labs Expand Research Into Machine “Consciousness”

Google DeepMind, Anthropic, and Meta are studying whether AI can become conscious and the human implications, but the post does not disclose methods, timelines, or evaluation criteria.

Why it matters: HKR-H and HKR-R pass because top labs studying machine consciousness is a live safety debate. HKR-K fails: the body names labs but gives no method, timeline, or criterion, so this stays at the 72 featured floor.

Jun 1Monday

AI HOT (Curated Pool)Jun 1, 23:45

Introducing Mellum2: JetBrains' 12B Mixture-of-Experts Model

JetBrains published a Hugging Face blog post introducing Mellum2, confirming a mixture-of-experts architecture and a 12B parameter scale; the snippet does not disclose training data, license, benchmarks, or deployment conditions.

Why it matters: HKR-H/K/R all pass, but the body only confirms 12B and MoE, with no benchmarks, license, context window, or IDE integration terms. Treat as a mid-weight model release at the lower featured band.

Import AI (Jack Clark)Jun 1, 21:31

Import AI 459: AI oversight is difficult; scaling laws for protein folding models; and pricing the extinction risk of AI systems

Import AI 459 summarizes papers on AI-economy measurement and AI oversight: one estimates U.S. nominal AI GDP at about $250 billion in 2025, with quality-adjusted real growth near 2,600% per year.

Why it matters: HKR-H/K/R all pass: the extinction-risk pricing hook is unusual, the summary gives $250B and 2600% as concrete figures, and oversight risk has practitioner resonance. It is still a secondary roundup, not a same-day must-write release.

r/LocalLLaMAJun 1, 10:05

I bolted an 8-arm reasoning MoE onto a frozen 1.4B Mamba backbone on a single RTX 3060

The author trained Mamba-Titan-1.4B-Reasoning on a 12GB RTX 3060: a frozen 1.4B Mamba-1 backbone with 8 trainable MoE arms, 2.54B total parameters, Top-2 routing at layers 24/25, and about 50% math accuracy.

Why it matters: HKR-H/K/R all pass via a numbered first-person experiment, but it is a single Reddit post with no independent replication and a fairly technical setup, so it stays in the low featured band.

Take

The useful part is not the claimed 50% math accuracy. It is the attempt to make “frozen backbone plus trainable experts” fit on a consumer GPU. The scraped summary gives real hooks: frozen Mamba-1 at 1.4B, eight trainable MoE arms, 2.54B total parameters, Top-2 routing at layers 24 and 25, and a 12GB RTX 3060. The post body is blocked, so the dataset, training steps, eval set, and ablations are missing.

I’d file this under the LoRA / adapter / small-MoE modification track, not under frontier reasoning. DeepSeek-style MoE is about scaling training and inference. This Reddit build is about structural grafting under brutal VRAM limits. If the promised autopsy has routing collapse, expert specialization, and Mamba state-transfer details, that is more useful than the headline accuracy.

May 31Sunday

Synced · WeChatMay 31, 13:08

Rubrics Survey: How to Define a Good Answer in the Agent Era

Renmin University Gaoling School of Artificial Intelligence released a 40-page survey on rubrics for LLMs, organizing the topic into five parts: definitions, construction methods, training uses, evaluation scenarios, and open challenges.

Why it matters: HKR-H/K/R all pass, but this is a survey rather than a model or product launch. The 40-page rubric framework is useful for agent evaluation, placing it at the featured threshold.

Take

Rubrics are back because agent tasks broke the idea of one correct answer. The RUC Gaoling survey is 40 pages and splits the field into five buckets: definition, construction, training use, evaluation, and open challenges. The clean hook is useful: LLM-as-a-Judge decides who scores; rubrics decide what gets scored.

I don’t buy the optimistic version where clearer rules automatically improve behavior. Once rubrics feed PPO or GRPO, they become a reward surface the policy can game. The article names the right failure modes: veto conditions for medical safety, saturation, online evolution, and reward hacking. OpenAI and Anthropic have both moved toward interpretable reward and process supervision for the same reason. Listing 12 criteria is cheap; keeping a long-horizon agent from learning rubric-shaped theater is the actual engineering problem.

QbitAI · WeChatMay 31, 13:00

Fudan and Tongyi introduce ToolCUA for GUI-Tool path selection in agents

Fudan University and Tongyi Lab introduced ToolCUA-8B, which reaches 46.85% accuracy on OSWorld-MCP after training with about 4k synthetic tools and 180k interleaved GUI-Tool trajectory steps.

Why it matters: HKR-H/K/R all pass: the tool-selection failure hook is concrete, with OSWorld-MCP 46.85% and 180k steps. It stays in the 78–84 band because this is a research release, not a major model or product launch.