#数据/训练

0 today

Yesterday · Sep 29Tuesday

MIT Technology Review · AISep 29, 18:43

Making AI an asset, not an expense

HPE 提出，当 AI 从试验走向客服、IT、研究等常驻生产负载，按 token 消费的模式会让支出变成难以预测的月度变动项，企业需按工作负载判断是否转向自有算力。

Jun 7Sunday

AI HOT (Curated Pool)Jun 7, 08:00

AI Substitution Wave: Three Forces Reshape Cost Structures

Coinbase, Lindy, Harvey, and Cursor shifted workloads to cheaper models; Harvey reported Kimi 2.6 reached a 15% all-pass rate on Legal Agent Benchmark, versus Opus at 14%, with 100 tasks costing $84 versus $954.

Why it matters: HKR-H/K/R all pass: the $84 vs $954 cost delta and named cases from Coinbase, Lindy, Harvey, and Cursor give it concrete signal. It is a strong cost-structure commentary, not a major model or product release, so it fits the 72-77 band.

Jun 5Friday

Xinzhiyuan · WeChatJun 5, 16:01

Anthropic warns of AI self-acceleration as OpenAI is said to cross a reliability threshold

Xinzhiyuan cites a Yann Dubois interview saying OpenAI crossed a reliability threshold around last December, while Anthropic’s internal data says per-person quarterly code contribution reached 8× the Q1 2024 level by Q2 2026.

Why it matters: HKR-H/K/R all pass: the cliff-edge framing is clickable, and the summary includes a timing claim plus Anthropic’s 8x coding metric. Capped at 82 because this is second-hand interview analysis, not an official release or reproducible test.

Take

The “self-acceleration” framing is too easy to inflate into ASI theater. The harder claim is org-level productivity: Anthropic says per-person quarterly code contribution hit 8× its Q1 2024 level by Q2 2026. That is not AI autonomously building its successor. It is researchers, coding agents, tooling, eval loops, and infra all compressing the iteration cycle. Dubois says OpenAI crossed a reliability threshold around last December, but the article gives no internal metric or reproducible test.

I buy the “error probability every two minutes” frame more than the threshold headline. Agents fail over duration, not single prompts. A vertical harness that moves reliability from 80% to 85% has value, but it also becomes maintenance debt as models change underneath it. If a startup is selling the “AGI smell” here, ask for repeatable task length, failure rate, and who ran the benchmark.

Jun 2Tuesday

r/LocalLLaMAJun 2, 06:46

I spent months inside verl, forked it, then stopped: internals, fork costs, and an NCCL bug

ReinforcedKnowledge analyzes ByteDance’s verl RLHF loop, covering DataProto plus rollout, reward, advantage, and update paths. The author stopped a private fork because near-daily upstream changes made sync cost exceed refactoring work, and describes an NCCL hang fixed on one node by setting NCCL_SOCKET_IFNAME=lo.

Why it matters: Niche but useful RL post-training field report, not an industry release. HKR-H comes from the fork-then-quit twist; HKR-K has verl’s five paths and NCCL_SOCKET_IFNAME=lo; HKR-R hits the cost of maintaining open-source training forks.

May 17Sunday

Dwarkesh Patel podcastMay 17, 03:01

Notes on Pretraining Parallelisms and Failed Training Runs

Dwarkesh documents pretraining failure modes and parallelism tradeoffs: expert choice and token dropping can break causality in MoE routing, FP16 collectives can bias repeated additions after values exceed 1024, pretraining FLOPs are given as 6ND, B300 HBM is listed as 288GB, and FSDP communication can reach params × 3 with reduce-scatter.

Why it matters: HKR-H/K/R all pass: Dwarkesh’s notes expose concrete pretraining failure modes and numbers. The systems-training focus is specialized, so it sits in the high-quality band rather than same-day must-write.

May 8Friday

QbitAI · WeChatMay 8, 12:05

All Labs Watch ByteDance, Everyone Praises DeepSeek: A U.S. Researcher’s 36-Hour China AI Trip

Ai2 researcher Nathan Lambert visited Zhipu, Moonshot AI, Tsinghua, Meituan, Xiaomi, and 01.AI within 36 hours, and said Chinese labs closely watch ByteDance and respect DeepSeek, while student participation in core work, open source habits, and in-house control of the technical stack mark key differences.

Why it matters: HKR-H/K/R all pass: the piece has a named US researcher’s dense China-lab tour plus concrete claims on ByteDance, DeepSeek, open source, and in-house stacks. It is strong industry field reporting, not a model launch or major deal, so it sits at featured rather than p1.

Take

The useful signal here is not “Chinese labs are humbler.” It is that Beijing’s AI density is now obvious to an outside researcher after one compressed trip. Nathan Lambert saw Zhipu, Moonshot, Tsinghua, Meituan, Xiaomi, and 01.AI in 36 hours, then came away with two repeated tells: everyone watches ByteDance, and everyone respects DeepSeek.

I don’t fully buy the “less ego, faster catch-up” frame. The harder mechanism is labor and control: students sit near core model work, companies build their own data and RL environments, and non-AI giants like Meituan and Xiaomi still train foundation models. OpenAI and Anthropic largely keep interns away from core frontier work; Chinese labs appear to use students as real engineering bandwidth. But the piece also says Nvidia compute is scarce and data suppliers are uneven, so culture should not become a fairy tale explanation for infrastructure constraints.

Apr 20Monday

r/LocalLLaMAApr 20, 18:36

Actually put Gemma 4 26B to work on something real: extract trading signals from 2,400 earnings calls

A Reddit user fine-tuned Gemma 4 26B on 800 labeled earnings-call transcripts and ran inference on 2,400 transcripts over 3 years on one RTX 4090 in about 14 hours. On 600 out-of-sample transcripts, one signal linked vaguer CFO guidance to about 1.8% sector-relative underperformance over 5 days with IC 0.04. A stronger signal showed 0.85 correlation with sector returns after checks and was discarded as a ghost factor; the key point is factor sanity checks, not the profit claim.

Why it matters: Strong HKR-H/K/R: this is a named first-person experiment with concrete setup, metrics, and a useful negative result. It stays at featured, not P1, because it is one Reddit test rather than a product release or industry-wide event.

Take

The useful part is not that Gemma 4 26B found a tradable signal; it is that the author killed the signal that almost fooled them. The setup is concrete: 800 labeled earnings-call transcripts for fine-tuning, 2,400 transcripts over three years for inference, one RTX 4090, about 14 hours. In the 600-call out-of-sample slice, vaguer CFO guidance mapped to 1.8% sector-relative underperformance over five days, with IC at 0.04.

Most finance LLM demos quietly confuse sector exposure, sentiment beta, or leakage with language understanding. Here, the stronger-looking factor showed 0.85 correlation with sector returns and got thrown out as a ghost factor. That makes the post more credible than another “LLM beats markets” chart. Gemma 4 26B’s win is not magic alpha; it is cheap enough iteration for factor hygiene.

Jan 3Saturday

TechCrunch · AIJan 3, 01:33

How AI is reshaping work and who gets to do it, according to Mercor's CEO

Mercor reached a $10 billion valuation in 3 years and acts as a talent middleman in AI's data boom. The RSS snippet says it connects labs such as OpenAI and Anthropic with former Goldman Sachs, McKinsey, and elite law firm employees, paying up to $200 an hour to provide domain expertise and train models. The real signal is the labor pipeline: experts from automatable fields are helping build these systems; the post does not disclose scale, contract terms, or task allocation.

Why it matters: Featured on HKR-H/K/R: the angle is displaced experts getting paid up to $200/hour to train models, plus a concrete $10B-in-3-years data point. The post does not disclose scale, contract structure, or task allocation, so it stays in the low-featured band.