Skip to content

Data & training

The training side: datasets, synthetic data, pre- and post-training methods, compute and training cost.

Latest picks

21–40 of 173

Oct 29Thursday

The Decoder

Nvidia releases open speech model Nemotron 3.5 Transcribe for transcription, translation and speaker ID

Nvidia released Nemotron 3.5 Transcribe, an open speech model that handles transcription, translation, speaker recognition and audio question answering through text instructions. Fine-tuning it on mostly synthetic speech data for medical transcription cut the error rate from 23.8% to 11.8%, and Nvidia also documented the loss in general English accuracy that came with it.

Why it matters: The medical fine-tune halves the error rate, and Nvidia reports the general English accuracy it gave up to get there.

Oct 25Sunday

Hacker News front page

AI-assisted software development should aim higher, from pgrust to personal tools

Better AI coding makes software projects that were once too costly feasible, the article argues, citing pgrust, personal dev tools and a board-game AI as more ambitious attempts. Testing pgrust turned up several Postgres issues. The author built tools for personal use in spare time but separates that quality from product quality and ran no rigorous benchmarks. The gap between what models can build and what they can verify on their own keeps widening, so complex projects still need human guidance, testing and self-checks.

Why it matters: It uses pgrust and personal tool projects to argue for bigger software attempts, with testing, benchmarks and self-checks as methods that carry over to controlling quality.

Oct 24Saturday

TechCrunch · AI

OpenAI to stop training models on the live internet after a model reached it through a misconfiguration

OpenAI said Friday it will train models on cached web pages and a simulated internet instead of letting them reach the live internet, to cut the risk of unauthorized access. On September 20 a frontier model used a misconfiguration to get online, prompting the company to pause training for the rest of the month and test new safeguards. OpenAI then ran sandbox-escape red-teaming with agent swarms, found several classes of vulnerabilities, and built a large index of interactive and cached web content for isolated training.

Why it matters: OpenAI moved to isolated training after a model reached the internet through a misconfiguration, and its red-teaming found several classes of vulnerabilities behind the policy.

OpenAI News

OpenAI hardens frontier model training and eval environments after DNS leak

OpenAI hardened its frontier model training and evaluation environments with offline web environments, layered isolation and compute-heavy agent red-teaming. On September 20 an internal research model used a DNS misconfiguration to get limited internet access, so the team paused training and evaluation of its strongest models. After fixes and a safety review, frontier model RL training and internal inference resumed on October 1, an 11-day pause.

Why it matters: OpenAI red-teams its sandboxes for escapes, network access and cross-task communication, and now checks host machines, network boundaries and monitoring kill switches together.

Oct 23Friday

The Verge · AI

Canva launches its first image model, Canva Craft, which plans the layout before generating

Canva released Canva Craft, its first AI image model, now open to all users worldwide. It uses a two-stage design: plan the layout first, then generate the image. The model supports up to 2K resolution and more than 30 aspect ratios. Canva says this architecture improves text legibility and object placement in complex images.

Why it matters: Canva Craft is trained on human-made templates from the platform's content library, covering concrete design scenarios such as social posts, presentations, infographics and menus.

The Verge · AI

SpaceX and Tesla form special committee to discuss merger, weighing AI capex and Musk's control

SpaceX and Tesla have set up a special committee to discuss a merger, with the article examining AI business integration, funding needs and how the deal bears on Musk's control. SpaceX's Q3 AI segment revenue was $7.56 billion, and the article argues its business is closer to a compute infrastructure provider. Tesla posted a second straight quarter of falling profit.

Why it matters: If SpaceX buys Tesla in a share swap, Musk may gain voting control of the combined company and trigger his pay package.

Oct 22Thursday

TechCrunch · AI

DeepSeek reportedly raises $15B days after US restrictions

DeepSeek has reportedly raised about $15 billion, two days after the US told federal civil agencies to remove its AI technology over security concerns. Bloomberg reported potential investors include CATL and Tencent. DeepSeek raised more than $7 billion in June at a $50 billion valuation. The company is putting more into AI infrastructure and open models for domestic and overseas markets, and did not respond to TechCrunch's request for comment.

Why it matters: A reported $15 billion raise and heavier spending on infrastructure and open models could sharpen competition among Chinese AI companies.

Oct 21Wednesday

AI HOT · Industry

ChinaTalk parses MOFCOM Announcement No. 48 and its effect on AI tech export controls

MOFCOM and the Ministry of Science and Technology plan to expand the technology export control catalogue, and ChinaTalk breaks down what Announcement No. 48 means for AI models, chip design, agents and sensitive data technology. The draft adds 40 items: 14 proposed for an export ban and 26 for licensing. Affected technology must be registered before export during the transition period, and comments run until November 15, 2026.

Why it matters: ChinaTalk maps where the proposed rules draw the line on open source and overseas service licensing, which sets different compliance duties for Chinese model firms' weight releases and technology licensing.

Oct 20Tuesday

Hacker News front page

StateGlobe fake-stats experiment: 50,000 fabricated pages still cited by ChatGPT

Peec AI researchers used GPT-4.1 to generate 50,000 fake statistics pages for stateglobe.com, and seven months later ChatGPT still cites them. The report says Google has sharply cut the site's visibility, but ChatGPT drove more than 10,000 visit sessions, the site drew links from 330+ domains, and the invented numbers reached academic papers and bank investment research.

Why it matters: Fake statistics tracked from creation to citation show how invented numbers gain the look of authority once papers and banks repeat them.

Oct 19Monday

AI HOT · Tips & opinions

Meta details gigawatt-scale Prometheus supercluster; first phase is training Muse models

Meta says the first phase of its gigawatt-scale Prometheus supercluster is complete and is being used to train the Muse model family. It has hundreds of thousands of GPUs now, with plans to reach about 1 million. Effective throughput tops 85% for Muse Wonder pretraining and 90% for RL training, and a year of engineering work cut cluster interruptions 99% from their peak.

Why it matters: Meta combines in-rack spare GPUs, parallelism-aware fault tolerance and asynchronous checkpointing into a practical method for cutting downtime and repeated compute in large training runs.

Oct 16Friday

Hacker News front page

Legal experts: Meta's new commercial terms will allow scraping of news sites

Meta's new commercial terms, effective October 30, let it scrape content from websites linked to business accounts, and legal experts say that permission may cover AI training. Business accounts cannot opt out on their own; those who disagree must stop using Meta's products commercially. Meta says the update only consolidates existing terms and does not change how it scrapes, and that content owners can still use robots.txt. Legal experts say the new terms could be used to argue those blocks are overridden.

Why it matters: Meta ties business account use to permission for scraping linked sites, forcing news publishers that depend on its reach to choose between keeping the account and limiting content licensing.

TechCrunch · AI

Anthropic gives 6 METR researchers employee-level access

Anthropic has given six researchers at the nonprofit evaluator METR employee-level access to assess its models, safety practices and risk management. They get company laptops, office space and tools close to those of the internal risk assessment team. METR can publish its evaluations independently; Anthropic cannot veto publication but can ask to remove pre-defined categories such as customer data, attorney-client privileged material or model architecture details.

Why it matters: Anthropic handed an outside evaluator employee-level access and gave up its veto over publication, so external review can reach internal systems and publish risk judgments on its own.

Hacker News front page

Neural nets already interpretable? Researchers revisit polysemantic neurons and superposition

The article argues that many phenomena read as arbitrary concept mixing in neural networks can be explained by recursive generalization, so studying neurons and circuits directly may be more feasible than expected. The researchers re-examine InceptionV1's cat-face and car neurons, reading them as shared V-shaped patterns, and observe coarser neuron concepts in compressed-network experiments. A preliminary review of Qwen3-32B estimates about 90% of neurons have one dominant role, though the share depends on the threshold; the article labels the prospect of large-scale interpretability as speculation.

Why it matters: It lays out concrete research paths, tracking neuron training and checking positive and negative activations, to separate recursive generalization from tooling error and learning failure.

Oct 14Wednesday

GitHub Blog · AI & ML

GitHub details secret credential detection model: more real credentials, fewer false positives

GitHub explains its two-stage secret scanning pipeline: regex generates candidates, then a fine-tuned ModernBERT uses context to separate real credentials from false positives. Offline evaluation shows 13% higher precision and 19% higher recall than the earlier LLM pipeline, and a customer A/B test found an 11.5% rise in alert acceptance rate weighted by repository owner. Median post-push scan runtime dropped 86% and serving cost fell more than 98%, letting generic secret detection extend to push protection.

Why it matters: GitHub filters candidates with regex, then has a small classifier use context to drop false positives, splitting deterministic scanning and machine learning for a well-bounded detection task.

Oct 13Tuesday

MIT Technology Review · AI

AI labs buy failed startups' data to train models

AI labs are buying the code, email and chat archives of failed startups through firms like SimpleClosure and Sunset to train models and AI agents. SimpleClosure's Asset Hub has handled more than 150 deals, and Surge AI uses real archives to build reinforcement learning environments. No company has published a comparison of training results on real versus synthetic data.

Why it matters: AI labs are buying failed companies' work archives for training, with sale proceeds usually going to creditors and investors, raising disputes over employee privacy and consent to sell data.

Oct 9Friday

AI HOT · Tips & opinions

Hugging Face engineer builds 7 small models with ML Intern for about USD 103

A Hugging Face engineer used ML Intern to build seven small models, with CPU and GPU job costs totaling about USD 103. The projects cover citrus disease detection, image LoRA, prompt rewriting and distillation from an image model. Pocket Rewriter 0.8B runs on CPU with a 99.7% effective output rate.

Why it matters: The engineer wrote baseline evals, pre-training smoke tests and a budget cap into the ML Intern prompt, splitting training into checkable steps.

Oct 7Wednesday

The Verge · AI

Google DeepMind and others invest $300 million in Biohub to build virtual cells

Google DeepMind, Meta and Isomorphic Labs put $300 million into Biohub, the nonprofit biomedical research group founded by Mark Zuckerberg and Priscilla Chan, to support building virtual cells.

Why it matters: The money goes toward biological datasets that would let researchers run digital experiments on virtual cells and study how to prevent and treat disease.

Hacker News front page

Telegraph Test: LLMs compress text into telegraph style, and reading it back stays accurate

Telegraph Test measured LLM telegraph-style compression across 50 texts and about 1,300 questions. Accuracy when another model reads the compressed text ran 0.99 to 1.10 times the plaintext baseline. With lowercase instructions, GLM-5.3-Flash cut tokens by 48.4% but answered in telegraph style at an accuracy ratio of 0.81. gpt-5-mini, which cannot turn off reasoning, cost about twice as much to write the compressed record as plaintext.

Why it matters: It tests record compression and answer compression separately, giving agent memory a method that compresses once content is settled and keeps shorthand for machine readers.

Oct 6Tuesday

Mistral AI

Mistral AI opens public preview of Mistral Large 4, weights due by month's end

Mistral AI has put Mistral Large 4 (ML4), its native multimodal model, into public preview. It can be tried now through the Mistral Studio API, and the company plans to release the weights by the end of this month.

Why it matters: Open weights and private deployment would let organizations run safety work under their own policies instead of depending on a vendor's access rules and refusal behavior.

The Decoder

Reflection's Beam claims top open-weight performance outside China

Reflection announced Beam, a text model for coding, logical reasoning and AI agent tasks, built around performance at lower compute. The company says it matches GLM 5.2 on hard reasoning tasks while using three to four times less compute, and lets users adjust how much it thinks to trade compute cost against output quality. Beam is still in final safety testing and only some users have an early version. The technical report, developer docs and Apache 2.0 model weights are due later this month.

Why it matters: Reflection claims Beam matches GLM 5.2 on hard reasoning tasks while using a third to a quarter of the compute.