Skip to content

Data & training

The training side: datasets, synthetic data, pre- and post-training methods, compute and training cost.

Latest picks

41–60 of 173

Oct 3Saturday

Hacker News front page

Aleph Alpha releases Kolibri, an open-weight English-German model with 1M-token context

Aleph Alpha released Kolibri, an English-German MoE model with 78B total parameters and about 3B active, supporting up to 1M tokens of context. Full weights are open on Hugging Face under Apache 2.0. Deployment needs the aleph-alpha-inference package with the Kolibri vLLM plugin.

Why it matters: Aleph Alpha turns industry failure cases into a continuously running evaluation and improves the model with synthetic training environments, without training on customer data.

Oct 2Friday

Hacker News front page

Schwartz reviews using Claude and BootLoops for cross-disciplinary science

Matthew Schwartz built BootLoops, an open-source scientific computing toolkit, with Claude, and in three months ran research across 18 fields with 19 collaborators, producing 36 manuscripts. He had Claude find quantitative problems suited to its abilities, then let domain experts steer the research direction. He stresses that calculations, conclusions and scientific value still need human checking.

Why it matters: Schwartz uses BootLoops to find science problems suited to Claude, then has domain experts pick the direction, pushing technical computation toward questions with scientific value.

Computing Life · Share · Yage

How OpenAI's AI worm was produced in sandbox training and used for safety training

OpenAI's self-replicating prompt worm was produced inside controlled sandbox training, and the report says no effects were seen beyond simulated tool calls during training and evaluation. The article explains how GPT-Red trains attackers by rewarding unauthorized actions and forwarding of attack instructions, generating injections that spread through email and other channels, with bulk attack samples fed into GPT-5.6 training.

Why it matters: OpenAI uses automated red teaming to mass-produce attack samples and fold them into model training, moving defense from pre-release manual testing to continuous adversarial work during training.

Oct 1Thursday

The Decoder

OpenAI says it blocked an attack that stole model reasoning, but the method still works on Azure

OpenAI says it stopped an attack that stole its model reasoning. The campaign started at low volume on July 1, jumped to 16,000 requests from over 4,000 users on July 24-25, and involved more than 15,000 linked accounts. All were shut down by July 28. OpenAI says the key people are tied to Moonshot AI.

Why it matters: OpenAI disclosed the scale of a July inference distillation attack and how it shut it down, and researchers then found the same method still worked on several models on Azure.

OpenAI News

OpenAI says it detected and blocked a coordinated model-distillation attack

OpenAI says it found and blocked a coordinated distillation attack that first appeared in the first week of July, with attackers trying to extract protected reasoning from its models. Peaks came on July 24 and 25: 16,000 requests from more than 4,000 users, with prompt-pattern activity tied to over 15,000 users. OpenAI says it fully blocked the activity by July 28.

Why it matters: OpenAI lays out the timeline and response to a coordinated distillation attack that copied encrypted reasoning content to trick models into transcribing protected reasoning.

Sep 30Wednesday

Google DeepMind

Google DeepMind launches SynthID Bio, watermarking AI-generated proteins

Google DeepMind released SynthID Bio, applying watermarking to synthetic biology. The method embeds an imperceptible signature in biological code that can be verified on the resulting physical proteins while their biological function is preserved.

Why it matters: Google DeepMind's SynthID Bio watermarks AI-generated protein sequences and 3D structures, verified on proteins synthesized in the lab while keeping their binding function.

The Decoder

DeepSeek and Huawei release open-source programming tools for Ascend chips

DeepSeek and Huawei built programming tools for Huawei's Ascend chips and open-sourced all of them. The centerpiece is TileLang, a language the pair describe as easier to pick up than CUDA. The two also optimized a supernode made up of 128 Ascend 950 chips.

Why it matters: DeepSeek and Huawei open-sourced Ascend chip programming tools, with the TileLang language easier to pick up than CUDA, offering one reference path for domestic chip software.

Hacker News front page

Text classification from bag-of-words to Jev: a language model that classifies without per-task fine-tuning

Jev turns text classification into a general decision service that needs no per-task fine-tuning. The article traces the history of classification techniques and tests Jev's limits. On the 25,000-review IMDb test set, the Choice API hit 96.47% accuracy at a total cost of $0.6492. The article notes it cannot confirm whether the test set was part of the training data.

Why it matters: It compares Jev with bag-of-words models, ModernBERT and general LLMs, and lays out the cost and fit of direct cross-task classification versus fine-tuning a dedicated model.

AI HOT · Industry

Anthropic signs compute deal with SpaceX worth up to $84.5B, cancellable on 90 days' notice

Anthropic signed a compute deal with SpaceX worth up to $84.5 billion, renting Nvidia GPU capacity in SpaceX data centers. The documents show the deal was signed in May 2026 and covers the Colossus 1 and Colossus 2 data centers. Anthropic pays $1.25 billion a month through May 2029, with discounted rates during the May-June 2026 ramp-up.

Why it matters: Anthropic's compute deal with SpaceX caps at $84.5 billion and can be cancelled on 90 days' notice, showing both the scale and the flexibility AI labs now buy compute with.

The Decoder

AMD to acquire world-model startup World Labs for $8.2B

AMD announced an all-stock deal to acquire spatial-intelligence startup World Labs for about $8.2 billion. The deal is expected to close by the end of 2026 and still needs regulatory approval. World Labs founder Fei-Fei Li will join AMD's leadership as executive vice president and chief scientist, reporting directly to CEO Lisa Su and leading frontier research.

Why it matters: Beyond the price and where the team goes, the original lays out the world-model route and the hardware logic behind AMD chasing Nvidia.

Sep 29Tuesday

OpenAI News

OpenAI drafts a safety case framework for frontier AI training

OpenAI is building a safety case framework for frontier reinforcement learning training, arguing that structured, evidence-based risk arguments should come before training continues. The proposal covers alignment training, containment and monitoring, plus cross-team challenge, an executive veto, pause mechanisms, incident investigation and public disclosure. It is being rolled out inside OpenAI, and the safety case stays a goal built up over time. The document does not cover the broader alignment requirements for internal and external deployment.

Why it matters: OpenAI splits training safety into technical safeguards, operational approval and incident investigation, folding monitoring thresholds, pause mechanisms and accountability into how it manages training risk.

Sep 22Tuesday

Anthropic News

Anthropic, WHO and partners use Claude in DRC Ebola outbreak response

Anthropic's Beneficial Deployments and Applied AI teams worked with CEPI, the WHO African Regional Office and INRB to use Claude in the response to the Bundibugyo ebolavirus (BDBV) outbreak in the Democratic Republic of the Congo.

Why it matters: Anthropic is working with CEPI and WHO's Africa office to run Ebola outbreak data from the DRC through Claude, cutting a briefing that took a full day to under an hour.

Sep 17Thursday

Hacker News front page

I had Gemini train its own replacement for $9

The author paid Gemini 3.1 Pro $9 to label 4,290 Reddit comments with knife brands, models, and steels, then fine-tuned GLiNER large v2.5 on those labels. The resulting model runs locally and hits 0.83 F1 against Gemini's labels after 24 minutes on a Tesla T4. Zero-shot GLiNER scored roughly 0.65 F1. The hardest bug was words_mask: the docs suggest a binary mask, but it's actually a word index; filling it with ones kept loss flat at 70. Five of ten runs produced no usable model—three config failures, two from words_mask. The post doesn't report human accuracy on Gemini's labels, so 0.83 is measured against Gemini, not ground truth.

Why it matters: The author paid $9 for Gemini labels on 4,290 comments, fine-tuned GLiNER from 0.65 to 0.83 F1, and now runs it locally instead of paying per comment.

Sep 14Monday

Hacker News front page

The AI job market in 2026: who gets hired, what they earn, and which roles are fading

Maksim Ilin pulls together LinkedIn, WEF, Stanford, PwC, and Bain data for a September 2026 snapshot of the AI labor market. AI Engineer is the most-hired role; Research Scientist at a frontier lab is the most prestigious, with median pay around $746K at Anthropic and $1.15M at OpenAI L5. The fastest-growing niches are agentic systems and Forward Deployed Engineering—postings for the latter jumped over 1,000% YoY. Prompt engineer has faded as a job title; the skill remains but dissolved into other roles. AI skills now command a 62% wage premium in the US. Bain projects more than 1.3M AI jobs in the US by 2027 against roughly 645K available workers. In Europe, over half of AI postings sit outside tech departments, and Germany shows seven AI-user roles for every AI-developer role. Entry-level hiring got harder: employment for 22–25-year-olds in AI-exposed occupations now trails the rest by 19%.

Why it matters: The author pulls LinkedIn, WEF and PwC data into one picture of 2026 AI hiring and pay, including roughly 1,550 new AI Engineer postings a week and a $176,000 median salary.

Sep 8Tuesday

Google DeepMind

Google DeepMind releases AlphaGenome Atlas, predicting every single-base variant in the human genome

Google DeepMind released AlphaGenome Atlas, a platform holding effect predictions for 9 billion single-nucleotide variants across the human genome. It spans 1PB, more than 30 times the size of the AlphaFold Database.

Why it matters: Google DeepMind precomputed AlphaGenome predictions into a 1PB dataset covering all 9 billion single-base variants in the human genome, over 30 times the size of the AlphaFold Database, with a sortable AVI score alongside it.

Sep 4Friday

Computing Life · Share · Yage

Three ledgers to check before self-hosting open models

Lambda engineer Zach Mueller admits his home GPU rack doesn't save money—the return is skill investment. The article uses H1 2026 data to show open models are viable, but self-hosting math is counterintuitive. Three ledgers: cost (cloud API wins for most, two H100s need ~2B tokens/month to break even), data (commercial agreements often suffice), and capability (fine-tuning and hands-on skills are the real payoff). Three tiers from renting tokens to owning hardware, with a two-to-three-week rental test recommended before buying.

Why it matters: It offers a three-part cost framework for self-hosted open models, plus a break-even point: two H100s must push about 2 billion tokens a month to beat mid-priced cloud APIs.

Sep 3Thursday

Hugging Face Blog

A 350M model fine-tuned with GRPO in 100 steps lifts structured-output compliance from 22.6% to 29.7%

A hands-on guide from Hugging Face and Liquid AI that fine-tunes LFM2.5-350M with GRPO via the TRL library. Using only 500 samples and 100 training steps on a free Colab GPU, structured-output compliance on the IFStruct benchmark jumps from 22.6% to 29.7%. The post includes the full notebook, reward-function design, and a local evaluation setup with llama.cpp on a MacBook.

Why it matters: Hugging Face published a full TRL GRPO recipe for a 350M model, and 100 steps lifted IFStruct structured-output pass rate from 22.6% to 29.7%.

Aug 28Friday

Hacker News front page

Free, framework-free Colab notebooks for RAG, agents, and evals on the Groq API

calmrocks published a set of Colab notebooks on GitHub for AI engineers and forward-deployed engineers. They cover model APIs, structured output, tool calling, RAG, evals-as-the-spine, agent loops from scratch, tool design, guardrails, MCP, Skills, fine-tuning vs LoRA, prompt injection, LLMOps, and customer craft. Everything runs on the free Groq API with no frameworks. The post doesn't specify the number of notebooks or an update schedule.

Why it matters: calmrocks published a set of free Colab notebooks on GitHub covering RAG, agent loops, evals, fine-tuning and guardrails, all framework-free and portable to day-to-day engineering work.

Aug 27Thursday

Anthropic News

Anthropic opens 10,000 Claude seats to researchers, expands AI for Science

Anthropic announced a new Claude team plan for scientists, opening 10,000 seats to researchers worldwide. Standard seats are free; a higher-tier seat with 5x usage limits costs $15 per month for one year. Anthropic says it plans to grow the program beyond 10,000 seats in the coming months.

Why it matters: Anthropic is giving researchers 10,000 Claude seats for a year, free at the standard tier and $15 a month for 5x usage, while widening AI for Science funding.

Aug 6Thursday

Computing Life · Share · Yage

Fine-tuning is back in 2026, but now it's a cost-engineering play

Engineering teams in 2026 are fine-tuning again—not to make models smarter, but to slash inference costs on high-volume narrow tasks. FermiSense fine-tuned Qwen3.5-9B for e-commerce review, cutting cost from tens of dollars to $0.50 per 1k calls. Intercom's customer-support small model hit 73.1% resolution rate at one-fifth the cost of GPT-5.4. On the vision side, a DINOv3 classifier workflow trained a zero-API-cost local classifier with only 839 reviewed samples, reaching AP 0.9731. The article provides a decision matrix: fine-tuning pays off above ~50k daily requests with automatically verifiable outputs; below that, Prompt Caching plus RAG is the better bet. Most vendor-reported high scores lack third-party reproducible test sets, so hybrid routing remains the pragmatic middle ground.

Why it matters: FermiSense and Intercom used fine-tuning to cut narrow-domain inference costs to a fifth or less, tying the fine-tuning revival to concrete numbers.