Hacker News front pageJul 31, 2026, 02:13 UTC+8

Distilling DeepSeek into GPT-OSS Doesn't Transfer Censorship

Show HN: Distilling DeepSeek into GPT-OSS doesn't transfer censorship. Try it

CTGT distilled a 120B finance model from DeepSeek V4 Flash. Across 152 matched prompt pairs, the teacher scored 45.45 points more censored on China-sensitive topics, but the student showed no censorship at all—four US lab judges agreed. Self-distillation on corrected outputs matched the DeepSeek-taught model on financial reasoning, at 62× lower cost than Inkling. Code, data, and models are open.

Why it matters: CTGT distilled DeepSeek V4 Flash into a 120B finance model and found censorship didn't transfer, while self-distillation matched the teacher on finance reasoning at a fraction of the cost. Ships with weights, a playground, and a reproducible eval framework. HKR all hit. Not sc...

Read the original ↗Export Markdown

Take

This one's worth opening because it tests a question a lot of people ask privately: if you distill from a Chinese model, does the censorship come along for the ride? CTGT used DeepSeek V4 Flash as the teacher for a 120B finance model. Across 152 matched prompt pairs, the teacher scored 45.45 points more censored on China-sensitive topics, while four US lab judges unanimously found the student uncensored. The wilder part: letting the student correct its own mistakes and retrain matched the DeepSeek-taught version on financial reasoning, at 1/62 the cost of Inkling.

I'd discount this a bit — the post doesn't spell out the full test set or how the matched prompts were constructed, 152 pairs isn't huge, and all four judges are US models, which could introduce systematic bias. But the code, data, and model weights are all open, so you can run it yourself.

Don't read this as 'distillation washes out censorship.' The more precise take: in this specific finance-domain distillation run, the student didn't pick up the teacher's politically sensitive refusal patterns. Whether that holds for other tasks or other teacher models — we've only got this one experiment so far.

Source: Hacker News front page · ctgt.ai
Published: Jul 31, 2026, 02:13 UTC+8 · Jul 31, 2026