Post-training a visual code completion model on Dweet data
What happened
A Hacker News post on November 1 described a post-training experiment for demoscene code autocompletion. The author continued pretraining Qwen2.5-Code-7B on about 25,000 JavaScript demos (Dweets) from Dwitter.net, each 140 characters. On 30 held-out samples, perplexity fell from 28.63 to 8.57. The author then built a curriculum from Dweet "awesomeness" scores, raising the sampling probability of the top 1,000 samples to about 67% within 400 steps to push an aesthetic preference into the completion model.
Written by AI from the coverage · updated 1 hour ago
Coverage
Follow the reports to see the story from different sides.
- Hacker News front pageI post trained a model on Demoscenes
作者以 Qwen2.5-Code-7B 为基座，在 Dwitter.net 的 25k 条 140 字符 JavaScript demo（Dweet）上做继续预训练，30 条留出样本的困惑度从 28.63 降到 8.57。随后按 Dweet 的“awesomeness”评分构建课程学习，400 步内让前 1000 名样本的采样概率升至约 67%，以把审美偏好注入补全模型。
Heat over time
Not enough continuous observations to draw a trend yet.