Understanding SFT Mechanisms in LLMs: Resolving Practice Disputes and Avoiding Wasted Compute
吃透大模型SFT底层机理:终结实践争议,规避无效算力
Junpeng Zhang and coauthors argue that SFT on highly homogeneous data has an effective window of only hundreds to about 1,000 training steps, and their interaction-based warning signal detects overfitting before loss gaps appear, saving roughly 30%–50% of training compute.
Why it matters: Shows SFT on homogeneous data has a narrow training window, with an early-warning method that flags overfitting before the loss gap widens.