Skills Are Checklists, Not Textbooks: Princeton et al. Paper Reveals How Agent Skills Actually Work
A five-university paper analyzed 8,135 agent execution traces and found that 65.7% of skill effectiveness comes from procedural guidance and checklists, while only 4.5% comes from filling knowledge gaps. Distilling the same raw traces into a guide lifted success from 59.1% to 61.9%; feeding raw logs dropped it to 55.9%. Guides distilled from all-failure traces scored below the no-skill baseline. Expanding the skill pool from 5 to 100 crashed exact-match retrieval from 29.6% to 3.3%, yet task success edged up from 36.4% to 39.3%. The paper is an arXiv preprint, not yet peer-reviewed; evaluation is limited to terminal and tool-use tasks on Codex and Gemini CLI models.
Why it matters: A five-university paper with 8,135 agent traces and a clean controlled experiment: skills work as checklists, not knowledge dumps. Hits all three HKR axes with direct practical implications for agent builders. Docked slightly because this is a secondary write-up of a paper tha...