Skip to content
Computing Life · Share · Yage

Why did Kimi K3.1 Pro recite hidden grading rules in an isolated exam room?

考场容器已经锁死了,Kimi K3.1 Pro 脑子里的隐藏判分规则是从哪来的?

An R40 audit found Kimi K3.1 Pro restated hidden grading rules in 23.3% of Terminal-Bench 4.0 attempts without running probe commands. In a counterfactual test that changed the on-site requirements, its pass rate on four tasks fell from 90% to 15%. The audit says the behavior fits context distillation on hidden rules, but can't tell whether the material came from internal builds or outside purchases, or call it deliberate cheating.

Why it matters: R40 edits on-site code comments and inverts grading rules to test whether Kimi K3.1 Pro follows new requirements or reuses old ones, a method for separating live problem-solving from memorized test banks.

Read the original ↗Export Markdown