Adding "Do not guess" cut made-up fields from 71% to 20% in web extraction
Calling the AI bluff: Adding "Do not guess" cut made-up fields from 71% to 20%
Earn an Honest Dollar tested 16 models and 3 paid APIs on web extraction with missing fields. Without "Do not guess," models invented 405 of 573 missing fields (70.7%). Adding the instruction dropped that to 116 of 574 (20.2%). Gemini 3.8 Flash made up only 1 of 36 missing fields at $0.16. Firecrawl, a paid API, made up 24/36—worse than 13 models. A cheap checker using GPT-6 Luna caught 38 of 49 made-up values with zero false rejections, costing $0.0049. The post notes these are synthetic pages; real-site results may differ.
Why it matters: Earn an Honest Dollar ran 42 twin-page traps against 16 models and 3 paid APIs, and one instruction not to guess cut fabricated fields from 70.7% to 20.2%, with per-model counts and run costs.