Skip to content
Hacker News front page

Adding "Do not guess" cut made-up fields from 71% to 20% in web extraction

Calling the AI bluff: Adding "Do not guess" cut made-up fields from 71% to 20%

Earn an Honest Dollar tested 16 models and 3 paid APIs on web extraction with missing fields. Without "Do not guess," models invented 405 of 573 missing fields (70.7%). Adding the instruction dropped that to 116 of 574 (20.2%). Gemini 3.8 Flash made up only 1 of 36 missing fields at $0.16. Firecrawl, a paid API, made up 24/36—worse than 13 models. A cheap checker using GPT-6 Luna caught 38 of 49 made-up values with zero false rejections, costing $0.0049. The post notes these are synthetic pages; real-site results may differ.

Why it matters: Earn an Honest Dollar ran 42 twin-page traps against 16 models and 3 paid APIs, and one instruction not to guess cut fabricated fields from 70.7% to 20.2%, with per-model counts and run costs.

Read the original ↗Export Markdown