AI search currently has practically zero defense against data poisoning. A fifty-dollar test generating 50,000 fake records successfully baited ChatGPT for seven straight months. Google purged the offending domain from its index long ago. Yet ChatGPT kept crawling that exact site 72,000 times in a single month, serving the invented numbers as live answers.
When the model runs retrieval, it only looks at whether the format resembles statistical data. It ignores factual accuracy entirely. Those fabricated data points flowed straight into bank research reports and academic papers. Without source-weight filtering built into the pipeline, making a retrieval system search more aggressively will only pull in more garbage.