Google DeepMind pilots world's first double-blind AI evaluation

Piloting the world's first double-blind AI evaluations

Google DeepMind announced the first double-blind evaluation for proprietary frontier AI models, confining external testing to an encrypted environment so models cannot see test questions in advance. The pilot runs with the Singapore AI Safety Institute, OpenMined, AVERI and MLCommons, testing a Gemini Flash Lite model on confidential benchmarks in a privacy-preserving setup. Google says the aim is benchmark contamination, adding technical and cryptographic protection on top of zero-log protocols and contractual guarantees.

Why it matters: DeepMind and partners including Singapore's AI Safety Institute are piloting double-blind evaluation, showing one technical route against benchmark contamination.