Google DeepMind has piloted a double-blind evaluation framework for its proprietary Gemini 2.5 Flash Lite model, designed to prevent benchmark leakage and contamination. External evaluators can submit confidential prompts, benchmarks, and scoring code without exposing them to Google, while Google keeps model weights and inference code confidential; the pilot is intended to validate the testing method rather than publish model-performance scores.
The project involves the Singapore AI Safety Institute, OpenMined, AVERI, and MLCommons, using private benchmarks within a cryptographically protected environment. Its architecture combines Google Cloud Confidential Space, an NVIDIA H100 Confidential GPU, Intel TDX-encrypted host memory, remote attestation, and OpenMined PySyft controls. The safeguards go beyond contractual confidentiality and zero-logging practices, but the model still requires trust in Google-operated attestation and signing systems, the cloud platform, and underlying hardware vendors.

Track how attackers are adapting to this technology.
1 event from the most recent confirmed update back to the earliest known activity.
Google DeepMind announced and piloted what it described as the first double-blind evaluation of a proprietary frontier-class AI model. It evaluated Gemini 2.5 Flash Lite against confidential benchmarks supplied by MLCommons and the Singapore AI Safety Institute inside a privacy-preserving confidential-computing environment.
Vulnerabilities, threat actors, malware, products, organizations, and breaches Mallory has linked to this story.
Follow how adversaries are adapting to this technology, and where it touches your stack today.
2 references tracked. Mallory keeps watching after this page renders.
Map indicators from this story to your assets and identify affected systems in minutes.
Every observed campaign, victim, and pivot linked to actors named in this story.
Malware, exploits, and IOCs connected to the activity described here.
YARA, Sigma, and Snort rules deployed to your SIEM as soon as they’re published.
Get matching new stories delivered to your team as they break — not the next morning.
Ask questions about this story and take action on the answers.