Why it matters
Google DeepMind’s December 2025 FACTS suite evaluates factual answers under four different information conditions: model knowledge alone, web search, images and supplied documents. Its search track standardizes the retrieval tool across models, while public examples accompany a private held-out evaluation set managed by Kaggle. The combined score averages results across tracks and sets, which can conceal sharply different failure patterns. The release provides a reusable evaluation structure, but benchmark accuracy does not establish factual reliability on a different application’s questions, retrieval system or user population.
My takeaway: Evaluate factuality separately for each information source your application uses. Hold retrieval conditions constant when comparing models, retain private tests and inspect per-track errors before relying on an aggregate score.