Large language models can produce remarkably convincing answers while hiding a fatal error inside one line of reasoning. That weakness has created a fundamental problem for advanced AI systems: generating possible solutions is becoming easier, but reliably identifying the correct one remains difficult.
Google DeepMind’s ICLR 2025 study, “Generative Verifiers: Reward Modeling as Next-Token Prediction,” proposes an elegant solution. Instead of training a verifier to compress its judgment into an opaque numerical score, the researchers train it to generate a critique and predict whether the solution is correct through an ordinary Yes or No token.
This apparently small redesign produces striking results. On algorithmic reasoning tasks, Best-of-32 performance rises from 5.0% to 45.3%. On GSM8K grade-school mathematics, it increases from 73.0% to 93.4%. A verifier trained only on grade-school mathematics also generalises to harder competition problems, raising MATH500 performance from 28.0% to 44.6%.
The deeper lesson is clear. Reasoning systems may improve dramatically when evaluation becomes a form of reasoning itself.
