An improving model can expose mistakes that its evaluator no longer recognizes. VeriFine studies that problem in driving and robot navigation reasoning by updating the policy, its training curriculum and its judge together.
The approach uses two improvement loops. One analyzes recurring policy failures and selects training examples. The other requests human guidance when verification limits further progress, then revises the judging rubric. The authors describe improvements under reinforcement fine-tuning for driving and supervised fine-tuning for navigation.
Judge reasoning against the physical context
VeriFine's reference-free judge evaluates a model's reasoning and proposed actions using observable context, including images, ego states and task instructions. It does not require a human-written reasoning answer for every training example.
The rubric separates action criteria from components of the reasoning. In driving, action criteria include instruction consistency and safety. Component criteria examine visual grounding, relevant scene elements and their causal relationship to the proposed action.
A frontier vision-language model acts as a teacher judge. The authors distill its evaluation capability into a smaller student judge for large-scale optimization. This teacher-student arrangement addresses the cost of repeatedly using a frontier model as the training evaluator.
Use failure patterns to choose useful supervision
The policy improvement loop groups evaluation results across scenarios and rubric dimensions. It distinguishes recurring weaknesses from isolated failures, then converts those patterns into criteria for selecting examples from a candidate pool.
Selection has two purposes. A difficult scenario may expose a useful learning opportunity, while an incorrect answer to that scenario may be unsuitable as a supervised demonstration. The system revisits those decisions as both the policy and judge change.
For driving, judge scores supply reinforcement-learning rewards. For navigation, the judge selects demonstrations for supervised training; its score does not enter that optimization objective directly. These are different uses of verification, even though both operate inside the same improvement process.
Calibrate the evaluator when progress stalls
VeriFine monitors progress with a separate reference-based anchor judge built on human annotations. A plateau or disagreement between increasing training reward and anchor performance can trigger judge improvement.
Human experts inspect selected cases, and an agent proposes and tests rubric revisions. Experts can correct individual dimensions or clarify ambiguous criteria. The procedure also permits experts to revise their earlier judgments. The paper calls this process coactive calibration rather than treating a human score as infallible.
Earlier cases remain in a cumulative evaluation set to check that revisions preserve previously acquired verification ability. In the controlled experiments, each new policy starts from the same base initialization with a fixed per-policy training budget. The system's accumulated improvement therefore resides in the evolving supervision process, not an uninterrupted chain of policy-weight updates.
The result is a method for studying feedback quality alongside model training. Improvements on the evaluated reasoning tasks do not establish deployment safety for autonomous driving or physical robots. A training judge's ability to identify errors and a real system's operational safety still require distinct evidence.
Comments