When an ARC solver improves, the change may come from training data, inference procedure or how the available GPUs are used. A study of rules, traces and routing statistics examines those factors across three open models under a constrained offline setup.
A fixed environment for different approaches
The solver runs on four L4 GPUs for at most 12 hours. The authors report 189 runs on 120 hidden semi-private tasks, alongside comparisons on public tasks.
For the NVARC Qwen3-4B grid model, they built a corpus of 12,000 executable rules and 95,993 synthetic tasks with checks intended to reject ambiguous or shortcut-solvable examples. More synthetic data did not automatically produce the strongest result: the reported synthetic checkpoints did not improve on the model trained with real task records.
Why the inference procedure matters
For Qwen3.8-27B, the work includes answer-verified reasoning traces and an inference procedure that checks a proposed Python program against demonstrations. The authors attribute most of that configuration’s improvement to the inference procedure.
For the mixture-of-experts model, the study also examines expert pruning and cache capacity. Those experiments show how memory and concurrency constraints can affect a solver’s available work within the time limit.
The reported results therefore belong to complete solver configurations. They are not isolated measurements of the value of one additional dataset.
A useful way to evaluate an improvement
Teams building reasoning systems can use this paper to plan ablations: hold the hardware and time budget fixed, then change training data, checking tools and scheduling separately. Report whether an apparent gain survives on hidden tasks and across repeated runs.
The practical lesson is to preserve the whole recipe behind a score. A training-data claim becomes much easier to assess when the reader can distinguish what the model learned from what its runtime and verification tools contributed.
Comments