Back to AI Research

AI Research

Structured Recipe Optimization makes post-training choices reusable across experiments

Key Takeaways

  • SRO combines an adaptive recipe space with evidence-guided candidate construction and multi-objective trial selection.
  • ## Make the training recipe an explicit object
  • The [Structured Recipe Optimization paper](https://arxiv.org/abs/2610.18572) studies automated post-training under a limited experimental budget.
  • The choices include data sources, processing, composition across domains and the training configuration itself.
  • The authors identify a gap between searching a fixed predefined space and letting an open-ended agent edit individual trials.

Make the training recipe an explicit object

The Structured Recipe Optimization paper studies automated post-training under a limited experimental budget. The choices include data sources, processing, composition across domains and the training configuration itself.
The authors identify a gap between searching a fixed predefined space and letting an open-ended agent edit individual trials. A fixed space can miss choices suited to the task. Trial-level edits can discover useful options without representing them clearly enough to reuse and recombine across later experiments.
SRO instead builds a task-adapted recipe space and keeps its choices explicit. Its Recipe-Space Designer uses a language model to construct that space. A Multi-Domain Recipe Optimizer then combines structured evidence from trials, language-model candidate construction and multi-objective Bayesian trial selection.

Results belong to the evaluated training setups

The paper reports the highest average score across its evaluated target-model and agent-model combinations. In joint post-training across three capability domains and six benchmarks, SRO reaches 41.1 against 15.4 for the strongest baseline on Qwen3-1.7B with GPT-5.4.
Another reported comparison reaches 53.8 against 40.8 on Qwen3-4B with DeepSeek-V4-Flash. These are aggregate scores under the paper's evaluation setup, not percentages of all tasks an arbitrary model can solve.
The experiments support the proposed approach in the tested settings. They do not establish that every new recipe space will be productive or that the reported margin transfers unchanged to another dataset, training budget or target model.

Preserve the decisions behind each trial

Franklin analysis: compare automated training systems with a shared experimental budget and a held-out evaluation. Save the recipe, the reason for proposing it and the trial result together.
That record lets a team distinguish a reusable discovery from a lucky trial. It also makes it possible to investigate whether gains came from better data choices, a different training configuration or simply more opportunities to search.

Comments