Back to AI Research

AI Research

Terminal-RSI trains terminal agents through verified rewrites and policy refinement

Key Takeaways

  • The research alternates trajectory rewriting with policy updates, separating partial progress from strict task completion.
  • ## The problem: progress is not completion
  • Terminal-RSI studies how a terminal agent can turn feedback into more reliable task completion.
  • The researchers focus on a familiar gap in agent evaluation: a sequence of useful local changes can still fail the final verifier.
  • Training needs to recognize progress without treating an unfinished task as a success.

The problem: progress is not completion

Terminal-RSI studies how a terminal agent can turn feedback into more reliable task completion. The researchers focus on a familiar gap in agent evaluation: a sequence of useful local changes can still fail the final verifier. Training needs to recognize progress without treating an unfinished task as a success.
Their system alternates two phases. Recursive Self-Rewrite revises verified trajectories, while Recursive Policy-Refine updates the policy and inherits a critic from earlier training. The five-stage sequence starts from Qwen3.8-27B and moves through two rounds of each method to produce Terminal-RSI-27B.

How the training signal is structured

The paper introduces a turn-boundary advantage-estimation method that keeps token-specific baselines while applying decay across interaction turns. It also uses family-level dense rewards to calibrate semantic progress and a success-prioritized reward to distinguish strict completion from bounded partial credit.
A warm-up critic, critic handoff and stage-specific learning rates coordinate the phases. The contribution is therefore a particular combination of rewriting, feedback and training structure. It is not evidence that an already deployed terminal agent can improve itself indefinitely without a verifier or a controlled training process.

Read the reported outcomes separately

The authors report 19.84% on Terminal-Bench 4.0, 86.52% on Terminal-Bench 2.1 and 16.95% on Terminal-Bench 3.0. Those are different benchmark results, not interchangeable estimates of general reliability. On LHTB, strict completion increased from zero to five tasks out of 46, while process reward rose from 0.21 to 0.37.
Franklin's reading is to examine strict success alongside partial-progress measures. The LHTB figures show why: improved intermediate behavior can coexist with a low final completion count. For researchers reproducing the approach, the verifier, task family and retained training stage are essential context for interpreting a score.
Read the Terminal-RSI paper.

Comments