Back to AI Research

AI Research

Prism-1 scores options independently to prevent order and rival effects

Key Takeaways

  • The architecture enforces two choice invariances in one forward pass, while the paper separately measures answer accuracy.
  • ## A choice should survive a reordered list
  • Prism-1 studies models that select among options such as intents, tools or interface elements in one forward pass.
  • The authors focus on two requirements: reordering the list should not change the decision, and adding an option should not change the relative odds among existing options.
  • Those requirements matter when another component controls the displayed order or the available set.

A choice should survive a reordered list

Prism-1 studies models that select among options such as intents, tools or interface elements in one forward pass. The authors focus on two requirements: reordering the list should not change the decision, and adding an option should not change the relative odds among existing options.
Those requirements matter when another component controls the displayed order or the available set. A model can appear accurate on a fixed presentation while making different choices when the same options are rearranged.

Keep each option's score independent

The paper derives a construction based on scores that depend on one option at a time. Prism-1 reads each option in its own block at shared positions, using the model's ordinary left-to-right attention in a single forward pass. A token that scores an option does not see another option.
This makes the two specified invariances architectural properties. The claim concerns how choices respond to order and option-set changes; it does not establish that the highest-scored option is the correct answer for every task.

Robustness and accuracy are separate measures

The authors evaluate five benchmark sets using Qwen3-8B and Gemma 4 12B. A single-choice prompting baseline trained on the same data still changes 7% of answers under one shuffle, while an added irrelevant option steers 12% of decisions in the reported comparison.
Prism-1 stays within one percentage point of that baseline's accuracy and gains two to four points when counting only answers that survive reordering. The authors report no answer changes from shuffling and no change among existing options from adding an irrelevant rival.
Franklin's reading is to measure presentation sensitivity alongside ordinary accuracy for routing tasks. Hold the underlying state fixed, vary the list and inspect whether a decision changes. The paper supplies a precise invariance mechanism; choosing a suitable task definition and checking the resulting answers remain separate evaluation responsibilities.
Read the Prism-1 paper.

Comments