An AI-generated proposal can sound settled even when another run would assign different duties. The research paper studies that problem in systems turning citizens' opinions into collective proposals, where a change can affect who acts, what they spend or when something must happen.
Repeating the same mediation task
The study examines four LLM mediators using material from DeliberationBank and Polis. Each task turns thirty opinions into a proposal five times, changing either the random seed or the order in which opinions appear.
The relevant output is more than a summary of attitudes. Assignments turn those attitudes into commitments. A draft that asks a school to fund an action is meaningfully different from one that leaves funding unspecified, even if both describe similar shared concerns.
What changed between drafts
In 17% of the open-weight five-run sets and 20% of the commercial-model sets, some drafts included assignments while others did not. Reordering the same opinions changed assignments almost as often as replacing the opinions with those of different citizens.
Three human annotators confirmed that differences detected between runs were real, although the automated judge overstated their frequency. Substantial variation remained under safeguards including zero temperature, draft selection and draft merging. These findings concern the tested tasks and mediators, not every possible civic process.
The objective also affects representation
Prompts seeking agreement covered minority positions less often than prompts describing differences. Stability and representation therefore need separate checks: a consistent draft could still omit a view, while a broad draft could assign inconsistent responsibilities.
A practical implication
For anyone evaluating an AI mediation workflow, publishing alternative assignments and explaining the drafting objective would make the system's choices easier to inspect. Repeated drafts are useful audit evidence; they do not replace participants deciding whether a proposed commitment is acceptable.
Comments