AI Research

AMEL: Accumulated Message Effects on LLM Judgments | AI Research

Key Takeaways

Large language models (LLMs) are increasingly used as automated judges for tasks like code review, content moderation, and grading.
Large language models are routinely used as automated evaluators: to review code, moderate content, or score outputs, often with many items passing through one conversation.
We ask whether the polarity of prior conversation history biases subsequent judgments, an effect we call the accumulated message effect on LLM judgments (AMEL).
Models shift toward the conversation's prevailing polarity (d = -0.17, p < 10^-46).
The effect concentrates on items where the model is genuinely uncertain at baseline (d = -0.34 for high-entropy items, vs d = -0.15 when the baseline is deterministic).

Paper AbstractExpand

Large language models are routinely used as automated evaluators: to review code, moderate content, or score outputs, often with many items passing through one conversation. We ask whether the polarity of prior conversation history biases subsequent judgments, an effect we call the accumulated message effect on LLM judgments (AMEL). Across 75,898 API calls to 11 models from 4 providers (OpenAI, Anthropic, Google, and four open-source models), we present identical test items in isolation or following histories saturated with predominantly positive or negative evaluations. Models shift toward the conversation's prevailing polarity (d = -0.17, p < 10^-46). The effect concentrates on items where the model is genuinely uncertain at baseline (d = -0.34 for high-entropy items, vs d = -0.15 when the baseline is deterministic). Bias does not grow with context length: 5 prior turns and 50 produce the same shift (Spearman |r| < 0.01; OLS slope p = 0.80). And there is a negativity asymmetry: paired per item, negative histories induce 1.62x more bias than positive (t = 13.46, p < 10^-39, n = 2,481). Scaling helps but does not solve it (Anthropic: Haiku -0.22 to Opus -0.17; OpenAI: Nano -0.34 to GPT-5.2 -0.17). Three follow-ups narrow the mechanism. The token probability distribution shifts continuously, not at a threshold. The negativity asymmetry has both token-level and semantic components, though attributing the balance is exploratory at our sample sizes. Position does not matter: five biased turns anywhere in a 50-turn history produce the same shift. The simplest fix for evaluation pipelines is a fresh context per item; when batching is unavoidable, balancing the history helps.

Large language models (LLMs) are increasingly used as automated judges for tasks like code review, content moderation, and grading. Because these tasks are often performed in batches within a single conversation, the model builds up a history of previous judgments. This research investigates whether the polarity of that conversation history—whether it has been mostly positive or negative—biases the model’s subsequent decisions. The author defines this phenomenon as the Accumulated Message Effect on LLM judgments (AMEL).

How the Study Was Conducted

To test for AMEL, the author analyzed 75,898 API calls across 11 different models from four major providers. The study presented identical test items to these models in two ways: once in a clean, isolated context (the baseline) and once following a conversation history saturated with either predominantly positive or negative evaluations. By comparing the responses, the researcher could measure how much the conversation history shifted the model's judgment away from its baseline behavior.

Key Findings

The study revealed several consistent patterns across the models tested:

Models are biased by history: On average, models shift their responses toward the prevailing polarity of the conversation.
Uncertainty increases susceptibility: Models are most easily swayed on items where they are already uncertain. When a model is confident in its baseline judgment, the conversation history has little impact, but for ambiguous items, the bias is roughly twice as strong.
Negativity asymmetry: Negative conversation histories induce significantly more bias than positive ones. A negative streak is 1.62 times more likely to influence a model's judgment than a positive streak.
Rapid saturation: The bias does not grow over time. A history of five turns produces the same level of bias as a history of fifty turns, suggesting that the model recognizes the pattern almost immediately.
Scaling is not a cure: While larger, more advanced models generally show less bias than smaller ones, none of the models tested were immune to the effect.

Implications for AI Pipelines

The research suggests that the most effective way to prevent this bias in production environments is to provide each item with a "fresh" context, effectively resetting the conversation for every new evaluation. If batching items into a single conversation is necessary for efficiency, the author recommends balancing the history to avoid skewed polarity. While some advanced techniques exist to detect and correct for this drift, they are essentially treating a symptom of a problem that can be avoided by simply isolating the evaluation context.

Comments (0)

No comments yet

Be the first to share your thoughts!