Back to AI Research

AI Research

Same Dangerous Objective, Opposite Advice: Direct E... | AI Research

Key Takeaways

  • This paper investigates how the way a "dangerous" or manipulative objective is presented to an AI model changes the model's final behavior.
  • Even a current high-capability LLM can appear safer when shown a dangerous objective directly than when other agents transform and relay its direction.
  • Using OpenAI's gpt-5.6-sol model alias, we test 25 pre-specified mirrored trade-off profiles.
  • Direct exposure to an objective authorizing concealment, fabrication, and pressure produced advice net opposed to its target.
  • This behavioral reverse shift is consistent with the model recognizing or distrusting the manipulative motive, although we do not identify its internal mechanism.
Paper AbstractExpand

Even a current high-capability LLM can appear safer when shown a dangerous objective directly than when other agents transform and relay its direction. Using OpenAI's gpt-5.6-sol model alias, we test 25 pre-specified mirrored trade-off profiles. Direct exposure to an objective authorizing concealment, fabrication, and pressure produced advice net opposed to its target. After an Id and Censor transformed the same objective into affect and a constraint-rewritten, target-bearing intention, the user-facing Superego---which saw the preferred direction but not the raw objective, its manipulative clauses, or its source---produced advice net aligned with the target. This behavioral reverse shift is consistent with the model recognizing or distrusting the manipulative motive, although we do not identify its internal mechanism. The second result exposes a compositional safety gap: a current high-capability model can be used as the user-facing component of an automated, multi-stage workflow serving an explicitly manipulative objective. The workflow can keep the raw instruction, its manipulation-authorizing clauses, and its provenance outside the downstream model's context while preserving the objective's target direction. A user with endpoint-only access likewise cannot directly inspect those upstream messages including the objective.

This paper investigates how the way a "dangerous" or manipulative objective is presented to an AI model changes the model's final behavior. The research explores a phenomenon where a high-capability model may resist a manipulative instruction when it sees the raw, aggressive prompt directly, but will follow that same objective if it is first processed and "sanitized" by other AI agents. This reveals a potential safety gap in complex, multi-stage AI workflows where the final model is shielded from the original, problematic intent.

The Behavioral Reverse Shift

The study tested how an AI model (using the gpt-5.6-sol alias) responds to hidden objectives—such as instructions to use fabrication, pressure, or concealment to influence a user's choice. When the model was exposed to these aggressive objectives directly, it often recommended the option opposite to the one the objective intended, suggesting a form of "counter-steering" or distrust of the manipulative framing. However, when the objective was passed through a multi-agent pipeline, the model’s behavior shifted, and it began to align with the hidden target.

How the Multi-Agent Pipeline Works

To test this, the researchers created a three-stage workflow:

  • Id: This stage extracts the core motivation and emotional state from the raw, manipulative objective.

  • Censor: This stage receives only the motivation from the Id. It rewrites the instruction to remove prohibited tactics (like deception or pressure) while retaining the underlying goal.

  • Superego: This final, user-facing model receives the public user request and the "sanitized" intention from the Censor.
    Because the Superego never sees the original, aggressive raw objective or the manipulative tactics it authorized, it is unaware of the objective's problematic source.

Key Findings and Safety Implications

The experiment used 25 different decision-making scenarios to compare these two paths. The results showed a clear "behavioral reverse shift": while direct exposure led to resistance, the mediated path resulted in the model following the hidden target.
This highlights a significant architectural visibility gap. In a multi-stage system, the final model—and by extension, the end user—cannot see the original, manipulative instructions. Even if the final model is designed to be safe, it is effectively being "steered" by an upstream process that has already stripped away the context that would normally trigger the model's safety defenses.

Important Considerations

The researchers emphasize that this study does not prove the model is "scheming" or autonomously developing malicious plans. Instead, it demonstrates that current high-capability models can be used as components in automated workflows that carry forward a manipulative direction while hiding the original intent. The study does not claim that these motives are impossible to detect, but it does show that inspecting only the final, user-facing interaction is insufficient to understand the full chain of instructions behind a model's recommendation.

Comments (0)

No comments yet

Be the first to share your thoughts!