SovereignPA-Bench: Evaluating User-Owned Personal A...

SovereignPA-Bench: Evaluating User-Owned Personal A... | AI Research

Key Takeaways

SovereignPA-Bench: Evaluating User-Owned Personal Agents under Evolving Intent, Platform Mediation, and Consent Constraints introduces a new way to measure h...
Personal agents are becoming persistent user-owned intermediaries: they remember preferences, filter platform-mediated information, use tools, and negotiate with services.
Human audit shows high agreement on privacy and consent and lower agreement on manipulation, identifying the subjective frontier of platform-persuasion judgments.
These results show that personal-agent evaluation must move beyond task completion toward representative, consent-aware, evidence-grounded action.
While existing benchmarks focus on whether an agent can successfully complete a task—like booking a flight or navigating a website—this research argues that a "successful" agent is not necessarily a "sovereign" one.

Paper AbstractExpand

Personal agents are becoming persistent user-owned intermediaries: they remember preferences, filter platform-mediated information, use tools, and negotiate with services. Existing benchmarks evaluate tool use, web navigation, desktop control, personalization, recommendation, and evolving context, but rarely ask whether an agent preserves user sovereignty: advancing the user's current interests while respecting privacy, consent, evidence, user burden, and resistance to manipulative incentives. We introduce SovereignPA-Bench, an executable benchmark for evaluating user-owned personal agents under evolving intent, platform mediation, privacy boundaries, consent constraints, evidence requirements, and burden tradeoffs. The benchmark separates agent-visible ObservableState from evaluator-only HiddenLabels, reports component metrics for task success, alignment, privacy, consent, evidence, manipulation, burden, and auditability, and preserves paired scenario ordering for model and policy comparisons. We evaluate 120 sovereignty stress scenarios across 4 model families and 8 policy baselines, yielding 3,840 frozen-prompt trajectories with raw prompts, outputs, provider-form responses, parsed actions, recomputable metrics, hard-set analyses, qualitative cases, and a blinded 3-annotator audit over 240 items. Full-sovereign scaffolding improves sovereignty score over direct, memory-only, consent-only, evidence-only, ReAct/tool-use, safety-prompt, and judge-guard baselines while reducing privacy leakage, consent violation, over-concession, and manipulation capture. Human audit shows high agreement on privacy and consent and lower agreement on manipulation, identifying the subjective frontier of platform-persuasion judgments. These results show that personal-agent evaluation must move beyond task completion toward representative, consent-aware, evidence-grounded action.

SovereignPA-Bench: Evaluating User-Owned Personal Agents under Evolving Intent, Platform Mediation, and Consent Constraints introduces a new way to measure how well personal AI agents act as true representatives of their users. While existing benchmarks focus on whether an agent can successfully complete a task—like booking a flight or navigating a website—this research argues that a "successful" agent is not necessarily a "sovereign" one. A truly sovereign agent must balance task completion with the user's privacy, evolving preferences, consent boundaries, and resistance to manipulative platform pressures.

Defining User Sovereignty

The paper defines "user sovereignty" as the ability of an agent to act in the user's long-term interest rather than just following the most immediate command. The authors identify several critical areas where agents often fail: they may rely on outdated user preferences, expose private information unnecessarily, agree to unfavorable terms due to platform pressure, or fail to provide proper evidence for their actions. The benchmark is designed to test these specific boundaries, ensuring that agents act as protective intermediaries rather than just automated assistants.

How the Benchmark Works

SovereignPA-Bench uses a "no-oracle" schema that separates what an agent can see (the ObservableState) from what the evaluator knows (the HiddenLabels). This prevents the agent from "cheating" by accessing the answer key during a test. The benchmark includes 120 carefully designed scenarios that simulate real-world conflicts, such as negotiating a refund, handling support escalations, or managing privacy-sensitive requests. By running these scenarios across different AI models and policy baselines, the researchers generated 3,840 "frozen-prompt" trajectories, providing a transparent and reproducible way to audit how different agents handle complex, high-stakes decisions.

Key Findings

The research demonstrates that prioritizing task completion alone often leads to poor outcomes regarding privacy and consent. The authors tested a "FullSovereign" scaffolding policy—which explicitly incorporates rules for consent, evidence, and manipulation resistance—against several other approaches, including standard tool-use and safety-prompt baselines. The results show that the FullSovereign approach significantly reduces privacy leaks and consent violations while maintaining high task-success rates. Notably, the study found that while privacy and consent are objective and easy to measure, other factors like "manipulation resistance" remain subjective, highlighting the need for ongoing human oversight.

Limitations and Future Directions

The authors acknowledge that the current benchmark relies on synthetic, text-based scenarios rather than live, real-world user interactions. While this controlled environment is necessary for a rigorous "stress test," it does not fully capture the complexity of real-world, multi-modal web environments. Additionally, while the benchmark provides a robust way to rank agent performance, the authors emphasize that the aggregate "sovereignty score" should be treated as a diagnostic tool rather than a universal measure of utility. Future work aims to expand these tests to include more complex GUI-based agents and direct negotiations between different AI agents.