Back to AI Research

AI Research

Personalized Privacy Control in LLMs via Attention... | AI Research

Key Takeaways

  • Personalized Privacy Control in LLMs via Attention Head Intervention introduces a method to align Large Language Model (LLM) responses with individual user p...
  • The rise of agentic AI enables LLMs to access diverse user data, raising critical privacy concerns.
  • Prior work on contextual privacy studies whether LLMs regulate information disclosure according to context-dependent norms.
  • However, acceptable disclosure boundaries may vary across users even within the same context.
  • To address this limitation, we introduce \textit{personalized privacy}, which incorporates user-specific disclosure preferences into privacy control.
Paper AbstractExpand

The rise of agentic AI enables LLMs to access diverse user data, raising critical privacy concerns. Prior work on contextual privacy studies whether LLMs regulate information disclosure according to context-dependent norms. However, acceptable disclosure boundaries may vary across users even within the same context. To address this limitation, we introduce \textit{personalized privacy}, which incorporates user-specific disclosure preferences into privacy control. We further present P3Bench~(\textbf{P}ersonalized \textbf{P}rivacy \textbf{P}reservation \textbf{Bench}mark), a novel benchmark extending contextual privacy policies with personalized disclosure policies. Experiments show that prompt-based policies fail to reliably enforce personalized privacy policies, with Qwen2.5-7B and Gemma3-4B showing average policy ignorance ratios of 51.25\% and 74.28\%, respectively. Finally, to address this problem, we propose \textsc{Repair}, a robust inference-time attention head intervention method that adjusts disclosure behavior toward policy-consistent responses. Our method significantly improves adherence to user-specific privacy preferences by reducing cases where the model fails to follow the given policy.

Personalized Privacy Control in LLMs via Attention Head Intervention introduces a method to align Large Language Model (LLM) responses with individual user privacy preferences. While existing research focuses on contextual privacy—where models follow general rules based on the situation—this paper addresses the fact that users have different comfort levels regarding what information they share, even within the same context.

The Challenge of Personalized Privacy

The authors, Junseok Kim, Nakyeong Yang, and Kyomin Jung, argue that standard prompt-based policies are often ignored by LLMs. To measure this, they developed P3Bench, a benchmark that tests how models handle four specific privacy settings: Privacy-Max, Contact-Open, Health-Open, and Preference-Open.
Experiments using Qwen2.5-7B and Gemma3-4B revealed that these models frequently fail to follow user-provided privacy instructions. The researchers observed "policy ignorance ratios" of 51.25% for Qwen2.5-7B and 74.28% for Gemma3-4B. The models often defaulted to their own internal biases, leading to either "over-refusal" (refusing to share information the user actually permitted) or "over-sharing" (disclosing information the user wanted to keep private).

The Repair Intervention Method

To solve this, the researchers proposed Repair, an inference-time technique that intervenes on specific attention heads within the LLM. The process follows three steps: 1. Identification: The team uses linear probes to identify which attention heads are responsible for processing privacy-related information. 2. State Prediction: At inference time, the model uses these selected heads to predict the required disclosure state (whether to answer or refuse) based on the user's specific policy. 3. Intervention: The model applies "state-adaptive" interventions. If the policy requires a refusal, the model patches the activation of the selected heads toward a refusal-oriented representation. If the policy allows disclosure, it uses a steering direction to ensure the model provides the requested information without triggering a refusal.

Performance and Results

The study indicates that Repair significantly improves adherence to user-specific privacy preferences compared to direct prompting. By adjusting the model's internal representations rather than relying on text prompts, the method reduces both over-refusal and over-sharing behaviors. The researchers found that this approach works without the need for retraining the model, allowing for more flexible and robust privacy control across different user profiles and policy configurations.

Considerations

The authors note that LLMs possess inherent "default disclosure policies" formed during pretraining and instruction tuning. These internal priors are often strong enough that simple prompting cannot override them, which is why the researchers focused on mechanistic intervention at the attention-head level. The effectiveness of Repair relies on the ability to accurately identify and steer these specific, policy-relevant components within the model's architecture.

Comments (0)

No comments yet

Be the first to share your thoughts!