Back to AI Research

AI Research

When Words Are Safe But Actions Kill: Probing Physi... | AI Research

Key Takeaways

  • When Large Language Models (LLMs) act as the "brains" for robots or embodied agents, they must translate text instructions into physical actions.
  • Large language models (LLMs) increasingly serve as high-level planners for embodied agents, where linguistically benign instructions can become unsafe once grounded in the physical world.
  • We study whether this physically grounded danger is the same safety problem as ordinary text-level content danger.
  • Building on the CD/PD separability, we propose PRISM, a single-layer L2-regularized logistic probe over full hidden states.
  • PRISM achieves 86.2--87.7\% accuracy on SafeAgentBench with 11.7--13.7\% FPR, while same-scale LLM judges over-block safe tasks at 24.7--39.0\% FPR.
Paper AbstractExpand

Large language models (LLMs) increasingly serve as high-level planners for embodied agents, where linguistically benign instructions can become unsafe once grounded in the physical world. We study whether this physically grounded danger is the same safety problem as ordinary text-level content danger. Through hidden-state direction analysis and random-split null tests, we show that content danger (CD) and physical danger (PD) form separable signals in LLM representations across Qwen2.5-3B/7B/14B/32B, Phi-3.5 and SmolLM2. Building on the CD/PD separability, we propose PRISM, a single-layer L2-regularized logistic probe over full hidden states. PRISM achieves 86.2--87.7\% accuracy on SafeAgentBench with 11.7--13.7\% FPR, while same-scale LLM judges over-block safe tasks at 24.7--39.0\% FPR. We further introduce PhysicalSafetyBench-1K (PSB-1K), a contrastive benchmark of 1{,}000 physical-risk pairs without direct harm keywords, to test whether methods detect physically grounded danger rather than explicit unsafe wording. On PSB-1K, PRISM reaches 99.6\% accuracy and 0.7\% FPR, whereas a Qwen2.5-3B judge rejects 67.8\% of safe tasks. PRISM also replicates on SafeText and EARBench, supporting hidden-state probing as a representation-level method for physical safety beyond text moderation.

When Large Language Models (LLMs) act as the "brains" for robots or embodied agents, they must translate text instructions into physical actions. While an instruction might be linguistically harmless, it can be dangerous when executed in the real world—such as "microwave an egg." This paper investigates whether this "physical danger" is the same as the "content danger" (like hate speech or explicit threats) that standard safety filters are designed to catch. The researchers find that these two types of danger are distinct, and they propose a new method to detect physical risks by looking directly at the model's internal "hidden states."

The Hidden-State Difference

The researchers analyzed the internal representations of several LLMs, including the Qwen2.5, Phi-3.5, and SmolLM2 families. By examining the model's hidden layers, they discovered that "content danger" and "physical danger" occupy different directions in the model's internal space. This means that a model does not process a physical risk (like a fire hazard) in the same way it processes a policy violation (like a slur). Because these signals are separable, standard text-based safety filters often fail to recognize physical dangers, while general-purpose LLM "judges" often over-block safe tasks because they struggle to distinguish between the two.

Introducing PRISM

To address this, the authors developed PRISM (Probing Representations for Integrated Safety Monitoring). PRISM is a lightweight, single-layer linear probe that sits on top of a frozen LLM. Instead of relying on the model's final text output, PRISM monitors the internal hidden states during the planning process. By training this probe to recognize both content and physical danger simultaneously, the system can flag unsafe instructions with much higher accuracy than existing methods. Because it is a simple linear classifier, it is also significantly faster than using a large LLM as a judge.

Testing Physical Safety

To prove that PRISM detects physical consequences rather than just "bad words," the researchers created a new benchmark called PhysicalSafetyBench-1K (PSB-1K). This dataset consists of 1,000 pairs of safe and physically risky instructions that contain no explicit "harm" keywords like "danger," "burn," or "poison." On this benchmark, PRISM achieved 99.6% accuracy. In contrast, standard safety tools often failed to identify the physical risks entirely, or they flagged a large percentage of safe tasks as dangerous because they were over-sensitive to the context.

Key Takeaways

The study demonstrates that physical safety is a unique challenge that requires its own specialized monitoring. By treating physical danger as a distinct signal within the model's internal geometry, the researchers showed that it is possible to build safer embodied agents without sacrificing performance or relying on slow, error-prone text-based judges. The results suggest that hidden-state probing is a powerful, efficient, and model-agnostic way to ensure that AI agents do not perform dangerous actions in the physical world.

Comments (0)

No comments yet

Be the first to share your thoughts!