StateBridge: Training-free Hidden-state Alignment f... | AI Research

StateBridge is a training-free communication method designed to improve how Large Language Model (LLM) agents collaborate in multi-agent systems. By allowing agents to share continuous hidden states directly rather than converting messages into discrete text, the system preserves information that is typically lost during the tokenization process.

The Communication Bottleneck

In standard multi-agent systems, agents communicate by generating text. This forces the sender to compress its internal continuous hidden states into discrete tokens, which the receiver must then interpret. This process acts as a bottleneck, discarding nuanced information—such as confidence levels or alternative reasoning paths—that token identities cannot capture. While some existing methods attempt to bypass this by transferring internal states directly, they often require training specific projectors for each model or injecting states across every layer of the transformer, which limits their portability and increases memory usage.

How StateBridge Works

StateBridge enables agents to communicate using a continuous prefix that is injected directly into the receiver’s input embedding layer. The process involves three primary steps:

• Procrustes Alignment: The sender’s final-layer hidden states are aligned to the receiver’s input space using a closed-form orthogonal transformation. This preserves the geometric relationships between the sender’s hidden states without requiring model updates.

• Norm Calibration: Because final-layer hidden states often have significantly larger norms than input embeddings, the system calibrates the aligned vectors to match the typical scale of the receiver’s vocabulary embeddings.

• Vocabulary Anchoring: The aligned vectors are moved slightly toward their nearest vocabulary embeddings using cosine similarity. This ensures the continuous prefix remains compatible with the receiver’s pretrained input distribution.
  This approach is computationally efficient, as the alignment is performed in closed-form and the memory overhead is limited to the final layer of the transformer, rather than requiring the storage of states across all layers.

Performance and Results

Researchers evaluated StateBridge across four models from two families (Qwen3 and OLMo3) on tasks including mathematical reasoning, code generation, and question answering. The method achieved the best or tied-best score on 22 out of 26 model-task pairs.
The results indicate that StateBridge consistently outperforms both standard text-based communication and existing latent communication baselines like KV-cache transfer. Notably, the method showed significant gains on challenging benchmarks such as GPQA and AIME, suggesting that the continuous prefix carries semantic information that standard text communication fails to convey.

Considerations

StateBridge is designed as a universal interface that does not require architectural modifications or retraining, making it highly portable across different LLM families. However, the researchers noted that performance gains are not uniform across all tasks; for example, on the GSM8K benchmark using Qwen3 models, StateBridge did not outperform the strongest baseline. The effectiveness of the system relies on the assumption that the sender and receiver share the same underlying LLM, and the alignment process is optimized for homogeneous multi-agent environments.