Back to AI Research

AI Research

LACO: Adaptive Latent Communication for Collaborati... | AI Research

Key Takeaways

  • Collaborative driving aims to improve road safety and efficiency by allowing connected vehicles to share information and coordinate their actions.
  • Collaborative driving aims to improve safety and efficiency by enabling connected vehicles to coordinate under partial observability.
  • Recent approaches have evolved from sharing visual features for perception to exchanging language-based reasoning through foundation models for behavioral coordination.
  • To address these challenges, we analyze latent communication in collaborative driving under inherent limitations of multi-agent settings.
  • Our analysis reveals agent identity confusion, where direct fusion of latent states entangles decision representations across vehicles.
Paper AbstractExpand

Collaborative driving aims to improve safety and efficiency by enabling connected vehicles to coordinate under partial observability. Recent approaches have evolved from sharing visual features for perception to exchanging language-based reasoning through foundation models for behavioral coordination. Though communicating in language provides intuitive information, it introduces two challenges: high latency caused by autoregressive decoding and information loss caused by compressing rich internal representations into discrete tokens. To address these challenges, we analyze latent communication in collaborative driving under inherent limitations of multi-agent settings. Our analysis reveals agent identity confusion, where direct fusion of latent states entangles decision representations across vehicles. Motivated by this, we propose LACO, a training-free \textbf{LA}tent \textbf{CO}mmunication paradigm that seamlessly adapts pretrained driving models to collaborative settings. LACO introduces Iterative Latent Deliberation (ILD) for latent reasoning, Cross-Horizon Saliency Attribution (CHSA) for communication-efficient information selection, and Structured Semantic Knowledge Distillation (SSKD) to stabilize ego-centric decision making. Closed-loop experiments in CARLA show that LACO notably reduces communication and inference latency while maintaining strong collaborative driving performance.

Collaborative driving aims to improve road safety and efficiency by allowing connected vehicles to share information and coordinate their actions. While recent methods have used language-based models to exchange reasoning, these approaches often suffer from high latency due to the slow process of generating text and information loss caused by compressing complex visual data into simple tokens. LACO (Adaptive Latent Communication) is a new, training-free paradigm that allows autonomous vehicles to communicate by directly sharing their internal "thinking" states—specifically their transformer Key-Value (KV) caches—bypassing the need for language generation entirely.

Solving the Latent Communication Challenge

When vehicles share their internal latent states, they face a problem called "agent identity confusion." If one vehicle receives too much raw internal data from another, it may become over-reliant on that information, leading to erratic driving or "policy hijacking" where the ego vehicle ignores its own observations in favor of the collaborator's. LACO addresses this by ensuring that communication is both selective and structured, preventing the entanglement of decision-making processes between different vehicles.

How LACO Works

LACO utilizes three core technical components to enable efficient and stable collaboration:

  • Iterative Latent Deliberation (ILD): Instead of using language, the vehicle performs a series of internal reasoning steps that embed its spatial awareness and driving intent directly into its latent memory. This creates a compact, semantically rich trace of the vehicle's "thought process." * Cross-Horizon Saliency Attribution (CHSA): To keep communication fast, LACO does not send all the data it generates. It identifies and retains only the most "salient" or important tokens—those that contribute most significantly to the vehicle's reasoning—thereby reducing bandwidth usage without losing critical context.

  • Shallow-Stream Knowledge Distillation (SSKD): To prevent identity confusion, LACO only shares data from the "shallow" (early) layers of its neural network. These layers contain general environmental context, while the "deep" (later) layers, which are responsible for final control and specific maneuvers, remain private to each vehicle.

Results and Performance

Experiments conducted in the CARLA simulation environment demonstrate that LACO significantly improves collaborative driving performance compared to traditional methods. By removing the need for autoregressive language decoding, LACO notably reduces both communication and inference latency. This allows vehicles to coordinate their movements more quickly and reliably, maintaining high safety standards while operating under the strict real-time constraints required for autonomous driving.

Key Considerations

LACO is designed to be a training-free framework, meaning it can be applied to existing pretrained driving models without the need for resource-intensive retraining. By focusing on the exchange of structured latent representations rather than natural language, it provides a more faithful and efficient channel for multi-agent coordination, effectively bridging the gap between individual vehicle autonomy and collective intelligence.

Comments (0)

No comments yet

Be the first to share your thoughts!