Never Stop Thinking: Continuous-Time Language Agents | AI Research

Voice agents powered by Large Language Models (LLMs) typically operate in a rigid "listen-think-speak" loop, which creates awkward silences and prevents the agent from reacting while the user is still talking. This paper introduces a method to enable "continuous-time cognition," allowing agents to think while they listen and while they speak. By using a lightweight orchestrator, the researchers enable unmodified text models to handle interruptions and process information in real-time, significantly reducing latency and making interactions feel more natural.

Making Agents Continuous

The core of this approach is an "interrupt-and-resume" mechanism. Instead of waiting for a user to finish speaking, the system treats environment observations—such as partial speech transcripts, tool results, or user interruptions—as new inputs injected into the model's ongoing reasoning stream. When an event occurs, the orchestrator forces the model to close its current "thinking" block, incorporates the new information, and resumes reasoning. Because LLMs generate tokens much faster than humans speak, the system uses the time the user spends talking as "free compute" to plan responses or trigger tools before the user has even finished their sentence. The ai search story also surfaces in OpenAI Says AI Found Possible Navier–Stokes..., adding another angle.

Measuring Performance with ReactiveBench

To determine if this continuous thinking actually improves performance, the authors created ReactiveBench. This benchmark includes 120 interactive scenarios and 200 verifiable streaming tasks. A key finding from this benchmark is that standard LLM-based "judges" are unreliable for evaluating interactive agents. When a judge can see the agent's internal "thinking" process, it tends to reward the agent simply for showing its work, even if the actual spoken output is worse. By using objective, pre-registered checklists that score only what the user hears, the researchers found that "judge-trained" models often perform worse when they think, as their reasoning blocks end up displacing the actual content they should be speaking.

The Importance of Training Signals

The paper demonstrates that orchestration alone makes continuous-time interaction possible, but training is what makes it good. The researchers tested five different training signals and discovered that the source and structure of the reward are critical:

• Source: Training on verifiable objectives (like factual correctness) turns thinking from a distraction into a helpful tool, whereas training on subjective judge scores often leads to models that prioritize "looking smart" over being helpful.

• Structure: If a reward is too uniform—such as always rewarding brevity—the model may sacrifice complex tasks like multi-hop tool chaining, where a longer, more detailed process is actually required.

• Optimizer: Preference-based optimization often forces the model to trade one goal for another. In contrast, using on-policy reinforcement learning (RL) over a type-shaped reward allows the model to improve across multiple correctness axes simultaneously, leading to significantly higher completion rates. The ai search story also surfaces in Qwen Developers Open-Source Local-First Search Layer..., adding another angle.

Key Takeaways

Continuous-time cognition offers clear benefits in latency, efficiency, and the ability to handle dynamic interruptions. However, the researchers caution that simply adding "thinking" to an agent is not a universal fix. If the training signal is not carefully shaped to prioritize the quality of the final spoken output, the agent may become less effective. The study concludes that for voice agents to truly feel alive, they require both the right orchestration to handle the flow of conversation and a verifiable training objective that ensures their internal reasoning directly supports the user's needs. The same ai evaluation question is explored in MAPLE, which adds a research perspective. as detailed in the full paper on Arxiv