PUBG Ally: A Conversational Embodied Agent as an AI... | AI Research

What the paper is about

We introduce PUBG Ally, an embodied agent for PUBG: BATTLEGROUNDS that can reason, act autonomously, and play alongside players as a voice-enabled teammate. Building such a teammate requires combining two difficult capabilities: it must perceive and respond to a constantly changing game world under strict latency constraints while interacting naturally with players, keeping its speech synchronized with its actions. Ally therefore combines agentic tool use with real-time game control. A language-model agent uses a controlled interface to inspect game information, interpret player speech, maintain context, decide what to say, and issue high-level action choices that steer a faster control layer for movement, combat, and recovery. Because the player's and Ally's speech and actions continually shape each other and the course of the match, training requires data from actual gameplay. We therefore collect data across nearly 39k sessions in which real players play alongside Ally, recording gameplay, player speech, agent decisions, tool use, actions, and player feedback, and use these records for iterative training. To evaluate teammate quality, we use player feedback and preference comparisons to identify gaps between offline evaluations and player preferences, and iteratively refine the evaluation criteria. Deploying Ally in live service further requires low-latency on-device execution and safeguards for player-facing communication, which we address through model compression, context compaction, targeted safety training, runtime guardrails, and memory redaction. During the live service, we surveyed players in 141 countries. Among respondents whose play with Ally was confirmed in game records, positive responses exceeded negative responses by 25.1 percentage points when asked whether they would recommend Ally, with players describing Ally not only as a tool but also as a teammate or companion.

What it covers

\uselogo \paperdate September 24, 2026 PUBG Ally: A Conversational Embodied Agent as an AI Teammate PUBG Ally Team † Abstract We introduce PUBG Ally (hereafter Ally ), an embodied agent for PUBG: BATTLEGROUNDS that can reason, act autonomously, and play alongside players as a voice-enabled teammate. Building such a teammate requires combining two difficult capabilities: it must perceive and respond to a constantly changing game world under strict latency constraints while interacting naturally with players. These demands compound each other because speech and action must remain synchronized, so the agent’s communication stays consistent with what it is doing in the game. To address these challenges, we design Ally to combine agentic tool use with real-time game control. A language-model agent uses a controlled interface to inspect relevant game information, interpret player speech, maintain context, decide what to say, and issue high-level action choices that steer a faster control layer for time-sensitive movement, combat, and recovery. Training Ally poses a distinct data challenge: the human player’s and Ally’s speech and actions continually shape each other’s behavior and the course of the match, requiring data from actual gameplay. We therefore collect data across nearly 39k sessions in which real players play alongside Ally, recording gameplay, player speech, agent decisions, tool use, actions, and player feedback. Gameplay and interaction records support iterative training and system improvement. To evaluate teammate quality, we use player feedback and preference comparisons to identify gaps between offline evaluations and actual player preferences, and iteratively refine the evaluation criteria to better reflect what players value in a teammate. Deploying Ally in live service further requires low-latency on-device execution and safeguards for player-facing communication. We address these requirements through model compression, context compaction, targeted safety training, runtime guardrails, and memory redaction. During the live service, we surveyed players in 141 countries. Among respondents whose play with Ally was confirmed in game records, positive responses exceeded negative responses by 25.1 percentage points when asked whether they would recommend Ally, with players describing Ally not only as a tool but also as a teammate or companion. † † footnotetext: † A detailed list of contributors and acknowledgments can be found in Section 10 of this paper. Figure 1 : PUBG Ally as a Co-Playable Character (CPC). Ally shares a live match with a human player, communicating through voice while taking in-game actions as an AI teammate. The figure highlights representative interactions throughout a match: suggesting a drop location, supporting the player during looting, calling out enemies while rotating, and deploying smoke to revive a downed player. Together, these examples illustrate how Ally observes the evolving game state, coordinates with the player, and acts autonomously as a co-playable character in the shared game world. \printoutline 1 Introduction We introduce PUBG Ally (hereafter Ally), a voice-enabled embodied agent deployed in PUBG: BATTLEGROUNDS (PUBG) as an AI duo partner. PUBG is a battle-royale game in which players scavenge for weapons and supplies, navigate a shrinking safe zone, and fight to be the last surviving player or team. In duo mode, two teammates coordinate their movements and tactics, share resources, and support each other in combat to survive together. Ally joins a live match alongside a human player, reasons about the game, acts autonomously, and communicates through voice as their teammate. As illustrated in Figure 1 , Ally can coordinate a drop location, follow voice commands while looting, call out enemies, support the player during combat, and revive them when they are downed. For example, when the player is knocked during a firefight, Ally can assess the situation, deploy smoke for cover, move to the player, and attempt a revive. We refer to this type of agent as a co-playable character (CPC): an embodied game agent that communicates and coordinates with human players while acting alongside them in a shared game world. This report focuses on a scoped PUBG duo setting in which one human player is paired with Ally on the Sanhok map in battle-royale matches (Section 2 ). Building such a teammate requires combining two challenging capabilities: real-time embodied gameplay and voice interaction . Many prior game agents have primarily been developed to excel at autonomous gameplay ( Vinyals et al., 2019 ; Jaderberg et al., 2019 ; OpenAI et al., 2019 ) , whereas Ally must also communicate and coordinate with a human teammate through ongoing voice interaction. As an embodied agent, Ally must perceive and react to a large, noisy, and continuously changing world, with especially strict latency requirements in a fast-paced game such as PUBG. As a conversational agent, it must understand player speech and respond naturally during an ongoing interaction. Moreover, these challenges do not merely add together: Ally must keep its speech and actions synchronized , so that what it says remains consistent with what it observes, decides, and does as the match evolves. This creates a distinct challenge for a conversational embodied teammate: it must communicate intentions that a human partner can act on while adapting its own behavior to a game world that continues to change throughout the interaction. To address this challenge, Ally uses a language-model agent that interacts with the game and the player through a bounded tool interface, as illustrated in Figure 2 . Rather than relying on a fixed set of observations ( OpenAI et al., 2019 ; Vinyals et al., 2019 ; Fan et al., 2022 ) , Ally observes game information relevant to the current situation ( Yao et al., 2023 ) . Through the interface, it can query relevant game state, retrieve memory and game knowledge, access player communication, and issue high-level actions. The interface converts raw, rapidly changing game signals into compact textual observations and restricts the actions available to the agent, keeping both perception and action within a controlled context. Ally uses these tools to gather the context relevant to the current situation and decide when and what to communicate, and whether to maintain or update its current high-level action. This helps keep its communication consistent with its understanding of the situation and its intended behavior. The bounded interface gives the LM agent control over high level decisions, but executing them at PUBG’s control frequency requires a faster control layer. Ally therefore separates deliberate LM reasoning from fast control, following the System 1 and System 2 distinction ( Kahneman, 2011 ) . The LM agent serves as System 2, interpreting player intent, coordinating with the player, producing speech, and selecting high-level actions. A deterministic behavior-tree layer serves as System 1, translating those decisions into movement, combat, recovery, and other latency-critical behaviors ( Isla, 2005 ; Colledanchise and Ögren, 2018 ) . The two layers are coupled rather than independent: the LM agent sets the current intent, and the behavior tree carries it out while reacting immediately to changes in the game. This allows deliberate LM-based decisions to guide Ally’s moment-to-moment behavior without placing language-model inference directly in the real-time control loop. Training such a teammate poses a distinct data challenge: the human player’s and Ally’s speech and actions continually shape each other’s behavior and the course of the match. For example, when Ally offers to cover the player, the player may advance, creating a new combat situation that Ally must respond to. Capturing these evolving exchanges requires data from real matches that connects what the player says, how the agent responds and acts, and what happens next. To collect such data at scale, we organized full matches between human players and Ally at a rented gaming cafe in Korea, recording their communication and gameplay as they interacted. Across 28 collection days, 1,046 participants played 38,956 sessions with Ally. The resulting interaction rollouts include player speech, game events, the information Ally requested, tool results, Ally’s responses, and executed actions. We used the collected data to iteratively train a small language model (SLM) for on-device deployment. During the first two weeks, a 31B teacher model with a prompt optimized using GEPA ( Agrawal et al., 2025 ) played alongside human players to collect initial demonstrations. During the following two weeks, successive SLM versions played with human players, allowing us to collect rollouts that capture situations arising from the students’ own behavior. Inspired by DAgger ( Ross et al., 2011 ; Li et al., 2026 ) , we used the teacher to generate corrections for selected student interactions and progressively added them to the initial demonstrations. At each iteration, we used the accumulated corpus to train an intermediate 8B teacher and distill a deployable 2B student, followed by on-policy knowledge distillation. Evaluating Ally poses a different challenge: the quality of a teammate cannot be captured by combat performance or command completion alone. It also depends on whether players experience the agent as responsive, useful, natural, and cooperative during play. We therefore used player preferences, survey responses, and interaction records collected during the gameplay sessions to refine an evaluation framework for Ally. In particular, we examined cases in which internal evaluations disagreed with player preferences and used these discrepancies to identify missing or poorly specified aspects of teammate quality. This process led us to revise criteria for conversational quality, gameplay behavior, and cooperation, and to introduce evaluation scenarios grounded in situations observed during actual matches. The resulting framework was more closely aligned with the teammate qualities that players valued in actual play, providing a better basis for comparing model and system variants during subsequent development. Shipping Ally into a live service required production hardening beyond the agent design itself. Ally’s SLM, speech-to-text (STT), and text-to-speech (TTS) components run alongside the PUBG client on the player’s machine under tight latency and memory constraints. Player-facing deployment also introduces a contextual safety challenge: ordinary in-game combat language must remain playable, while speech targeting real-world people or groups, unsafe escalation across turns, and unsafe information entering persistent memory must be handled safely. We address these deployment requirements through model compression and context compaction, together with safety specifications, targeted training, runtime guardrails, and memory redaction. In measurements taken during the gameplay sessions, a single spoken exchange completed in approximately 1.6s on-device versus 3.4s with the cloud configuration. Following this development and production process, we launched Ally in a two-week live-service beta of PUBG, supporting English, Korean, and Chinese with locale-specific on-device language and speech models. A live-service survey reached players in 141 countries. Among respondents whose play with Ally was confirmed in game records, positive responses exceeded negative responses by 25.1 percentage points when asked whether they would recommend Ally. Players also varied in how they framed Ally: 18.5% selected “teammate” and 31.5% selected companion framings, together accounting for 50.0% of respondents. Specifically, we make the following contributions:

• An architecture enabling real-time gameplay and voice interaction. We present a conversational embodied agent architecture that integrates language-model reasoning, voice interaction, and autonomous gameplay through a bounded tool interface and a System 1–System 2 control hierarchy.

• Large-scale interaction data and training from real-player gameplay. We collect nearly 39k gameplay sessions with real players and train an on-device model using teacher demonstrations and teacher-corrected student rollouts.

• Player-centered evaluation of teammate quality. We develop an evaluation framework grounded in player preferences and real gameplay interactions, covering conversational quality, gameplay behavior, and cooperation.

• Contextual safety for a player-facing embodied agent. We develop and evaluate safety mechanisms that allow ordinary in-game communication while addressing harmful speech and unsafe information entering persistent memory.

• Production engineering and live-service deployment. We describe the production engineering required to run Ally’s language and speech models on-device under real-time constraints and deploy the system as a multilingual live-service beta. To our knowledge, Ally is the first conversational embodied teammate in a commercial live battle-royale game that can reason, act autonomously, and coordinate with human players through voice, with its language and speech models running on-device (Appendix A ). Ally’s architectural influence already extends to physical robotics: Ludi 0.1 draws on Ally’s agentic design principles to integrate reasoning, communication, memory, and physical action ( Ludo Robotics, 2026 ) . Together, these contributions establish a practical foundation for developing, training, and deploying embodied agents that communicate and act with people in real time. 2 PUBG Duo Play & Deployment Setting 2.1 How a Duo Plays PUBG ( PUBG Studios, 2017 ) is a multiplayer battle royale game. A match begins with up to a hundred players parachuting onto a large island, each starting with nothing. Players scavenge buildings for weapons, armor, and supplies while avoiding or fighting the others they run into. A safe zone, drawn as a circle on the map, periodically shrinks, and players left outside it in the encroaching blue zone steadily lose health, so everyone is pushed into an ever smaller area. As the zone closes, teams relocate across terrain, manage their exposure, and choose when to fight and when to stay hidden, and encounters grow more frequent until the last surviving player or team wins. A match therefore unfolds as a sequence of changing tactical conditions rather than a fixed script. In duo mode, two players form a single team and try to survive together from the drop to the final circle. Coordination is mostly voice-based, supported by pings and map markers. Players call out enemies, loot, danger, and the closing blue zone, often using landmark names and community shorthand. They also negotiate match-level decisions, such as where to drop, when to rotate, and whether to fight or avoid a third party. Beyond communication, teammates share resources, cover each other in fights, and revive a partner who has been knocked down and incapacitated but has not yet been eliminated. The partnership also has a social layer. Quiet stretches are filled with casual talk, and preferences, prior decisions, and in-match promises carry from one situation to the next. Being a good teammate therefore means coordinating speech, action, memory, and timing in a way that suits the partner and the moment. Each act that feels natural to a human partner becomes a problem for an AI in the same seat. Spoken commands and callouts are often incomplete or implicit, so the agent must infer intent from phrases such as “play safe” or “watch that side.” This requires understanding the vocabulary players use during play, from place names and weapon attachments to community slang. The agent must also carry context across the match, remembering relevant preferences, prior decisions, and commitments made earlier. Its speech and actions must stay connected, with actions chosen from the current situation rather than from a fixed script. Finally, all of this must happen quickly, since late reactions, stale destinations, or long replies can put the team out of step with the match. Ally targets the duo-teammate role described above, not as a lone agent, but as a partner who shares one team’s fate with a human player. Figure 2 : PUBG Ally runtime pipeline. Speech-to-text converts player speech into text. Ally obtains information from the current game state. System 2, the language-model agent, receives this information in textualized form through game events and requested observations. The language-model agent generates Ally’s reply as text, which text-to-speech converts into Ally speech. System 2 sends high-level action requests to System 1, a behavior tree that executes them in the shared game world at game-tick rate. 2.2 Deployment Setting Ally serves as a voice-enabled AI duo partner in 64-player battle-royale matches on Sanhok, paired with one human player and supporting Korean, English, and Chinese. Players communicate with Ally through a dedicated push-to-talk channel, giving each utterance a clear start and end. Because the match continues during inference, Ally must respond quickly enough for its communication and actions to remain relevant to the current situation. Figure 2 summarizes Ally’s runtime pipeline. STT converts player utterances into text for the SLM, which serves as System 2 and uses game information obtained through tools to select speech and high-level actions. TTS produces voice output, while a behavior-tree controller, System 1, executes actions at game-tick rate. We use an SLM to reduce decoding latency and run all language and speech models on-device to avoid network round trips. This introduces an additional resource constraint: Ally must share compute and memory with the PUBG client’s real-time rendering and gameplay. The deployed configuration targets consumer GPUs with at least 8,GB of VRAM and runs a single language-model inference at a time. Section 3 details the architecture and coordination between the two layers. 3 PUBG Ally Architecture This section describes the agent harness that enables a language-model agent to operate as a real-time teammate in PUBG. An embodied agent must track a continuously changing game world and react within the latency budget imposed by the match, while a conversational agent must understand and respond to player speech in real time. Combining these capabilities introduces an additional requirement: Ally’s speech must remain synchronized with its actions, because a callout can mislead the player if it no longer reflects what the agent is doing. The harness addresses these requirements by separating deliberation from control and defining what the agent can observe, when it runs, how it speaks and acts through tools, and what context carries across agent loops. 3.1 A Layered Agent Architecture A single control rate cannot support both deliberative reasoning and latency critical control. Low level behaviors, such as moving under fire or continuing toward a destination, must be updated every game tick. In contrast, interpreting teammate intent, selecting relevant observations, deciding what to say, and committing to a high level plan benefit from language model inference. However, within the current on-device compute budget described in Section 2.2 , language model inference is not yet practical at the game tick rate. Ally therefore adopts a dual-system architecture following the distinction between fast and slow reasoning described by Kahneman (2011) , as shown in Figure 2 . System 2 is the language model agent. It is invoked by events rather than by a fixed clock, reasons over a bounded tool interface, and decides what to observe, what to say, and which high level action to commit to. System 1 is a behavior tree, also referred to as the execution layer . It is evaluated every tick and translates those commitments into movement, combat, and recovery behaviors ( Isla, 2005 ; Colledanchise and Ögren, 2018 ) . The architecture uses four control channels instead of a one way pipeline. System 2 sends intent to System 1 and receives state in return. System 1 reads the game world and acts on it. Only System 1 interacts with the match on every game tick. This keeps the language model out of the latency critical path while the game world continues to evolve during System 2 inference. 3.2 System 2: Event-Driven Agent Harness Controlled tool interface. Rather than providing the model with the full match state ( OpenAI et al., 2019 ; Vinyals et al., 2019 ; Fan et al., 2022 ) , Ally instead interacts with the game environment through a bounded set of tools ( Yao et al., 2023 ; Yang et al., 2024 ; Anthropic, 2026a ; OpenAI, 2026a ) .

• Tool interface. Table 1 summarizes the tools available to the agent during a live match. Observation tools provide focused views of decision relevant match state. Speech tools let the agent respond to the player through voice, while action tools let it dispatch high level game actions. When needed, the agent can also retrieve static game knowledge and remembered information about the player or ongoing commitments. The interface also includes tools for ending the current agent loop and handling safety sensitive input. Each tool call returns its execution status together with any available result. Table 1 : Ally’s tool interface for a live match. The 16 callable tools are summarized in six functional groups. Counts show the number of tools in each group. Examples use shortened arguments and show either illustrative text results or concise descriptions of tool effects. Functional group # Description Example Match observation 7 Retrieve focused views of status, combat, equipment, pings, and destinations. get_combat_info() → \rightarrow Returns a combat summary such as “Two enemies are 40 m east. One can be engaged now.” Knowledge & memory 2 Retrieve game knowledge and update persistent player memory. lookup_game_knowledge("M416") → \rightarrow Returns weapon guidance such as “The M416 is stable with full attachments but harder to control without them.” Action execution 2 Check whether an action is available, then dispatch it. check_action_availability(move_to) → \rightarrow Confirms that the move is available and provides the required destination parameters execute_action(move_to, ...) → \rightarrow Starts moving Ally to the selected destination Player communication 3 Produce speech or a brief acknowledgment, and enable or disable voice output. speak("Need cover.") → \rightarrow Delivers the message to the player through TTS Agent loop control 1 End the current agent loop and carry unfinished work forward. compact("Revive, then clear enemy") → \rightarrow Retains the unfinished plan for the next loop Safety control 1 Mark unsafe input for redaction. flag_unsafe(privacy) → \rightarrow Marks sensitive input for redaction
• Role and limits. To build a reliable agent, we define the model’s role, observable information, available actions, speech behavior, and functional limits ( Qiao et al., 2025 ) . Ally encodes these boundaries in the system prompt, which defines its role as a teammate in a live PUBG match and specifies its supported behaviors, observable information, communication constraints, and functional limits. Before dispatching an action, the agent uses the action availability tool to check which actions are currently executable. If the player asks for information that is not observable through the interface, the agent can report that the information is unavailable instead of guessing. If the player requests an action outside the available controls, the agent can suggest an executable alternative such as following the player, moving to a landmark, or using a ping. These constraints reduce ungrounded responses and actions by keeping the agent within the information and controls exposed by the interface. Agent loop and event scheduling. Ally operates as a closed loop in which feedback from tools and the game informs the model’s next response. Each invocation of the agent loop processes incoming match events such as player speech, game state changes, and execution feedback.

• Agent loop . At the beginning of each agent loop, the agent receives recent event histories, a short plan carried over from the preceding agent loop, current events, and Ally’s current speech and action status. This status is refreshed as the agent loop proceeds so that each decision reflects whether Ally is still speaking and whether a dispatched action is still running. The agent can therefore delay a reply until the current utterance completes, and avoid dispatching an action that conflicts with one already running. In later turns, the agent receives results from preceding tool calls, newly arrived events, and refreshed speech and action status. The model reasons over this input and issues further tool calls, so an agent loop can span a variable number of turns. A turn is one model response. A trajectory is the sequence of model responses and intervening tool results or feedback produced during one invocation of the agent loop. It ends with the model’s compact(plan=...) call, which returns control to the runtime and carries unfinished work into the next invocation. A session is one complete match and contains multiple trajectories. For example, when the player asks for a healing item, the agent first uses observation tools to check its inventory. If the observation shows that a healing item is available, the agent uses the action availability tool to check whether it can hand the item over. If the action is available, the agent tells the player and dispatches the action. Figure 3 shows how turn inputs alternate with model reasoning and tool calls over a variable number of turns, and how unfinished work is condensed into a concise plan for the next agent loop. For details on retaining selected events in a bounded history across agent loops, see Appendix B . Figure 3 : Agent trajectories and context compaction. Each trajectory records one invocation of the agent loop. At the start of each agent loop, the model receives recent History , the carried-over Plan , new Events , and Ally’s Status , then reasons over this context to issue tool calls. Later turns add tool results and updated events to the context. The final compact(plan=...) call carries a revised plan into the next agent loop. Ellipses indicate omitted turns. The scene strips show actions and dialogue during a knockdown and revive sequence. Speech is blue, actions are teal, and the player’s Downed state is red.

• Event-driven scheduling. Unlike agents that run at a fixed control rate ( SIMA Team et al., 2025 ; ByteDance Seed et al., 2025 ) , Ally invokes System 2 in response to selected events so that the model can focus on salient information. The runtime places supported events in a common queue. These events include player speech, consequential game state changes, speech lifecycle updates, action outcomes, and scheduled runtime checks. Events that arrive during LLM inference accumulate in the queue. When the inference finishes, the runtime batches pending events into the next agent loop instead of launching a separate inference for each event. Each event type has a predefined priority. When too many events are pending, the runtime admits higher priority events first and drops lower priority events when necessary. Figure 3 shows that admitted events can begin an agent loop or enter a later turn. Reactivity and proactivity. Two runtime controls govern when Ally speaks or acts. The first sets how strongly each event calls for a response, and the second determines whether Ally may initiate without being asked.

• Reactivity priors. For each event type, we define separate reactivity priors for speech and action. These priors are specified in the corresponding event description and summarized in Table 2 . They specify how strongly each event favors a speech response and an action response independently. Adjusting these priors controls how strongly Ally responds to the same event. For example, when Ally finds an item requested by the player, a higher action prior encourages delivery when the action is available. A lower action prior instead encourages Ally to report the item without fetching it by default. This configuration changes Ally’s response behavior without retraining the model. Table 2: Representative event reactivity. Each event type independently specifies how strongly it should elicit speech and action. Require directs a response, recommend favors one, optional leaves the choice to the agent, and do-not suppresses it. The rows show representative rather than exhaustive event configurations. Event (example) Speak Act Player spoke to you require recommend Player downed by an enemy require recommend Enemy exposed to attack recommend optional Found a requested item require do-not Finished the current action do-not recommend
• Controlled proactivity. A useful teammate should sometimes speak or act without an explicit request. The runtime enables this behavior by generating events in specific situations and periodically waking the agent after it has remained idle. However, overly frequent unsolicited speech or action can interrupt play. We bound this behavior through several mechanisms. Event cooldowns suppress repeated triggers. Reactivity priors control how strongly each event favors speech or action. Event descriptions are also phrased to discourage less important events from interrupting ongoing tasks. Together, these controls let Ally be proactive while reducing unnecessary or poorly timed speech and action. Context management across agent loops . What Ally carries across agent loops is chosen so that stale match facts do not persist, unfinished work is not lost, and the prompt prefix The ai agents story also surfaces in Google AI Introduces EnvHarness for Adaptive..., adding another angle. as detailed in the full paper on Arxiv