PaDoc: Layout-Grounded Parallel Decoding for Docume...

PaDoc: Layout-Grounded Parallel Decoding for Docume... | AI Research

Key Takeaways

PaDoc is a document parsing system designed to accelerate the conversion of page images into structured data—such as text, tables, and formulas—by enabling p...
End-to-end document parsers provide a unified interface, but serialize page layouts and regional contents into one autoregressive sequence.
To retain full-page context while removing dependencies, we propose PaDoc, a layout-grounded parser that treats the predicted layout as a branching structure over a shared page representation.
On OmniDocBench Full, PaDoc attains an Overall layout F1 of 91.1 and, among end-to-end parsers, a top-tier Overall score of 94.24 together with the best Text Edit (0.038) and Formula CDM (95.59).
PaDoc is a document parsing system designed to accelerate the conversion of page images into structured data—such as text, tables, and formulas—by enabling parallel processing.

Paper AbstractExpand

End-to-end document parsers provide a unified interface, but serialize page layouts and regional contents into one autoregressive sequence. This formulation forces independent regions onto a decoding path whose length grows with the total content, whereas crop-based two-stage parsers expose region-level parallelism at the cost of repeated visual prefills and fragmented page context. To retain full-page context while removing dependencies, we propose PaDoc, a layout-grounded parser that treats the predicted layout as a branching structure over a shared page representation. Under a region-sufficiency assumption, we derive a prefix-conditioned factorization in which the layout stream and regional content branches advance concurrently, reducing the decoding depth to the longest layout-content path. We realize this factorization within a single MLLM: packed variable-length ancestor attention preserves the visibility under standard next-token training, while masked parallel decoding creates branches that the evaluated vLLM backend serves as concurrent requests with cache-resident shared-prefix reuse. On OmniDocBench Full, PaDoc attains an Overall layout F1 of 91.1 and, among end-to-end parsers, a top-tier Overall score of 94.24 together with the best Text Edit (0.038) and Formula CDM (95.59). On a 384-page subset and one A800 GPU, it is the fastest end-to-end parser at five concurrency levels, improving valid-page throughput by 67.4-118% and reducing P95 latency by 39.2-54.9% relative to a same-backbone Sequential SFT baseline. Code is available at this https URL

PaDoc is a document parsing system designed to accelerate the conversion of page images into structured data—such as text, tables, and formulas—by enabling parallel processing. While traditional end-to-end parsers process documents as a single, long sequence, PaDoc uses the document's layout to create a branching structure, allowing the model to generate content for different regions of a page simultaneously.

Addressing Sequential Bottlenecks

Standard document parsers use an autoregressive approach, where the model must generate the entire page layout and all regional content in one continuous line. This creates a "critical path" that grows with the total amount of content on the page, forcing the model to wait for one region to finish before starting the next. While "two-stage" parsers attempt to solve this by cropping regions and processing them in parallel, they often suffer from fragmented context and the overhead of repeatedly encoding the same page image. PaDoc aims to bridge this gap by keeping the full page image as a shared visual prefix while allowing content branches to fork off based on the predicted layout.

How PaDoc Works

The researchers implemented PaDoc within a single Multimodal Large Language Model (MLLM). The system relies on a "region-sufficiency" assumption, which posits that once a specific layout region is defined, its content can be generated independently of other regions on the page.
To achieve this without auxiliary models, the authors use "ancestor attention." During training, the model uses a packed variable-length attention mechanism that allows content branches to see the shared page image and their specific layout ancestors, while ignoring irrelevant sibling content. At inference, the system uses a vLLM backend to treat these branches as concurrent requests. Because these branches share the same visual and layout prefix, the system can reuse the cached data, significantly reducing the time required to parse a document.

Performance and Efficiency

On the OmniDocBench dataset, PaDoc achieved an overall score of 94.24, placing it among the top-tier end-to-end parsers. It also recorded the best performance in Text Edit (0.038) and Formula CDM (95.59) metrics.
In terms of efficiency, the researchers tested the model on a 384-page subset using a single A800 GPU. Compared to a standard sequential baseline, PaDoc improved valid-page throughput by 67.4% to 118% and reduced P95 latency by 39.2% to 54.9%. The authors note that by reducing the decoding depth to the longest layout-content path rather than the sum of all regional content, the model maintains high quality while significantly increasing processing speed.

Key Considerations

The PaDoc approach is built on the assumption that content within a document region is independent of other regions once the layout is known. While this allows for efficient parallelization, the model's performance is tied to the accuracy of its layout predictions. The authors note that their implementation avoids the need for separate detector or recognizer heads, relying instead on a single MLLM architecture initialized from the Qwen3-VL-2B checkpoint and refined through continual pre-training and supervised fine-tuning.