Back to AI Research

AI Research

Stop Thinking, Start Looking: Efficient Post-Traini... | AI Research

Key Takeaways

  • Stop Thinking, Start Looking: Efficient Post-Training for Multimodal Document Question Answering via Reasoning-Free Alignment Multimodal document question an...
  • Efficient multimodal document question answering with explicit visual grounding, locating the precise document region that supports each answer remains an open challenge.
  • To rigorously evaluate the necessity of reasoning, we construct a reasoning variant under identical reward settings.
  • We further show that an early SFT$\rightarrow$RL transition achieves comparable precision with 65% less training data.
  • Stop Thinking, Start Looking: Efficient Post-Training for Multimodal Document Question Answering via Reasoning-Free Alignment
Paper AbstractExpand

Efficient multimodal document question answering with explicit visual grounding, locating the precise document region that supports each answer remains an open challenge. Current approaches bifurcate into Supervised Fine-Tuning (SFT), which requires large annotated datasets and reaches optimization plateaus, and reasoning-centric Reinforcement Learning (RL), which depends on verbose intermediate traces that inflate inference token cost without clear benefit. We introduce Perception-RFT, a training framework that applies Group Relative Policy Optimization (GRPO) to multimodal document QA, bypassing intermediate reasoning tokens to directly align visual features with structured grounding outputs. To rigorously evaluate the necessity of reasoning, we construct a reasoning variant under identical reward settings. We find that reasoning-enabled models suppress their reasoning traces during training, converging to direct perception-based policies at the 4B parameter scale, reducing per-query inference token length by more than 60%, while reasoning-enabled RL underperforms perception-only training. Through a fine-grained analysis of Qwen3-VL-4B optimization dynamics, we confirm that SFT saturation and cold-start RL instability established in text-domain post-training extend to multimodal, and identify a previously uncharacterized Grounding Divergence: a selective trade-off between semantic robustness and geometric precision on two out of distribution (OOD) benchmarks (4,828 samples) under joint RL optimization. We further show that an early SFT$\rightarrow$RL transition achieves comparable precision with 65% less training data.

Stop Thinking, Start Looking: Efficient Post-Training for Multimodal Document Question Answering via Reasoning-Free Alignment
Multimodal document question answering requires models to not only read text but also pinpoint the exact location of that information on a page. Current methods often rely on either massive datasets or complex "reasoning" steps—where the model writes out its thought process before answering. This paper introduces Perception-RFT, a framework that proves these intermediate reasoning steps are often unnecessary. By training models to map visual features directly to document coordinates, the researchers achieve higher precision while significantly reducing the computational cost of generating answers.

The Shift to Direct Perception

The researchers argue that document visual grounding is a perception-dominant task, not a logical one. While many modern models use "chain-of-thought" reasoning to bridge the gap between seeing and answering, this adds unnecessary length and latency to every query. The Perception-RFT framework uses a "Direct Perception" constraint, which forces the model to skip the internal monologue and output the final answer and bounding box coordinates immediately. When the researchers tested this against models that were allowed to "think," they found that the models naturally suppressed their own reasoning traces during training, ultimately converging on a faster, more efficient direct-answering style.

Training with Gated Rewards

To train the model effectively without relying on reasoning, the authors implemented a "Gated Dense Reward" system within a reinforcement learning framework. This system uses three specific goals: ensuring the output is a valid JSON format, matching the correct text, and hitting the precise visual location on the document. The "gated" part of the reward is crucial; it provides a small signal to guide the model when it is close to the right answer, but it only grants a full reward once the model hits a specific "success threshold." This prevents the model from settling for "good enough" or oversized bounding boxes, pushing it instead toward high-precision geometric accuracy.

Key Findings and Performance

The study reveals that supervised fine-tuning (SFT) eventually hits a performance ceiling, but applying reinforcement learning afterward allows the model to continue improving. A major takeaway is the importance of "warm-starting": models that were first trained with standard supervised methods and then refined with reinforcement learning performed significantly better than those that started reinforcement learning from scratch.
The researchers also identified a phenomenon they call "Grounding Divergence." When tested on documents outside of their training domain, the models showed a trade-off where they became much better at pinpointing locations but slightly less robust in their semantic understanding. Despite this, the overall ability to provide accurate, grounded answers remained high, proving that for tasks like financial auditing, the gain in spatial precision is a vital trade-off.

Efficiency and Practicality

By using this approach, the researchers were able to reduce the number of tokens generated per query by more than 60% compared to reasoning-heavy models. Furthermore, they demonstrated that an early transition from supervised training to reinforcement learning allows for high-quality results using 65% less training data. This makes the framework highly efficient, allowing for specialized, high-performance document analysis on accessible hardware like a single A100 GPU.

Comments (0)

No comments yet

Be the first to share your thoughts!