Back to AI Research

AI Research

A Table Is Worth 64 Tokens: Pixel-level Compression... | AI Research

Key Takeaways

  • A Table Is Worth 64 Tokens: Pixel-level Compression for Multi-Table Document Question Answering explores how to efficiently process long documents that conta...
  • Answering questions over real-world documents requires processing long inputs that interleave text with tables.
  • Optical context compression, which represents context as images, promises to reduce token cost, but its effect on table understanding remains unclear.
  • We study pixel-level table compression for question answering over documents with multiple tables, evaluating five VLMs across two benchmarks and five visual-token budgets.
  • Highly downscaled tables, however, preserve enough signal to identify whether they are relevant to a question.
Paper AbstractExpand

Answering questions over real-world documents requires processing long inputs that interleave text with tables. Optical context compression, which represents context as images, promises to reduce token cost, but its effect on table understanding remains unclear. We study pixel-level table compression for question answering over documents with multiple tables, evaluating five VLMs across two benchmarks and five visual-token budgets. Representing tables as images at native resolution matches text in both performance and efficiency, but downscaling them makes models compensate the loss in readability with longer, less effective reasoning traces that cancel the expected savings. Highly downscaled tables, however, preserve enough signal to identify whether they are relevant to a question. We exploit this asymmetry with a training-free, two-step method: the model first identifies the tables needed to answer a question from a pixel-compressed context, and then reasons over those at native resolution. On long documents, our method saves 41% of total tokens and gains 7 accuracy points over single-step QA with native resolution tables. It also uses 15% fewer tokens than the most efficient single-step compressed configuration, with no accuracy loss.

A Table Is Worth 64 Tokens: Pixel-level Compression for Multi-Table Document Question Answering explores how to efficiently process long documents that contain many tables. Researchers Iñigo Alonso and Mirella Lapata investigate whether representing tables as images—rather than text—can reduce the computational cost of question answering (QA) without sacrificing accuracy.

The Challenge of Table Compression

Processing long documents is computationally expensive, and performance often drops when inputs exceed the lengths used during model pre-training. While representing text as images (optical context compression) can save tokens, its impact on complex tasks like table-based numerical reasoning has been unclear. The authors evaluated five Vision-Language Models (VLMs) across two datasets, MultiHiertt and FinLongDocQA, using five different visual-token budgets. They found that while native-resolution images perform as well as text, aggressive downscaling makes tables illegible. When models cannot read cell values, they often compensate by generating longer, less effective reasoning traces, which negates the token savings gained from compression.

A Two-Stage "Identify First, Reason Later" Method

The researchers identified an asymmetry: while heavily compressed tables are too blurry for detailed reading, they still contain enough visual signal to determine which tables are relevant to a specific question. Based on this, they developed a training-free, two-stage method: 1. Identification: The model receives the full document with all tables compressed to a small visual-token budget (as low as 64 tokens) to identify which tables are needed. 2. Reasoning: The model is then provided only with the identified tables at their native resolution to perform the actual reasoning and answer the question.

Performance and Efficiency Gains

This two-stage approach significantly improves the accuracy-efficiency trade-off compared to single-step QA. On long documents, the method saves 41% of total tokens while gaining 7 accuracy points over single-step QA using native-resolution tables. Furthermore, it matches the accuracy of the most efficient single-step compressed configuration while using 15% fewer tokens. The authors note that this method is model-agnostic and can be applied to both full documents and contexts retrieved by external systems.

Key Considerations

The study highlights that simply lowering image resolution is insufficient for complex tasks because it degrades fine-grained information. The effectiveness of the two-stage method relies on the model's ability to distinguish relevant tables from irrelevant ones using only coarse visual cues. The authors observed that table identification remains robust even at the lowest tested budgets, whereas transcription and direct QA performance drop significantly as resolution decreases. This research suggests that decoupling evidence identification from reasoning is a viable strategy for handling information-dense, multi-table documents at scale.

Comments (0)

No comments yet

Be the first to share your thoughts!