Back to AI Research

AI Research

STAIR (STructure Aware Information Retriever): A no... | AI Research

Key Takeaways

  • STAIR (STructure Aware Information Retriever): A novel dataset and LLM based retriever for document structure augmentation Retrieval Augmented Generation (RA...
  • Retrieval Augmented Generation (RAG) is a key component for generating accurate and hallucination free answers using Large Language Models (LLMs).
  • LLMs are improving at handling long context, but still suffer from "lost in the middle" problem.
  • Thus, precise and accurate retrieval is important.
  • Current retrievers chunk long context into length-based manageable chunks - in the process throwing away rich and informative semantic global structure in the corpus.
Paper AbstractExpand

Retrieval Augmented Generation (RAG) is a key component for generating accurate and hallucination free answers using Large Language Models (LLMs). LLMs are improving at handling long context, but still suffer from "lost in the middle" problem. Thus, precise and accurate retrieval is important. Current retrievers chunk long context into length-based manageable chunks - in the process throwing away rich and informative semantic global structure in the corpus. We introduce a novel retrieval system STAIR that empowers an LLM to exploit global structure in a corpus such as a Table of Contents (ToC) to efficiently store and retrieve information from its model parameters. Our thorough and careful ablation studies with a finetuned Differentiable Search Index (DSI) system show that ToC helps build a low hallucination (less than 0.05%) generative Information Retrieval (IR) system and can generalize to examples where very few training samples are available. To further research in this novel direction of ToC based retrieval we release SearchTome - a diverse benchmark created from 18 books across 6 diverse domains to further research in this novel direction. STAIR achieves a high Recall@1 score of 82.6% on SearchTome as compared to DSI (76.9%), where the difference is found to be statistically significant. STAIR easily beats other strong baselines such as BM25 (59.5%), DPR (68.7%) and out-of-the-box Mistral (13.8%).

STAIR (STructure Aware Information Retriever): A novel dataset and LLM based retriever for document structure augmentation
Retrieval Augmented Generation (RAG) is essential for helping Large Language Models (LLMs) provide accurate, hallucination-free answers. However, current retrieval methods often break long documents into arbitrary chunks based on length, which destroys the natural semantic structure of the text and can lead to the "lost in the middle" problem. This paper introduces STAIR, a retrieval system that improves accuracy by leveraging the global structure of a document—specifically its Table of Contents (ToC)—to help the model better store and retrieve information. The ai search story also surfaces in Stanford AI discovery identifies natural weight..., adding another angle.

How STAIR Works

STAIR treats the Table of Contents as a map for the model. Instead of searching through disconnected text fragments, the system is trained to understand the hierarchical relationship between section titles and the content they contain. By providing the full ToC as part of the input, the model learns to identify which specific "leaf node" (the most granular section in the ToC) contains the answer to a user's query. This approach allows the model to use its own internal parameters to store knowledge more effectively, mapping queries directly to semantically coherent sections rather than random text blocks.

The SearchTome Benchmark

To support research in this area, the authors created and released SearchTome, a new, diverse benchmark for evaluating structure-aware retrieval. It consists of 18 books across six distinct domains: Education, Finance, Law, Medicine, Natural Sciences, and Social Sciences. Each book has been parsed to extract its Table of Contents and mapped to the corresponding text. The benchmark includes a comprehensive set of training, development, and test queries, providing a standardized way to measure how well different systems can navigate document structures to find precise information. The same ai search question is explored in Measure Before You Manage, which adds a research perspective.

Key Results

The researchers compared STAIR against several strong baselines, including BM25, DPR, and the Differentiable Search Index (DSI). STAIR demonstrated superior performance, achieving a Recall@1 score of 82.6%, which significantly outperformed DSI (76.9%) and other traditional methods. The study also found that incorporating the Table of Contents helps reduce hallucinations to less than 0.05% and allows the system to generalize well even when training data is limited. Statistical testing confirmed that the performance gains achieved by STAIR are significant across all six domains tested.

Why Structure Matters

The core insight of this research is that document structure is not just a formatting convenience for humans; it is a powerful tool for AI. By organizing information into semantically bounded units, the model can avoid the confusion caused by length-based chunking. This method proves that when an LLM is empowered with a global view of a document's organization, it can perform more precise retrieval, leading to more reliable and accurate generative outputs. The same large language models question is explored in Efficient Test-Time Adaptation through Human-AI Interaction, which adds a research perspective. as detailed in the full paper on Arxiv

More from Franklin

Mistral AI Releases Leanstral 1.5 for...newsMeasure Before You ManageresearchEfficient Test-Time Adaptation through Human-AI InteractionresearchNew AI Architecture Mimics the Human...newsStanford AI discovery identifies natural weight...newsGoogle AI Launches TabFM for Zero-Shot...news

Comments (0)

No comments yet

Be the first to share your thoughts!