Back to AI Research

AI Research

GLM-RAG: Graph Language Models for Graph-Based Retr... | AI Research

Key Takeaways

  • GLM-RAG is a framework designed to improve retrieval-augmented generation (RAG) by using a Graph Language Model (GLM) to better integrate graph structure wit...
  • Retrieval-augmented generation (RAG) over knowledge graphs requires retrievers that can effectively capture both graph structure and semantic information.
  • Recent approaches have explored graph neural network (GNN)-based retrievers to model graph topology in multi-hop reasoning tasks.
  • In parallel, graph language models (GLMs) have emerged as a promising paradigm that integrates graph reasoning and the semantic capabilities of language models.
  • Our findings suggest that finetuned GLM retrievers generalize better out of domain, achieving SOTA on two multi-hop benchmarks.
Paper AbstractExpand

Retrieval-augmented generation (RAG) over knowledge graphs requires retrievers that can effectively capture both graph structure and semantic information. Recent approaches have explored graph neural network (GNN)-based retrievers to model graph topology in multi-hop reasoning tasks. In parallel, graph language models (GLMs) have emerged as a promising paradigm that integrates graph reasoning and the semantic capabilities of language models. In this work, we introduce a GLM-based retriever and investigate the comparative strengths of GLM-based, GNN-based, and traditional vector-search-based retrievers in single- and multi-hop RAG settings, and with a particular focus on transferability to unseen domains. Our findings suggest that finetuned GLM retrievers generalize better out of domain, achieving SOTA on two multi-hop benchmarks. On in-domain multi-hop QA datasets they remain comparable to prior work, with promising scaling as parameters and subgraph coverage increase. GNN-based retrievers achieve higher graph coverage with an efficient training setup, whereas the vector-search baseline excels at single-hop datasets.

GLM-RAG is a framework designed to improve retrieval-augmented generation (RAG) by using a Graph Language Model (GLM) to better integrate graph structure with semantic text understanding. By replacing traditional GNN-based retrievers with a model that can process text-attributed graphs at the token level, the researchers aim to solve the limitations of current graph RAG methods, which often struggle with multi-hop reasoning and deep semantic integration.

How GLM-RAG Works

The framework converts knowledge graphs into sequences of textual triplets, allowing the model to process them using the architecture of a pretrained language model. Unlike GNN-based approaches that often rely on structural distance or static embeddings, GLM-RAG uses a graph encoder with structure-aware relative positions. This enables the model to fuse query text with graph information during the initial attention layers, allowing for a more nuanced understanding of entities and their relationships.

Performance and Generalization

In experiments conducted by Maya Arseven, Anette Frank, Beni Egressy, Johann Higl, and Moritz Plenz, GLM-RAG achieved state-of-the-art results on two multi-hop benchmarks. When tested for transferability to unseen domains, the model consistently outperformed GNN-based alternatives. While GNN-based retrievers are effective at covering larger portions of a graph, the researchers found that GLM-based retrievers excel at questions requiring deeper semantic understanding.

Comparison with Existing Methods

The study provides a comparative analysis of three retrieval paradigms:

  • Vector-search (Vanilla RAG): Found to be sufficient for single-hop datasets where complex reasoning is not required.

  • GNN-based retrievers: Capable of efficient training and broad graph coverage, but limited by shallow integration of text features.

  • GLM-based retrievers: Show strong scaling with model size and superior performance in multi-hop settings, particularly when applied to out-of-domain tasks.

Franklin Analysis

The evidence suggests that the choice of retriever should be task-dependent. The researchers' findings indicate that while graph-enhanced methods are necessary for multi-hop reasoning, they do not provide a universal advantage over simpler vector-search baselines for single-hop queries. The superior performance of GLM-RAG in out-of-domain settings supports the conclusion that deep semantic integration is a critical factor for generalizable graph-based retrieval, as the model's ability to process graph structure and text simultaneously allows it to outperform models that treat these components as separate inputs.

Comments