Back to AI Research

AI Research

GLM-RAG: Graph Language Models for Graph-Based Retr... | AI Research

Key Takeaways

  • GLM-RAG is a framework designed to improve retrieval-augmented generation (RAG) by using a Graph Language Model (GLM) to better integrate graph structure wit...
  • Retrieval-augmented generation (RAG) over knowledge graphs requires retrievers that can effectively capture both graph structure and semantic information.
  • Recent approaches have explored graph neural network (GNN)-based retrievers to model graph topology in multi-hop reasoning tasks.
  • In parallel, graph language models (GLMs) have emerged as a promising paradigm that integrates graph reasoning and the semantic capabilities of language models.
  • Our findings suggest that finetuned GLM retrievers generalize better out of domain, achieving SOTA on two multi-hop benchmarks.
Paper AbstractExpand

Retrieval-augmented generation (RAG) over knowledge graphs requires retrievers that can effectively capture both graph structure and semantic information. Recent approaches have explored graph neural network (GNN)-based retrievers to model graph topology in multi-hop reasoning tasks. In parallel, graph language models (GLMs) have emerged as a promising paradigm that integrates graph reasoning and the semantic capabilities of language models. In this work, we introduce a GLM-based retriever and investigate the comparative strengths of GLM-based, GNN-based, and traditional vector-search-based retrievers in single- and multi-hop RAG settings, and with a particular focus on transferability to unseen domains. Our findings suggest that finetuned GLM retrievers generalize better out of domain, achieving SOTA on two multi-hop benchmarks. On in-domain multi-hop QA datasets they remain comparable to prior work, with promising scaling as parameters and subgraph coverage increase. GNN-based retrievers achieve higher graph coverage with an efficient training setup, whereas the vector-search baseline excels at single-hop datasets.

GLM-RAG is a framework designed to improve retrieval-augmented generation (RAG) by using a Graph Language Model (GLM) to better integrate graph structure with semantic text understanding. By replacing traditional GNN-based retrievers with a model that can process text-attributed graphs at the token level, the researchers aim to solve the limitations of current graph RAG methods, which often struggle with multi-hop reasoning and deep semantic integration.

How GLM-RAG Works

The framework converts knowledge graphs into sequences of textual triplets, allowing the model to process them using the architecture of a pretrained language model. Unlike GNN-based approaches that often rely on structural distance or static embeddings, GLM-RAG uses a graph encoder with structure-aware relative positions. This enables the model to fuse query text with graph information during the initial attention layers, allowing for a more nuanced understanding of entities and their relationships.

Performance and Generalization

In experiments conducted by Maya Arseven, Anette Frank, Beni Egressy, Johann Higl, and Moritz Plenz, GLM-RAG achieved state-of-the-art results on two multi-hop benchmarks. When tested for transferability to unseen domains, the model consistently outperformed GNN-based alternatives. While GNN-based retrievers are effective at covering larger portions of a graph, the researchers found that GLM-based retrievers excel at questions requiring deeper semantic understanding.

Comparison with Existing Methods

The study provides a comparative analysis of three retrieval paradigms:

  • Vector-search (Vanilla RAG): Found to be sufficient for single-hop datasets where complex reasoning is not required.

  • GNN-based retrievers: Capable of efficient training and broad graph coverage, but limited by shallow integration of text features.

  • GLM-based retrievers: Show strong scaling with model size and superior performance in multi-hop settings, particularly when applied to out-of-domain tasks.

Franklin Analysis

The evidence suggests that the choice of retriever should be task-dependent. The researchers' findings indicate that while graph-enhanced methods are necessary for multi-hop reasoning, they do not provide a universal advantage over simpler vector-search baselines for single-hop queries. The superior performance of GLM-RAG in out-of-domain settings supports the conclusion that deep semantic integration is a critical factor for generalizable graph-based retrieval, as the model's ability to process graph structure and text simultaneously allows it to outperform models that treat these components as separate inputs.

Comments (0)

No comments yet

Be the first to share your thoughts!