Back to AI Research

AI Research

DictLLM compresses LLaMA-7B by sharing dictionaries across layers

Key Takeaways

  • DictLLM shares weight dictionaries across model layers and reports 5.1× compression on LLaMA-7B with a 2.9-point increase in WikiText-2 perplexity.
  • DictLLM reuses a shared dictionary within groups of layers and keeps sparse coefficients for each layer.
  • The authors report 5.1× compression of LLaMA-7B with higher WikiText-2 perplexity.
  • DictLLM compresses language model weights after training.
  • The authors group layers so they can share a dictionary of weight patterns, while sparse coefficients capture what each layer needs from that dictionary.

DictLLM compresses language model weights after training. The authors group layers so they can share a dictionary of weight patterns, while sparse coefficients capture what each layer needs from that dictionary. The goal is to retain enough of the original model’s behavior while storing a smaller representation. The preprint was submitted to arXiv on November 5, 2026.

How the compression works

DictLLM starts from a trained model. It uses singular value decomposition on input-scaled weight matrices to construct the shared dictionaries. It then prunes each layer’s coefficients to meet a target compression ratio. The pruning uses a dictionary-aware estimate of how removing a coefficient affects reconstruction. The authors refine the dictionary after pruning and quantize both the dictionary and coefficients into NVFP4.
This design lets several layers reuse the same stored patterns. The coefficients are still specific to each layer, so the compression can retain differences in how the layers use those patterns. The paper applies these steps as post-training optimization.

What the LLaMA-7B comparison measures

The authors report 5.1× compression of the original LLaMA-7B model and a 2.9-point increase in WikiText-2 perplexity. They describe that compression ratio as 3.6× higher than the evaluated low-rank comparison. The paper also evaluates zero-shot reasoning on seven benchmarks, reporting accuracy for those tasks separately from language modeling perplexity.

How to use the result

The size and perplexity figures describe a specific operating point on the paper’s compression comparison. Someone choosing a compressed model can compare the storage savings with how much quality they need to retain on their workload. The paper uses both language modeling and reasoning tests because those measurements tell readers different things about the compressed model. Device runtime will also depend on how the implementation handles the sparse coefficients and low-bit representation.

Comments