Remove weights in stages rather than all at once
ETAPS is a pruning method that progressively tightens a global sparsity constraint while training a sparse delta with a small fixed budget. The goal is to preserve model quality at high sparsity without maintaining model-sized optimizer state and auxiliary variables.
The underlying idea is that smaller pruning steps give adaptation a chance to compensate. The adapted model then supplies gradients that inform the next pruning decision, connecting the removal process to the end-to-end objective.
Compare quality and the cost of producing the sparse model
Across six evaluated models ranging from 125 million to 13 billion parameters, the authors report lower perplexity than ELSA at every tested sparsity from 50 to 90 percent. The largest reported perplexity reduction is 42 percent.
That maximum is not the average across all models or a measured improvement in every downstream task. Perplexity is the paper's quality measure for this comparison; it should not be translated directly into a claim about a user's answers.
At 90 percent sparsity, the paper reports pruning LLaMA-2-7B on one H200 GPU where ELSA uses four, with five-times smaller persistent state and 49 percent fewer GPU-hours. These figures describe the evaluated pruning process, rather than a universal inference-cost reduction.
Check the runtime as well as the sparse weights
A sparse model can reduce stored parameters, but practical inference speed also depends on the representation, kernel, hardware and workload. Removing 90 percent of weights does not automatically make an application ten times faster.
For a reproduction, compare the model's quality, total pruning budget and complete inference performance using the intended sparse runtime. ETAPS offers evidence that gradual removal and limited adaptation can improve the quality-cost tradeoff in the tested regime. The next useful question is whether that tradeoff survives the actual tasks and deployment constraints of a particular model.
Comments