Reducing numerical precision can make a language model cheaper to represent, but the same precision choice may not affect every part of generation equally. Different Tasks, Different Timesteps examines this issue in diffusion language models, where generation passes through iterative denoising steps.
Sensitivity changes during generation
The authors report that quantization sensitivity depends on both the downstream task and the denoising timestep. Different tasks show different precision requirements across those steps. A single uniform allocation can therefore overlook where a particular task is most sensitive to numerical changes.
They also report that these patterns can be predicted without ground-truth labels or task-specific evaluation metrics. The proposed sensitivity signal is intended to guide precision allocation when labelled evaluation data is unavailable for making that decision.
Allocate a fixed budget more selectively
T2Quant uses that label-free signal in a task- and time-aware mixed-precision framework. It allocates activation precision across denoising steps under a fixed bit-width budget. The method changes where precision is spent, so the relevant comparison keeps the total budget matched.
Across three diffusion language models and six benchmarks, the paper reports improvements in the average accuracy of existing quantization methods, with gains of up to 6.7 percentage points under the same budget. The maximum gain should not be treated as the improvement a reader will obtain on every model, task or quantization configuration.
Measure the deployment effect separately
The reported accuracy result does not by itself establish a particular latency reduction, memory saving or serving-cost improvement. Those depend on how the chosen mixed-precision operations are executed in the intended environment.
A useful evaluation would preserve the model, task, quantization baseline and bit-width budget, then inspect both accuracy and actual execution costs. It should also check whether the sensitivity signal remains informative for the new workload. The study offers a way to distribute a precision budget according to task and time, with the practical benefit still tied to the setting in which it is tested.
Comments