A CPU-ARC preprint describes a way to reuse the work a language model did to solve a problem. The method converts reasoning traces into code that runs on a CPU and can handle new instances of the same task. The researchers report that invoking these caches was 16 to 291 times cheaper than invoking the source model across the workloads they tested. The preprint was submitted to arXiv on November 6, 2026.
What the cache actually stores
CPU-ARC stores an executable procedure rather than the text of a previous answer. The authors start from a conversation in which a model worked through the task. They turn the reusable part of that conversation into a program that can read a new input and apply the same method.
The program can call small language models where a narrow text operation is still useful. The paper gives examples including extracting a value, converting a format or classifying text. The expensive model no longer has to rediscover the rest of the procedure each time.
When the system calls the full model
At inference, the proposed system tries the reasoning cache before sending a request to the full model. A cache miss or execution error sends the request to the original model. That fallback lets an agent keep serving inputs for which it has no usable cached procedure.
The reported cost range applies to invoking a cache, not to building it. The paper says constructing the caches cost 12 to 84 times as much as the original reasoning. Reuse is therefore part of the economics: one cheaply served hit may not repay the initial work.
Where the method is useful
The authors evaluate mathematical and scientific tasks and report performance comparable to frontier models on their tested workloads. The case for trying it is strongest when a workflow returns to the same procedure often. An engineer evaluating it could measure the fraction of inputs that hit a cache, the cost of failed attempts and the time needed to recover the construction cost.
Comments