Trace Integrity for LLM Data Agents: A Vision for A... | AI Research

Trace Integrity for LLM Data Agents: A Vision for Auditable Structured Reasoning in Real-World Systems proposes a new reliability standard for LLM-based data agents. The authors, Srimonti Dutta and Akshata Kishore Moharir of WAI USA Research Labs, argue that measuring only the final answer accuracy of an LLM is insufficient for real-world systems, as models can produce correct answers through invalid or unfaithful computational steps.

The Structure Gap and Silent Failures

The authors identify a "Structure Gap"—a mismatch between natural-language reasoning and the precise, operator-level programs (such as SQL queries) required for structured data tasks. Because LLMs generate text, they may produce a plausible explanation while silently failing to apply the correct filters, joins, or aggregations. This creates a "silent failure" where an agent returns a correct answer based on an incorrect or unverified computation. The paper notes that in operational settings like business intelligence or clinical analytics, users often lack the ability to reconstruct the query, making it critical that the underlying computation is transparent and auditable.

Operationalizing Trace Integrity

To address this, the authors introduce "Trace Integrity," a criterion for evaluating whether the computation behind an answer is explicit, executable, schema-valid, operator-faithful, replayable, answer-consistent, and auditable.
They operationalize this through "execution contracts"—structured artifacts that bind user intent to specific schema elements, operator plans, and assumptions. By following the "Isolation Principle," agents are encouraged to define these computational commitments before accessing data values. This separation prevents the model from retrospectively justifying an answer based on the data it has already seen, ensuring the plan remains independent of the result.

Evaluating Hidden Risks with CAIT

The paper introduces the "Correct Answer / Invalid Trace" (CAIT) Rate to measure how often systems receive credit for correct answers that are not supported by a valid computational trace. In an empirical demonstration using 100 examples from the BIRD Mini-Dev dataset, the authors compared three methods: Direct SQL, Operation Summary + SQL, and Contract-First SQL.
The results showed that while answer accuracy ranged from 20% to 24%, the CAIT rates remained high, between 45.8% and 59.1%. This indicates that a significant portion of "successful" answers are backed by flawed logic. The authors conclude that answer accuracy and trace validity are distinct signals; therefore, deployment teams should monitor both to avoid overestimating system reliability.

Limitations and Scope

The authors characterize this work as a vision and a proof-of-concept rather than a comprehensive benchmark. The study is limited to a single model (Claude Haiku 4.5) and a small, stratified sample of the BIRD Mini-Dev dataset. The authors note that their validator focuses on identifying specific, inspectable failures—such as missing filters or incorrect joins—rather than proving full semantic equivalence, which serves the study's goal of diagnosing why a system's computation might diverge from its final output.