Trace Integrity for LLM Data Agents: A Vision for Auditable Structured Reasoning in Real-World Systems

Trace Integrity evaluates whether a data agent's computation is executable, schema-valid, operator-faithful, replayable, answer-consistent, and auditable. On BIRD Mini-Dev, three SQL modes reached only **20–24% answer accuracy** and **39–43% trace-integrity pass rates**.

For production data agents, retain a structured execution contract linking user intent, schema fields, operator plans, assumptions, queries, verification, and the final answer. Evaluate both output accuracy and whether the recorded computation can actually support that output.

The reported **CAIT rates were 45.8–59.1%**, showing that answer-only grading can accept unsupported results. This is an empirical demonstration on one benchmark, so the criterion still needs validation across broader data systems and workloads.