Sketching Language Models for Context Attribution
A sketching method estimates how much each prompt token affects a model’s output with far fewer evaluations than removing tokens one by one, under a stability assumption.
The paper studies leave-one-out context attribution: measuring how removing each prompt token changes a model output. Direct evaluation takes **n calls** for n tokens; under a stability assumption, the proposed sketching method uses **O(√log(1/ε)/ε) calls** to estimate all effects.
For builders investigating prompts or retrieval context, this could make token-level attribution less costly. The method may help identify which context contributes to a chosen output measurement.
The paper studies leave-one-out context attribution: measuring how removing each prompt token changes a model output. Direct evaluation takes **n calls** for n tokens; under a stability assumption, the proposed sketching method uses **O(√log(1/ε)/ε) calls** to estimate all effects. For builders investigating prompts or retrieval context, this could make token-level attribution less costly. The method may help identify which context contributes to a chosen output measurement. The bound depends on the **stability assumption**. The reported empirical check uses a small language-model retrieval task, so the material does not show how well the method works across larger models or production prompts.
Sketching Language Models offers a lower-call way to estimate how individual prompt tokens affect an output, subject to a stability assumption. The small retrieval experiment makes this a possible context-debugging tool, not yet a demonstrated method for larger models or production prompts.