Eigen RadarAI
Analysis

A context-attribution method estimates the effect of removing prompt segments

A preprint proposes estimating how each segment of a prompt affects a fixed answer without rerunning every deletion separately. The method combines two differentiation techniques and measures uncertain effects directly. A small language-model demonstration illustrates the estimator; its theoretical error bound depends on a stability assumption about the model’s response to these changes.

Artificial Intelligence··Night
An empty computing research room: twelve blank cards in a row on a whiteboard, the seventh amber, two removed onto the ledge; in front, a desk and a workstation tower with a single graphics card.

A fixed answer provides the measurement

A new context-attribution preprint estimates how removing parts of a prompt changes a language model’s score for an already chosen answer. The demonstration gives Llama-3.2-1B-Instruct twelve context chunks, with the answer in the seventh and irrelevant information elsewhere. Researchers keep the completion fixed and add together the log-probabilities assigned to its tokens. The attribution therefore concerns a specified numerical response to deleting context rather than asking the model to generate a fresh answer each time.[1]

Differentiation approximates many deletions

A circuit obtained by backpropagation becomes the input to a forward differentiation calculation. Randomized sketches approximate individual deletion effects while reducing sampling variability relative to a forward-only approach. Estimates with high uncertainty can be evaluated exactly. Its theoretical error bound assumes a stability condition on higher-order Taylor coefficients. The authors distinguish uncertainty from sampling from error introduced by truncating the expansion and from trimming observations.[1]

Small trials examine estimator behavior

The numerical implementation runs 128 independent trials with eight sketches per trial. It removes the two extreme observations, then computes a mean from the six others, using 64-bit precision. Each trial uses one H100 graphics processor and the JAX computation framework. The authors describe the demonstration as a minimal proof of concept and note that directly testing every deletion may be faster at this small scale. Observed coefficient decay is consistent with their stability assumption.[1]

References

  1. News sourcearXivContext attribution method approximates deletion experiments↩1↩2↩3