Beyond Scores: Understanding LLM-as-a-Judge Mechanisms in Summarization Evaluation
Mechanistic tests suggest two open LLM judges route local errors early and form ratings late, giving eval builders concrete layers and components to probe for judge failures.
Across **eight perturbation attacks**, Themis and Prometheus showed a two-stage judging process: attention below layer 15 compared local errors and routed evidence to the last input position, while later MLPs integrated it into a rating. Decisions crystallized at **layer 26 for Themis** and **layer 25 for Prometheus**.
If an LLM judge drives agent evals or training, inspect intermediate behavior rather than trusting its scalar score alone. Controlled corruptions, causal tracing, vocabulary projection, and component knockouts can reveal whether the judge notices an error and where that signal becomes a verdict.
Across **eight perturbation attacks**, Themis and Prometheus showed a two-stage judging process: attention below layer 15 compared local errors and routed evidence to the last input position, while later MLPs integrated it into a rating. Decisions crystallized at **layer 26 for Themis** and **layer 25 for Prometheus**. If an LLM judge drives agent evals or training, inspect intermediate behavior rather than trusting its scalar score alone. Controlled corruptions, causal tracing, vocabulary projection, and component knockouts can reveal whether the judge notices an error and where that signal becomes a verdict. The experiments cover summarization quality in two evaluator models, with one same-scale base-model control. The reported mechanisms therefore should not be assumed to transfer unchanged to other judge architectures, tasks, or rubric formats.