{
  "schema_version": "1.0",
  "id": "archive:https://arxiv.org/abs/2607.05391v1",
  "slug": "2607-05391v1-10hi3eo",
  "url": "https://feed7.dev/p/2607-05391v1-10hi3eo",
  "title": "LLM-as-a-Verifier: A General-Purpose Verification Framework",
  "why_included": "Training-free framework that turns an LLM judge's token logits into continuous scores for verifying agent outputs — 78.2% on SWE-Bench Verified, 86.5% on Terminal-Bench V2 — and ships a Claude Code extension.",
  "summary": "**The gist** LLM-as-a-Verifier scores candidate solutions by taking the **expectation over scoring-token logits**, producing continuous rather than discrete grades with no extra training. Scaled via granularity, repeated evaluation, and criteria decomposition, it reaches **86.5% on Terminal-Bench V2**, **78.2% on SWE-Bench Verified**, and **87.4% on RoboRewardBench**, with a cost-aware algorithm for ranking candidates.",
  "practical_implication": "**Why it matters** A training-free verifier is the piece that makes best-of-N agent runs and self-checking loops pay off. The authors ship a **Claude Code extension** that turns the scores into a live task-progress signal for monitoring agentic systems, and the same dense feedback improves RL sample efficiency for **SAC and GRPO**.",
  "agent_context": "**The gist** LLM-as-a-Verifier scores candidate solutions by taking the **expectation over scoring-token logits**, producing continuous rather than discrete grades with no extra training. Scaled via granularity, repeated evaluation, and criteria decomposition, it reaches **86.5% on Terminal-Bench V2**, **78.2% on SWE-Bench Verified**, and **87.4% on RoboRewardBench**, with a cost-aware algorithm for ranking candidates.\n\n**Why it matters** A training-free verifier is the piece that makes best-of-N agent runs and self-checking loops pay off. The authors ship a **Claude Code extension** that turns the scores into a live task-progress signal for monitoring agentic systems, and the same dense feedback improves RL sample efficiency for **SAC and GRPO**.\n\n**Watch out** The method needs **logprob access** to scoring tokens, which not every hosted API exposes, and its gains come from spending more judge calls — repeated evaluation and criteria decomposition multiply **verification cost**. Judge-side bias isn't addressed in the material.",
  "source": {
    "name": "arXiv",
    "url": "https://arxiv.org/abs/2607.05391v1",
    "published_at": null
  },
  "source_class": "blog_post",
  "content_type": "Paper",
  "layer": "benchmark",
  "domains": [
    "coding",
    "research"
  ],
  "topics": [
    "agent-evals",
    "agent-reliability",
    "harness-engineering"
  ],
  "verification": {
    "status": "needs_review",
    "label": "Needs Review",
    "method": "unverified",
    "verified_at": null
  },
  "uncertainty": [
    "The method needs **logprob access** to scoring tokens, which not every hosted API exposes, and its gains come from spending more judge calls — repeated evaluation and criteria decomposition multiply **verification cost**. Judge-side bias isn't addressed in the material."
  ],
  "lifecycle": "Current",
  "published_at": null,
  "modified_at": null,
  "supersedes": [],
  "expires_at": null,
  "formats": {
    "html": "https://feed7.dev/p/2607-05391v1-10hi3eo",
    "json": "https://feed7.dev/p/2607-05391v1-10hi3eo.json",
    "markdown": "https://feed7.dev/p/2607-05391v1-10hi3eo.md"
  }
}