{
  "schema_version": "1.0",
  "id": "archive:https://arxiv.org/abs/2607.02502v1",
  "slug": "2607-02502v1-0wngknx",
  "url": "https://feed7.dev/p/2607-02502v1-0wngknx",
  "title": "DemoPSD: Disagreement-Modulated Policy Self-Distillation",
  "why_included": "DemoPSD gates self-distillation per token by teacher–student disagreement, cutting the answer-leakage shortcuts that hurt generalization; beats GRPO and SDPO on science QA in and out of domain.",
  "summary": "**The gist** DemoPSD tweaks on-policy self-distillation, where one LLM acts as both teacher (with privileged information like the answer) and student. Instead of matching the teacher everywhere, the student is pulled toward a **reverse-KL barycenter** — a teacher–student blend weighted per token by how much the two disagree. On **SciKnowEval** (four scientific fields) and out-of-distribution **GPQA**, it beats **GRPO and SDPO** while keeping training entropy higher.",
  "practical_implication": "**Why it matters** On-policy distillation is gaining ground as a cheaper alternative to RL for post-training reasoning models, and this paper names its core failure mode: dense teacher supervision leaks **answer-dependent shortcuts** that vanish at test time and suppresses **exploration**. If you distill or fine-tune your own models, gate teacher guidance selectively; if you only consume models, it explains why distilled ones can ace in-domain evals and stumble elsewhere.",
  "agent_context": "**The gist** DemoPSD tweaks on-policy self-distillation, where one LLM acts as both teacher (with privileged information like the answer) and student. Instead of matching the teacher everywhere, the student is pulled toward a **reverse-KL barycenter** — a teacher–student blend weighted per token by how much the two disagree. On **SciKnowEval** (four scientific fields) and out-of-distribution **GPQA**, it beats **GRPO and SDPO** while keeping training entropy higher.\n\n**Why it matters** On-policy distillation is gaining ground as a cheaper alternative to RL for post-training reasoning models, and this paper names its core failure mode: dense teacher supervision leaks **answer-dependent shortcuts** that vanish at test time and suppresses **exploration**. If you distill or fine-tune your own models, gate teacher guidance selectively; if you only consume models, it explains why distilled ones can ace in-domain evals and stumble elsewhere.\n\n**Watch out** The abstract reports **no concrete numbers, base models, or scale**, results cover scientific QA rather than coding or agent tasks, and this is a **v1 preprint** without peer review.",
  "source": {
    "name": "arXiv",
    "url": "https://arxiv.org/abs/2607.02502v1",
    "published_at": null
  },
  "source_class": "blog_post",
  "content_type": "Paper",
  "layer": "model",
  "domains": [
    "research"
  ],
  "topics": [
    "reasoning"
  ],
  "verification": {
    "status": "needs_review",
    "label": "Needs Review",
    "method": "unverified",
    "verified_at": null
  },
  "uncertainty": [
    "The abstract reports **no concrete numbers, base models, or scale**, results cover scientific QA rather than coding or agent tasks, and this is a **v1 preprint** without peer review."
  ],
  "lifecycle": "Current",
  "published_at": null,
  "modified_at": null,
  "supersedes": [],
  "expires_at": null,
  "formats": {
    "html": "https://feed7.dev/p/2607-02502v1-0wngknx",
    "json": "https://feed7.dev/p/2607-02502v1-0wngknx.json",
    "markdown": "https://feed7.dev/p/2607-02502v1-0wngknx.md"
  }
}