{
  "schema_version": "1.1",
  "id": "archive:https://arxiv.org/abs/2611.09872v1",
  "slug": "2611-09872v1-0hxr9dl",
  "url": "https://feed7.dev/p/2611-09872v1-0hxr9dl",
  "title": "On the Reliability of Majority-Vote Pseudo-Labels in Test-Time Reinforcement Learning",
  "why_included": "This TTRL study finds majority-vote pseudo-label reliability varies by prompt and tests a more conservative DPO update. Its method matched or beat the strongest baseline in 24 of 30 settings.",
  "summary": "The paper separates majority-vote error into population-majority bias and prompt-dependent finite-rollout variation. Its **PC-Dr.DPO** method uses observed vote reliability to make updates more conservative for sampling-sensitive prompts, with a cache mechanism to reduce forgetting.",
  "practical_implication": "Builders adapting models from unlabeled data should account for variation in vote reliability across prompts rather than treating every majority vote as equally dependable. The authors report matching or outperforming the strongest TTRL baseline in **24 of 30 settings**.",
  "agent_context": "The paper separates majority-vote error into population-majority bias and prompt-dependent finite-rollout variation. Its **PC-Dr.DPO** method uses observed vote reliability to make updates more conservative for sampling-sensitive prompts, with a cache mechanism to reduce forgetting.\n\nBuilders adapting models from unlabeled data should account for variation in vote reliability across prompts rather than treating every majority vote as equally dependable. The authors report matching or outperforming the strongest TTRL baseline in **24 of 30 settings**.\n\nPopulation-majority bias is generally unidentifiable without external supervision, so the method targets finite-rollout variation instead. Results cover **six benchmarks** and **five model backbones**; the abstract does not establish how the approach transfers beyond them.",
  "source": {
    "name": "arXiv",
    "url": "https://arxiv.org/abs/2611.09872v1",
    "published_at": "2026-11-06T18:49:02.000Z"
  },
  "source_class": "blog_post",
  "content_type": "Paper",
  "layer": "benchmark",
  "domains": [
    "research"
  ],
  "topics": [
    "agent-evals",
    "agent-reliability"
  ],
  "verification": {
    "status": "needs_review",
    "label": "Needs Review",
    "method": "unverified",
    "verified_at": null
  },
  "uncertainty": [
    "Population-majority bias is generally unidentifiable without external supervision, so the method targets finite-rollout variation instead. Results cover **six benchmarks** and **five model backbones**; the abstract does not establish how the approach transfers beyond them."
  ],
  "connected_context": {
    "meaning": "This adds a training-time response to uneven pseudo-label reliability: estimate which prompts are sensitive to finite sampling and make updates more conservative there. It complements benchmark work that treats reliability as task- or workflow-dependent, while its unidentifiable population-majority bias limits what the method can correct.",
    "corpus_size": 1032,
    "generated_at": "2026-11-09T11:01:38.055Z",
    "connections": [
      {
        "title": "Patient-specific rubric-based evaluation of large language models across longitudinal cancer care",
        "source_name": "arXiv",
        "source_url": "https://arxiv.org/abs/2610.35847v1",
        "feed7_url": "https://feed7.dev/p/2610-35847v1-0se27hb",
        "reason": "Both show that reliability varies across cases; this paper uses prompt-level vote sensitivity to adjust training, while NPC-RubricBench measures failures across care phases."
      },
      {
        "title": "Domain-Specific Hallucination Detection in Large Language Models",
        "source_name": "arXiv",
        "source_url": "https://arxiv.org/abs/2609.11878v1",
        "feed7_url": "https://feed7.dev/p/2609-11878v1-1ltrudx",
        "reason": "Both caution against treating one confidence or evaluation signal as universally dependable; this paper measures vote sensitivity, while the detector study finds transfer varies by domain."
      },
      {
        "title": "BLOOM-WILT: Logit Tilting for Behaviour Elicitation in Automated LLM Auditing",
        "source_name": "arXiv",
        "source_url": "https://arxiv.org/abs/2608.31105v1",
        "feed7_url": "https://feed7.dev/p/2608-31105v1-0vdo3q3",
        "reason": "Both narrow reliability claims to what their method can observe: finite-rollout variation here, and elicitable behaviors under logit access in BLOOM-WILT."
      }
    ]
  },
  "lifecycle": "Current",
  "published_at": "2026-11-06T18:49:02.000Z",
  "modified_at": "2026-11-06T18:49:02.000Z",
  "supersedes": [],
  "expires_at": null,
  "formats": {
    "html": "https://feed7.dev/p/2611-09872v1-0hxr9dl",
    "json": "https://feed7.dev/p/2611-09872v1-0hxr9dl.json",
    "markdown": "https://feed7.dev/p/2611-09872v1-0hxr9dl.md"
  }
}