{
  "schema_version": "1.1",
  "id": "s8:https://www.youtube.com/watch?v=xbPriQWXtWM",
  "slug": "the-base-model-is-dead-varun-singh-arcee-ai-02hts76",
  "url": "https://feed7.dev/p/the-base-model-is-dead-varun-singh-arcee-ai-02hts76",
  "title": "The Base Model Is Dead — Varun Singh, Arcee AI",
  "why_included": "Base-model data is shifting from broad web imitation toward code, reasoning, and agent-task priors. The unresolved choice is how early to introduce synthetic and instruction-shaped data.",
  "summary": "The talk contrasts GPT-3’s roughly **85% web-derived mix** with MAI Thinking 1 at **15% web text**. Newer recipes emphasize code, STEM, reasoning traces, and task-shaped data that better prepare models for downstream RL.",
  "practical_implication": "When selecting or training a model for agents, evaluate its pre-RL skill coverage, not just general knowledge. **NeMoTron 3 Ultra** pulls SFT-style data into pre-training, while synthetic rephrasing can expose the same information in several forms.",
  "agent_context": "The talk contrasts GPT-3’s roughly **85% web-derived mix** with MAI Thinking 1 at **15% web text**. Newer recipes emphasize code, STEM, reasoning traces, and task-shaped data that better prepare models for downstream RL.\n\nWhen selecting or training a model for agents, evaluate its pre-RL skill coverage, not just general knowledge. **NeMoTron 3 Ultra** pulls SFT-style data into pre-training, while synthetic rephrasing can expose the same information in several forms.\n\nThere is no settled recipe: MAI Thinking 1 deliberately avoids synthetic model-generated data, while NeMoTron leans into it. Synthetic data can degrade a model when used indiscriminately, and it remains unclear how far RL can displace supervised learning for language.",
  "source": {
    "name": "AI Engineer",
    "url": "https://www.youtube.com/watch?v=xbPriQWXtWM",
    "published_at": "2026-07-31T20:30:21.000Z"
  },
  "source_class": "video",
  "content_type": "Video",
  "layer": "model",
  "domains": [
    "coding"
  ],
  "topics": [
    "reasoning",
    "coding-agents",
    "model-selection"
  ],
  "verification": {
    "status": "source_linked",
    "label": "Source Linked",
    "method": "source_feed",
    "verified_at": null
  },
  "uncertainty": [
    "There is no settled recipe: MAI Thinking 1 deliberately avoids synthetic model-generated data, while NeMoTron leans into it. Synthetic data can degrade a model when used indiscriminately, and it remains unclear how far RL can displace supervised learning for language."
  ],
  "connected_context": {
    "meaning": "This shifts model selection beneath post-training results: agent readiness may depend on whether pre-training already covers code, STEM, reasoning, and task-shaped behavior. It makes gateway tiers, context sizes, and reasoning toggles insufficient selection criteria on their own. The conflicting MAI and NeMoTron recipes also prevent a general rule about synthetic data, so provenance and workload evaluation matter more than adopting either synthetic-heavy or synthetic-free training as doctrine.",
    "corpus_size": 318,
    "generated_at": "2026-08-01T10:08:44.175Z",
    "connections": [
      {
        "title": "Introducing Grok 4.5",
        "source_name": "Cursor",
        "source_url": "https://cursor.com/blog/grok-4-5",
        "feed7_url": "https://feed7.dev/p/grok-4-5-1n0zgxx",
        "reason": "Grok 4.5’s benchmark exclusion because training included an earlier code snapshot reinforces this signal’s demand to inspect pre-training composition and provenance, not interpret downstream scores without qualification."
      },
      {
        "title": "GPT 5.6 Sol, Luna, and Terra now available on AI Gateway",
        "source_name": "Vercel",
        "source_url": "https://vercel.com/changelog/gpt-5-6-now-available-on-ai-gateway",
        "feed7_url": "https://feed7.dev/p/gpt-5-6-now-available-on-ai-gateway-106pgsr",
        "reason": "Gateway tiering offers convenient flagship, balanced, and lower-cost routes, but this signal says those labels must be supplemented by testing whether each model’s pre-RL skill coverage matches the agent workload."
      },
      {
        "title": "DeepSeek V4 Flash now runs updated weights on AI Gateway",
        "source_name": "Vercel",
        "source_url": "https://vercel.com/changelog/deepseek-v4-flash-now-runs-updated-weights-on-ai-gateway",
        "feed7_url": "https://feed7.dev/p/deepseek-v4-flash-now-runs-updated-weights-on-ai-gateway-1qqe8mw",
        "reason": "The reported score jump after a weight replacement shows that underlying training changes can materially alter a fixed model route, while this signal cautions that one downstream benchmark does not reveal the broader data recipe or skill coverage."
      },
      {
        "title": "What's Next After RLHF? — Diogo Almeida, TypeSafe AI",
        "source_name": "AI Engineer",
        "source_url": "https://www.youtube.com/watch?v=cJ0EOzey--o",
        "feed7_url": "https://feed7.dev/p/what-s-next-after-rlhf-diogo-almeida-typesafe-ai-1scytnx",
        "reason": "The RLHF critique supports this signal’s unresolved boundary between supervised learning and RL: post-training for persuasive interaction does not establish the dependable decision-making skills needed for autonomous agents."
      }
    ]
  },
  "lifecycle": "Current",
  "published_at": "2026-07-31T20:30:21.000Z",
  "modified_at": "2026-07-31T20:30:21.000Z",
  "supersedes": [],
  "expires_at": null,
  "formats": {
    "html": "https://feed7.dev/p/the-base-model-is-dead-varun-singh-arcee-ai-02hts76",
    "json": "https://feed7.dev/p/the-base-model-is-dead-varun-singh-arcee-ai-02hts76.json",
    "markdown": "https://feed7.dev/p/the-base-model-is-dead-varun-singh-arcee-ai-02hts76.md"
  }
}