{
  "schema_version": "1.1",
  "id": "archive:https://www.youtube.com/watch?v=nKjPBAw_N_0",
  "slug": "how-to-build-a-software-factory-that-improves-itself-zach-lloyd-warp-050811m",
  "url": "https://feed7.dev/p/how-to-build-a-software-factory-that-improves-itself-zach-lloyd-warp-050811m",
  "title": "How to Build a Software Factory That Improves Itself - Zach Lloyd, Warp",
  "why_included": "Warp’s Zach Lloyd argues that teams can govern and improve coding agents by running them in sandboxed software factories, with workflows and evaluation defined in code. He recommends testing models on real team tasks.",
  "summary": "Lloyd describes a shift from engineers prompting agents on laptops to **sandboxed cloud factories**. He proposes keeping workflows, agent definitions, skills, MCPs, and benchmarks in a versioned repository, then using recorded runs and task-specific evaluations to improve the system. One example compares models on replayed coding tasks by cost and judged correctness.",
  "practical_implication": "If you run agents across a team, define the workflow and controls centrally, capture past tasks, and build a benchmark from representative work. Use those results to guide model routing and factory changes instead of relying only on public benchmarks.",
  "agent_context": "Lloyd describes a shift from engineers prompting agents on laptops to **sandboxed cloud factories**. He proposes keeping workflows, agent definitions, skills, MCPs, and benchmarks in a versioned repository, then using recorded runs and task-specific evaluations to improve the system. One example compares models on replayed coding tasks by cost and judged correctness.\n\nIf you run agents across a team, define the workflow and controls centrally, capture past tasks, and build a benchmark from representative work. Use those results to guide model routing and factory changes instead of relying only on public benchmarks.\n\nThe talk presents Warp Factories as one implementation, but the factory approach is broader. Evaluation can use an LLM judge, which Lloyd says may be supplemented by human grading; the examples do not establish that this approach fits every team or task.",
  "source": {
    "name": "AI Engineer",
    "url": "https://www.youtube.com/watch?v=nKjPBAw_N_0",
    "published_at": "2026-11-09T08:00:17.000Z"
  },
  "source_class": "video",
  "content_type": "Video",
  "layer": "agent",
  "domains": [
    "coding"
  ],
  "topics": [
    "harness-engineering",
    "cloud-agents",
    "agent-evals"
  ],
  "verification": {
    "status": "source_linked",
    "label": "Source Linked",
    "method": "source_feed",
    "verified_at": null
  },
  "uncertainty": [
    "The talk presents Warp Factories as one implementation, but the factory approach is broader. Evaluation can use an LLM judge, which Lloyd says may be supplemented by human grading; the examples do not establish that this approach fits every team or task."
  ],
  "connected_context": {
    "meaning": "Lloyd’s factory proposal connects centralized, versioned workflows with replayed, task-specific evaluation for routing and improvement. The candidates sharpen its limits and complements: production traces can seed evaluations, scores depend on harnesses, and stronger workflow automation still needs human ownership and task-relevant evidence.",
    "corpus_size": 1045,
    "generated_at": "2026-11-10T11:03:25.798Z",
    "connections": [
      {
        "title": "Agents That Self Improve While You Sleep - Fuad Ali, Arize AI",
        "source_name": "AI Engineer",
        "source_url": "https://www.youtube.com/watch?v=UGuNt2_Oz8s",
        "feed7_url": "https://feed7.dev/p/agents-that-self-improve-while-you-sleep-fuad-ali-arize-ai-19hmq9b",
        "reason": "Both use recorded runs to improve agent workflows; this candidate specifies a trace-to-fix loop that promotes recurring failures into regression evaluations with human review."
      },
      {
        "title": "Improving Agents is a Data Mining Problem — Vivek Trivedy, LangChain",
        "source_name": "AI Engineer",
        "source_url": "https://www.youtube.com/watch?v=CvRngaQZQ3Y",
        "feed7_url": "https://feed7.dev/p/improving-agents-is-a-data-mining-problem-vivek-trivedy-langchain-1cgibr8",
        "reason": "Its trace-mining approach supplies another source for the factory’s representative tasks and evaluations, while explicitly flagging misleading proxies and trace-scale costs."
      },
      {
        "title": "Designing Agents (The Floor Is the Frontier) — Ben Hylak, Raindrop",
        "source_name": "AI Engineer",
        "source_url": "https://www.youtube.com/watch?v=jHMiYtjoJfA",
        "feed7_url": "https://feed7.dev/p/designing-agents-the-floor-is-the-frontier-ben-hylak-raindrop-0uoems4",
        "reason": "This candidate narrows how factory benchmarks should be read: harness-coupled evaluations can decay, so production failures and explicit metrics should anchor checks."
      },
      {
        "title": "Harness Engineering is not Enough: Why Software Factories Fail — Dex Horthy, HumanLayer",
        "source_name": "AI Engineer",
        "source_url": "https://www.youtube.com/watch?v=Ib5GBkD555M",
        "feed7_url": "https://feed7.dev/p/harness-engineering-is-not-enough-why-software-factories-fail-dex-horthy-1tj1amt",
        "reason": "The factory’s throughput and automation proposal is qualified by this candidate’s emphasis on maintainability, upfront alignment, and continued human ownership of code."
      }
    ]
  },
  "lifecycle": "Current",
  "published_at": "2026-11-09T08:00:17.000Z",
  "modified_at": "2026-11-09T08:00:17.000Z",
  "supersedes": [],
  "expires_at": null,
  "formats": {
    "html": "https://feed7.dev/p/how-to-build-a-software-factory-that-improves-itself-zach-lloyd-warp-050811m",
    "json": "https://feed7.dev/p/how-to-build-a-software-factory-that-improves-itself-zach-lloyd-warp-050811m.json",
    "markdown": "https://feed7.dev/p/how-to-build-a-software-factory-that-improves-itself-zach-lloyd-warp-050811m.md"
  }
}