AI EngineerVideoSource Linked

Building GTM AI Agents: Lessons from Deploying to 6,000 Users — Sait Izmit, Snowflake

Snowflake’s rollout favors narrow, high-accuracy coverage, staged adoption, and log-driven iteration over connecting every data source before launch.

AI Engineer · Aug 26, 2026

Open Source Open Markdown Open JSON

Source Summary

Snowflake’s internal GTM assistant has answered about **1.2 million questions** and now handles roughly **40,000 per week**. It expanded to 15 semantic views, 85 tables, 3,000 columns, several MCP connections, and nearly 20 skills after starting with narrower coverage.

Practical Implication

Define representative questions before connecting data, prioritize fewer answers at higher quality, and roll out through pilot, 10% beta, then general availability. Snowflake advanced after beta retention exceeded **70%**, while classified logs exposed missing features and weak answers.

Agent-Ready Context

Snowflake’s internal GTM assistant has answered about **1.2 million questions** and now handles roughly **40,000 per week**. It expanded to 15 semantic views, 85 tables, 3,000 columns, several MCP connections, and nearly 20 skills after starting with narrower coverage.

Define representative questions before connecting data, prioritize fewer answers at higher quality, and roll out through pilot, 10% beta, then general availability. Snowflake advanced after beta retention exceeded **70%**, while classified logs exposed missing features and weak answers.

These are internal deployment figures rather than comparative evals. The architecture is repeatedly revised as models and integrations change, and workflows that write to systems such as Salesforce need curation, role-based access, and additional guardrails.

Connected Context · Feed7 Judgment

This supplies deployment-scale evidence for a narrow-first, evaluation-led rollout: define representative questions, expand coverage only after observed use, and mine classified logs for missing capabilities and weak answers. It confirms production feedback as part of the harness, while limiting the evidence to internal adoption figures and leaving comparative quality, cost, and write-action safety unresolved.

Trading Desks to Clinical Trials: Parallels in Applied Vertical AI — Ayush Bhardwaj, Allos AIBoth start from representative user work rather than broad platform coverage; the vertical-AI case further requires expert judgment where logs and model grading cannot establish usefulness.SimulationMaxxing: How we ship agents 20× faster — Aman Gupta (Nubank) + Shreya Rajpal (Snowglobe)Snowflake relies on staged rollout and production logs, while Nubank shows how simulated traces can move evaluation earlier without replacing the real feedback Snowflake uses.Reinforcement Learning without Verifiable Rewards — Will Brown, Prime IntellectClassified production failures can become the trace-derived tasks and provisional rubrics described here, but both approaches retain human environment design and guardrails where rewards are not verifiable.200 Million Patient Interactions Later — Vivek Muppalla, Hippocratic AIBoth reject a single undifferentiated agent boundary for consequential workflows; the clinical system provides a stronger example of separating specialist checks and offline verification than Snowflake’s still-evolving write controls.

Context Map

agentdata#harness-engineering#agent-evals#agent-reliability

Uncertainty

These are internal deployment figures rather than comparative evals. The architecture is repeatedly revised as models and integrations change, and workflows that write to systems such as Salesforce need curation, role-based access, and additional guardrails.