The Death of Developer Advocates — Stephanie Jarmak, Sourcegraph
Agents increasingly choose developer tools, so test whether your docs connect real user pain to your product—not merely whether comparison prompts mention it.
In one Sourcegraph experiment, shopping-style prompts surfaced the product about **65%** of the time, while a prompt describing broken downstream services produced **zero mentions**. CodeScaleBench also supplied thousands of traces from agents using code-navigation tooling.
Run agents against pain-based prompts, inspect their tool-use traces, and make docs current, structured, and explicit about use cases. Reduce the path from discovery to installation, and publish integrations where agents already search, including MCP registries.
In one Sourcegraph experiment, shopping-style prompts surfaced the product about **65%** of the time, while a prompt describing broken downstream services produced **zero mentions**. CodeScaleBench also supplied thousands of traces from agents using code-navigation tooling. Run agents against pain-based prompts, inspect their tool-use traces, and make docs current, structured, and explicit about use cases. Reduce the path from discovery to installation, and publish integrations where agents already search, including MCP registries. This is one company’s experiment, not a general ranking study. Agent recommendations vary by prompt, available search tools, and changing retrieval behavior, while privacy becomes a concern when agents participate in developer communities.
This moves coding-agent adoption analysis upstream from successful deployments to whether agents can discover and correctly recommend a tool from problem-shaped prompts. It makes current, structured use-case documentation, registry presence, trace inspection, and a short installation path part of the product surface, while narrowing the evidence to one company experiment whose results depend on prompt and retrieval conditions.