Teaching AI to Find Real Vulnerabilities — David Brumley, Bugcrowd
Cybersecurity agents need deterministic exploit oracles and tasks that reward finding distinct vulnerabilities, not self-reported wins or one crash. A capability ladder makes partial progress measurable.
Brumley describes reproducible vulnerable containers exposed through MCP, with setup, sandboxed file tools, and deterministic grading. He argues that crash-only checks miss exploit quality and duplicate findings; **50%** of DARPA Cyber Grand Challenge tasks reportedly contained unintended vulnerabilities, while AIxCC produced **18 unintended bugs**.
For security-agent evals, grade concrete effects rather than model claims, deduplicate proofs by the vulnerabilities they trigger, and measure precision and recall across an audit set. A **16-capability ladder** can show where an agent stalls between triggering a flaw and building an out-of-sandbox exploit.
Brumley describes reproducible vulnerable containers exposed through MCP, with setup, sandboxed file tools, and deterministic grading. He argues that crash-only checks miss exploit quality and duplicate findings; **50%** of DARPA Cyber Grand Challenge tasks reportedly contained unintended vulnerabilities, while AIxCC produced **18 unintended bugs**. For security-agent evals, grade concrete effects rather than model claims, deduplicate proofs by the vulnerabilities they trigger, and measure precision and recall across an audit set. A **16-capability ladder** can show where an agent stalls between triggering a flaw and building an out-of-sandbox exploit. Even deterministic oracles encode a chosen definition of progress, and newly discovered bugs can change the ground truth. Publishing transcripts is also unresolved when a benchmark agent produces weaponized, previously private exploits.
This sharpens general agent-eval guidance for offensive security: success should be graded by reproducible effects, duplicate proofs should collapse to the vulnerability they exercise, and capability stages should separate triggering a flaw from achieving a real exploit. It also exposes a domain-specific limit of fixed ground truth: unintended vulnerabilities can invalidate labels, while transcript publication may disclose weaponized results.