# Teaching AI to Find Real Vulnerabilities — David Brumley, Bugcrowd

Source: [AI Engineer](https://www.youtube.com/watch?v=ZFxh7sqbUZo)  
Feed7 permalink: https://feed7.dev/p/teaching-ai-to-find-real-vulnerabilities-david-brumley-bugcrowd-1ok0f7q  
Published: 2026-08-01T00:30:06.000Z  
Trust: Source Linked (source_linked)

## Why Included

Cybersecurity agents need deterministic exploit oracles and tasks that reward finding distinct vulnerabilities, not self-reported wins or one crash. A capability ladder makes partial progress measurable.

## Source Summary

Brumley describes reproducible vulnerable containers exposed through MCP, with setup, sandboxed file tools, and deterministic grading. He argues that crash-only checks miss exploit quality and duplicate findings; **50%** of DARPA Cyber Grand Challenge tasks reportedly contained unintended vulnerabilities, while AIxCC produced **18 unintended bugs**.

## Practical Implication

For security-agent evals, grade concrete effects rather than model claims, deduplicate proofs by the vulnerabilities they trigger, and measure precision and recall across an audit set. A **16-capability ladder** can show where an agent stalls between triggering a flaw and building an out-of-sandbox exploit.

## Agent-Ready Context

Brumley describes reproducible vulnerable containers exposed through MCP, with setup, sandboxed file tools, and deterministic grading. He argues that crash-only checks miss exploit quality and duplicate findings; **50%** of DARPA Cyber Grand Challenge tasks reportedly contained unintended vulnerabilities, while AIxCC produced **18 unintended bugs**.

For security-agent evals, grade concrete effects rather than model claims, deduplicate proofs by the vulnerabilities they trigger, and measure precision and recall across an audit set. A **16-capability ladder** can show where an agent stalls between triggering a flaw and building an out-of-sandbox exploit.

Even deterministic oracles encode a chosen definition of progress, and newly discovered bugs can change the ground truth. Publishing transcripts is also unresolved when a benchmark agent produces weaponized, previously private exploits.

## Connected Context

Feed7 judgment across 318 accumulated Signals:

This sharpens general agent-eval guidance for offensive security: success should be graded by reproducible effects, duplicate proofs should collapse to the vulnerability they exercise, and capability stages should separate triggering a flaw from achieving a real exploit. It also exposes a domain-specific limit of fixed ground truth: unintended vulnerabilities can invalidate labels, while transcript publication may disclose weaponized results.

- [Rethinking Environments for Long-Horizon Work — Rayan Garg, Theta Software](https://feed7.dev/p/rethinking-environments-for-long-horizon-work-rayan-garg-theta-software-11r7wbx) — Both reject proxy measures in favor of inspecting consequential state; here that principle becomes grading concrete exploit effects rather than model claims or nominal task duration.
- [Verifiable Environments for AI in Biology — Kenny Workman, LatchBio](https://feed7.dev/p/verifiable-environments-for-ai-in-biology-kenny-workman-latchbio-1vs6y66) — The biology candidate’s brittle-grader problem parallels security benchmarks where valid unintended vulnerabilities can defeat a fixed oracle, supporting human review when ground truth changes.
- [Vending-Bench: Long-Horizon Agent Evals — Lukas Petersson, Andon Labs](https://feed7.dev/p/vending-bench-long-horizon-agent-evals-lukas-petersson-andon-labs-0fu78nz) — Vending-Bench’s warning about agents exploiting evaluations is reinforced by security tasks containing unintended attack paths, making real-world checks and trajectory inspection important complements to deterministic containers.
- [Win by Silence: Deletion Non-Monotonicity, Autonomous Exploitation, and Typed-State Gating in LLM Plan Evaluation](https://feed7.dev/p/2607-12986v1-13g75k2) — Typed-state gating and the 16-capability ladder share a structural remedy: withhold credit until required intermediate states are demonstrated, rather than rewarding an incomplete plan or crash-only result.

## Context Map

- Layer: benchmark
- Domains: security
- Topics: agent-evals, benchmark-integrity, agent-reliability

## Uncertainty

- Even deterministic oracles encode a chosen definition of progress, and newly discovered bugs can change the ground truth. Publishing transcripts is also unresolved when a benchmark agent produces weaponized, previously private exploits.

## Agent Instruction

Use this item as source-backed context. Do not invent claims beyond the linked source. If this item conflicts with another source, call out the conflict.