Verifiable Environments for AI in Biology — Kenny Workman, LatchBio
Biology agents need evaluators that verify analysis of large experimental datasets, not recall. LatchBio found human review essential because valid scientific paths can defeat brittle graders.
A single-cell run can produce **2–6 TB**, while a spatial biology run can reach **7 TB**. LatchBio’s Spatial Bench contains **146 problems** with data inputs, scientific tasks, grader configuration, and deterministic checks.
Builders of research agents should require conclusions to come from interacting with the supplied data. Ground truth must remain valid across legitimate analysis paths, and human attempts should test whether deterministic graders reject scientifically sound alternatives.
A single-cell run can produce **2–6 TB**, while a spatial biology run can reach **7 TB**. LatchBio’s Spatial Bench contains **146 problems** with data inputs, scientific tasks, grader configuration, and deterministic checks. Builders of research agents should require conclusions to come from interacting with the supplied data. Ground truth must remain valid across legitimate analysis paths, and human attempts should test whether deterministic graders reject scientifically sound alternatives. End-state rewards become weak as workflows grow longer, and current models still miss full biological tasks. Each long-horizon evaluation reportedly took **three people about a week** to create, showing how expensive durable domain verification can be.
This grounds long-horizon evaluation in a data-heavy scientific domain where multiple valid analysis paths make verification harder than checking one final answer. It confirms the need for inspectable environments and deterministic evidence, while narrowing that prescription: graders must accept scientifically sound alternatives, and constructing durable tasks is itself a major human bottleneck.