Evals-Driven Development for a Mental Health AI Coach — Akele Reed & Dave Revere, SonderMind
SonderMind turns clinician-reviewed failures into release-gating evals, keeping mental-health guardrails modular and testing false positives, false negatives, category, and timing.
SonderMind built **separate input and output guardrail LLMs** around its coaching model instead of relying on general-purpose moderation. Flagged conversations are reviewed by clinicians, normalized into **typed evals**, and committed as **CI release gates**.
For high-stakes agents, trace ambiguous failures and make domain experts responsible for defining acceptable behavior. Evaluate whether the correct category fired at the correct conversational moment, including both false negatives and harmful over-triggering.
SonderMind built **separate input and output guardrail LLMs** around its coaching model instead of relying on general-purpose moderation. Flagged conversations are reviewed by clinicians, normalized into **typed evals**, and committed as **CI release gates**. For high-stakes agents, trace ambiguous failures and make domain experts responsible for defining acceptable behavior. Evaluate whether the correct category fired at the correct conversational moment, including both false negatives and harmful over-triggering. The talk provides an engineering process, not measured safety rates or a claim of perfect detection. Clinical language can remain ambiguous, separate judge calls add latency and cost, and benchmark optimization can drift away from the people the tests are meant to protect.
This operationalizes high-stakes safety evaluation as a clinician-owned production loop: ambiguous conversations become typed tests, and those tests become release gates around specialized input and output guardrails. It strengthens the case for trace-derived evals while narrowing claims to process quality, since no safety rates establish detection performance and judge calls introduce cost, latency, and their own failure modes.