Don't Ship Skills Without Evals — Philipp Schmid, Google DeepMind

A cited scan indexed more than **50,000 skills** and found almost none had evals. DeepMind keeps tests beside each internal skill, runs them on every change, and checks triggers, commands, traces, and outputs with scripts or an LLM judge.

Start with positive and negative prompts, then add production traces. Run **2–6 trials per case**, test across the models and harnesses you support, and compare results with the skill enabled and removed; deterministic workflows may belong in scripts instead.

Agent runs are nondeterministic, so a single pass proves little. Regex checks are cheap but narrow, while judge models add cost and judgment variance; retained evals remain useful after retiring a skill because they can expose later model regressions.