Patient-specific rubric-based evaluation of large language models across longitudinal cancer care
NPC-RubricBench finds that clinical LLM failure modes shift across cancer-care phases. For agent builders, it shows why a strong score on one task may not predict reliable performance across a workflow.
NPC-RubricBench evaluates **30 LLMs** across six cancer-care phases, using records from **405 patients** and 17,688 patient-specific criteria. Errors dominated anatomical staging, while omissions were more common in initial work-up and follow-up.
For agents that operate across a workflow, evaluate each phase and decision point instead of relying on a single diagnostic score. The study also finds a prompt trade-off: checklist-like requests raised decision quality but reduced evidence faithfulness and safety adherence.
NPC-RubricBench evaluates **30 LLMs** across six cancer-care phases, using records from **405 patients** and 17,688 patient-specific criteria. Errors dominated anatomical staging, while omissions were more common in initial work-up and follow-up. For agents that operate across a workflow, evaluate each phase and decision point instead of relying on a single diagnostic score. The study also finds a prompt trade-off: checklist-like requests raised decision quality but reduced evidence faithfulness and safety adherence. This benchmark covers nasopharyngeal cancer care at two centers. Its results do not establish performance in other diseases or clinical settings.
This adds a patient-specific, phase-by-phase view of clinical agent quality: staging errors and omissions vary across care stages, and checklist prompting trades better decision quality for weaker evidence faithfulness and safety adherence. It supports measuring consequential errors at each workflow step, while its two-center nasopharyngeal cancer scope limits broader claims.