D&A: Decoupling Description and Assessment Enables VLMs as Zero-Shot Robotic Reward Models
D&A splits video progress scoring into scene description and rubric-based assessment, aiming to reduce VLMs’ tendency to mistake failed robot runs for progress.
**D&A (Describe & Assess)** first produces a task-grounded description of scene dynamics, then scores progress against an explicit rubric. The authors report that this prompt-only method captures partial progress and penalizes failed robot executions.
For agents that evaluate video or tool outcomes, define what success means independently of the clip’s apparent trajectory, then assess evidence against that criterion.
**D&A (Describe & Assess)** first produces a task-grounded description of scene dynamics, then scores progress against an explicit rubric. The authors report that this prompt-only method captures partial progress and penalizes failed robot executions. For agents that evaluate video or tool outcomes, define what success means independently of the clip’s apparent trajectory, then assess evidence against that criterion. The reported results concern zero-shot robotic task-progress estimates. The supplied material gives no numeric results or details about how performance varies by task, embodiment, or benchmark.
This gives a concrete video-evaluation pattern: describe task-relevant scene changes, then assess them against an explicit rubric so apparent progress does not substitute for success criteria. Dyna’s robotics account also emphasizes progress scoring and inspected trajectories, making scorer repeatability relevant; the supplied brief gives no numeric evidence about D&A’s performance.