Show-Harness: Just a VLM Agent Can Play Robots
Show-Harness puts a compact semantic action layer between VLM intent and robot-specific control, a harness pattern for reusing frontier or small open models across embodiments.
Show-Harness exposes **discrete semantic action units** that a VLM selects, then uses embodiment-specific interpreters to ground them deterministically into local robot actions. **GUMI** applies the same action space to GUI-based demonstrations without specialized teleoperation hardware.
The transferable lesson for agent builders is to place a small, model-readable interface above deterministic adapters. The paper reports zero-shot use of closed frontier VLMs and adaptation of small open VLMs with **a few GPU-hours** of fine-tuning.
Show-Harness exposes **discrete semantic action units** that a VLM selects, then uses embodiment-specific interpreters to ground them deterministically into local robot actions. **GUMI** applies the same action space to GUI-based demonstrations without specialized teleoperation hardware. The transferable lesson for agent builders is to place a small, model-readable interface above deterministic adapters. The paper reports zero-shot use of closed frontier VLMs and adaptation of small open VLMs with **a few GPU-hours** of fine-tuning. The supplied abstract claims gains across tasks, robots, and environments, but provides no numerical results or deployment failure analysis. The approach still leaves fine-grained physical decisions with the VLM, so real-world safety remains an open concern.
This confirms a reusable harness pattern beyond software: expose a compact semantic action vocabulary to the model and keep embodiment-specific execution in deterministic adapters. GUMI broadens how demonstrations can be collected, while the missing numerical and failure evidence prevents treating interface simplification as proof of safe or reliable physical autonomy.