# Desktop-Delta Bench: Do Computer-Use Models Understand Desktop GUI Transitions?

Source: [arXiv](https://arxiv.org/abs/2607.26041v1)  
Feed7 permalink: https://feed7.dev/p/2607-26041v1-1x1gw81  
Published: 2026-07-28T17:49:51.000Z  
Trust: Needs Review (needs_review)

## Why Included

Desktop-Delta Bench tests whether computer-use models can verify what changed after an action, exposing a reliability gap hidden by end-task and single-frame scores.

## Source Summary

Desktop-Delta Bench contains **2,013 human-verified instances** from multi-app Linux trajectories across about 15 applications and 50 task domains. It tests temporal ordering and action inference from before-and-after observations.

## Practical Implication

Use step-level transition checks when evaluating computer-use agents, especially around stale screenshots, delayed rendering, and recovery. The best ordering exact-match rates reached only **65.1% without decoys** and **65.7% with decoys**.

## Agent-Ready Context

Desktop-Delta Bench contains **2,013 human-verified instances** from multi-app Linux trajectories across about 15 applications and 50 task domains. It tests temporal ordering and action inference from before-and-after observations.

Use step-level transition checks when evaluating computer-use agents, especially around stale screenshots, delayed rendering, and recovery. The best ordering exact-match rates reached only **65.1% without decoys** and **65.7% with decoys**.

This is an offline diagnostic rather than an end-to-end task benchmark. Action recognition also varies sharply: reported click and drag scores were **0.96 versus 0.76 F1**, so aggregate GUI grounding can hide operation-specific weaknesses.

## Connected Context

Feed7 judgment across 262 accumulated Signals:

Desktop-Delta Bench narrows the evaluation gap identified by broader coding and rollout benchmarks to a specific failure surface: whether an agent understands the transition between GUI states. Its low ordering ceiling and operation-specific score spread support adding step-level temporal and action checks to reproducible trajectory evaluation, rather than treating end-to-end completion or aggregate grounding as sufficient.

- [Everything Is a Rollout — Alex Shaw + Ryan Marten, Terminal-Bench, Harbor, Laude Institute](https://feed7.dev/p/everything-is-a-rollout-alex-shaw-ryan-marten-terminal-bench-harbor-laud-0iz4rgx) — Harbor’s trajectory inspection provides the evaluation loop into which Desktop-Delta’s step-level transition checks can fit.
- [From Agent Traces to Agent Simulations — Rustem Feyzkhanov, Snorkel AI](https://feed7.dev/p/from-agent-traces-to-agent-simulations-rustem-feyzkhanov-snorkel-ai-0zwlzjq) — Replayable production traces could supply fixed GUI transitions for diagnosing the stale-state, rendering-delay, and recovery failures this benchmark isolates.
- [DeepSWE: A Contamination-Resistant Coding Benchmark — James Shi, Datacurve](https://feed7.dev/p/deepswe-a-contamination-resistant-coding-benchmark-james-shi-datacurve-08p61c0) — Both show that end scores can conceal behavior-level weaknesses, but Desktop-Delta extends trajectory-sensitive evaluation from repository work to desktop state transitions.
- [How Evals and Prompts Shape Agent Behavior — Preetika Bhateja & Daniel Bump, YouTube Ads](https://feed7.dev/p/how-evals-and-prompts-shape-agent-behavior-preetika-bhateja-daniel-bump-1cmecaw) — Desktop-Delta supplies the small, behavior-specific transition cases that can make trace review and calibrated evaluation more diagnostic for computer-use agents.

## Context Map

- Layer: benchmark
- Domains: coding
- Topics: computer-use, agent-evals, agent-reliability

## Uncertainty

- This is an offline diagnostic rather than an end-to-end task benchmark. Action recognition also varies sharply: reported click and drag scores were **0.96 versus 0.76 F1**, so aggregate GUI grounding can hide operation-specific weaknesses.

## Agent Instruction

Use this item as source-backed context. Do not invent claims beyond the linked source. If this item conflicts with another source, call out the conflict.