{
  "schema_version": "1.1",
  "id": "archive:https://arxiv.org/abs/2610.42112v1",
  "slug": "2610-42112v1-1k0vz2w",
  "url": "https://feed7.dev/p/2610-42112v1-1k0vz2w",
  "title": "Subquadratic Attention with a Hierarchical Tree-Structured Indexer",
  "why_included": "ZHA uses a tree index over compressed KV-cache summaries to retrieve sparse keys with subquadratic attention scaling. The paper reports maintained results on long-context and agentic coding evaluations; million-token performance is a scaling claim.",
  "summary": "Zyphra Hierarchical Attention (ZHA) builds a tree of compressed key summaries over the KV cache, then routes queries through it to select keys for sparse attention. The authors describe its total cost as **subquadratic**, and report applying it to pretrained models with a short KL distillation stage over **a few billion tokens**.",
  "practical_implication": "For coding agents that work with very long histories, the method points to an alternative to repeatedly attending over every cached token. The paper reports maintained performance on long-context retrieval and agentic coding evaluations, which makes the approach relevant to future long-context model choices.",
  "agent_context": "Zyphra Hierarchical Attention (ZHA) builds a tree of compressed key summaries over the KV cache, then routes queries through it to select keys for sparse attention. The authors describe its total cost as **subquadratic**, and report applying it to pretrained models with a short KL distillation stage over **a few billion tokens**.\n\nFor coding agents that work with very long histories, the method points to an alternative to repeatedly attending over every cached token. The paper reports maintained performance on long-context retrieval and agentic coding evaluations, which makes the approach relevant to future long-context model choices.\n\nThe authors’ performance model projects an advantage beyond **1 million tokens** and suggests potential contexts in the hundreds of millions or billions. Those longer ranges are projections; the material does not establish that such context lengths are available in deployed models.",
  "source": {
    "name": "arXiv",
    "url": "https://arxiv.org/abs/2610.42112v1",
    "published_at": "2026-10-30T17:48:37.000Z"
  },
  "source_class": "blog_post",
  "content_type": "Paper",
  "layer": "model",
  "domains": [
    "coding",
    "research"
  ],
  "topics": [
    "context-engineering",
    "agent-evals"
  ],
  "verification": {
    "status": "needs_review",
    "label": "Needs Review",
    "method": "unverified",
    "verified_at": null
  },
  "uncertainty": [
    "The authors’ performance model projects an advantage beyond **1 million tokens** and suggests potential contexts in the hundreds of millions or billions. Those longer ranges are projections; the material does not establish that such context lengths are available in deployed models."
  ],
  "connected_context": {
    "meaning": "ZHA moves long-context scaling into the attention mechanism by routing queries through compressed summaries of cached keys. Its reported coding and retrieval results add a model-level alternative to harness-level compaction or retrieval, while the million-token advantage remains a projection rather than a demonstrated deployed context length.",
    "corpus_size": 974,
    "generated_at": "2026-11-02T11:02:39.748Z",
    "connections": [
      {
        "title": "ConvMem: Convolutional Memory for Long-Context Reasoning",
        "source_name": "arXiv",
        "source_url": "https://arxiv.org/abs/2609.10441v1",
        "feed7_url": "https://feed7.dev/p/2609-10441v1-066ygq8",
        "reason": "Both use hierarchical structure to handle long context, but ConvMem builds parallel summaries for reasoning while ZHA indexes cached keys to route sparse attention."
      },
      {
        "title": "Multi-State Linear Attention with Generalized Second-Order Routing",
        "source_name": "arXiv",
        "source_url": "https://arxiv.org/abs/2610.37645v1",
        "feed7_url": "https://feed7.dev/p/2610-37645v1-12yb9up",
        "reason": "Both alter model-side routing over compressed long-context representations; MERLIN adds moment estimates to segment routing, whereas ZHA uses a tree index over KV keys."
      },
      {
        "title": "CompactionRL: Reinforcement Learning with Context Compaction for Long-Horizon Agents",
        "source_name": "arXiv",
        "source_url": "https://arxiv.org/abs/2607.05378v1",
        "feed7_url": "https://feed7.dev/p/2607-05378v1-0j3vikd",
        "reason": "CompactionRL teaches an agent to manage a bounded context, while ZHA changes attention to scale over a long KV cache; the approaches address different layers of the long-horizon constraint."
      },
      {
        "title": "The Illusion of Robustness: Aggregate Accuracy Hides Prediction Flips under Task-Irrelevant Context",
        "source_name": "arXiv",
        "source_url": "https://arxiv.org/abs/2607.12963v1",
        "feed7_url": "https://feed7.dev/p/2607-12963v1-1oc0qmr",
        "reason": "ZHA reports maintained aggregate performance on long-context tasks, while this evaluation warns that aggregate accuracy can hide per-task flips under irrelevant context; item-level noise checks would qualify robustness claims."
      }
    ]
  },
  "lifecycle": "Current",
  "published_at": "2026-10-30T17:48:37.000Z",
  "modified_at": "2026-10-30T17:48:37.000Z",
  "supersedes": [],
  "expires_at": null,
  "formats": {
    "html": "https://feed7.dev/p/2610-42112v1-1k0vz2w",
    "json": "https://feed7.dev/p/2610-42112v1-1k0vz2w.json",
    "markdown": "https://feed7.dev/p/2610-42112v1-1k0vz2w.md"
  }
}