arXivPaperNeeds Review

Subquadratic Attention with a Hierarchical Tree-Structured Indexer

ZHA uses a tree index over compressed KV-cache summaries to retrieve sparse keys with subquadratic attention scaling. The paper reports maintained results on long-context and agentic coding evaluations; million-token performance is a scaling claim.

arXiv · Oct 30, 2026

Open Source Open Markdown Open JSON

Source Summary

Zyphra Hierarchical Attention (ZHA) builds a tree of compressed key summaries over the KV cache, then routes queries through it to select keys for sparse attention. The authors describe its total cost as **subquadratic**, and report applying it to pretrained models with a short KL distillation stage over **a few billion tokens**.

Practical Implication

For coding agents that work with very long histories, the method points to an alternative to repeatedly attending over every cached token. The paper reports maintained performance on long-context retrieval and agentic coding evaluations, which makes the approach relevant to future long-context model choices.

Agent-Ready Context

Zyphra Hierarchical Attention (ZHA) builds a tree of compressed key summaries over the KV cache, then routes queries through it to select keys for sparse attention. The authors describe its total cost as **subquadratic**, and report applying it to pretrained models with a short KL distillation stage over **a few billion tokens**.

For coding agents that work with very long histories, the method points to an alternative to repeatedly attending over every cached token. The paper reports maintained performance on long-context retrieval and agentic coding evaluations, which makes the approach relevant to future long-context model choices.

The authors’ performance model projects an advantage beyond **1 million tokens** and suggests potential contexts in the hundreds of millions or billions. Those longer ranges are projections; the material does not establish that such context lengths are available in deployed models.

Connected Context · Feed7 Judgment

ZHA moves long-context scaling into the attention mechanism by routing queries through compressed summaries of cached keys. Its reported coding and retrieval results add a model-level alternative to harness-level compaction or retrieval, while the million-token advantage remains a projection rather than a demonstrated deployed context length.

ConvMem: Convolutional Memory for Long-Context ReasoningBoth use hierarchical structure to handle long context, but ConvMem builds parallel summaries for reasoning while ZHA indexes cached keys to route sparse attention.Multi-State Linear Attention with Generalized Second-Order RoutingBoth alter model-side routing over compressed long-context representations; MERLIN adds moment estimates to segment routing, whereas ZHA uses a tree index over KV keys.CompactionRL: Reinforcement Learning with Context Compaction for Long-Horizon AgentsCompactionRL teaches an agent to manage a bounded context, while ZHA changes attention to scale over a long KV cache; the approaches address different layers of the long-horizon constraint.The Illusion of Robustness: Aggregate Accuracy Hides Prediction Flips under Task-Irrelevant ContextZHA reports maintained aggregate performance on long-context tasks, while this evaluation warns that aggregate accuracy can hide per-task flips under irrelevant context; item-level noise checks would qualify robustness claims.

Context Map

modelcodingresearch#context-engineering #agent-evals

Uncertainty

The authors’ performance model projects an advantage beyond **1 million tokens** and suggests potential contexts in the hundreds of millions or billions. Those longer ranges are projections; the material does not establish that such context lengths are available in deployed models.