# Subquadratic Attention with a Hierarchical Tree-Structured Indexer

Source: [arXiv](https://arxiv.org/abs/2610.42112v1)  
Feed7 permalink: https://feed7.dev/p/2610-42112v1-1k0vz2w  
Published: 2026-10-30T17:48:37.000Z  
Trust: Needs Review (needs_review)

## Why Included

ZHA uses a tree index over compressed KV-cache summaries to retrieve sparse keys with subquadratic attention scaling. The paper reports maintained results on long-context and agentic coding evaluations; million-token performance is a scaling claim.

## Source Summary

Zyphra Hierarchical Attention (ZHA) builds a tree of compressed key summaries over the KV cache, then routes queries through it to select keys for sparse attention. The authors describe its total cost as **subquadratic**, and report applying it to pretrained models with a short KL distillation stage over **a few billion tokens**.

## Practical Implication

For coding agents that work with very long histories, the method points to an alternative to repeatedly attending over every cached token. The paper reports maintained performance on long-context retrieval and agentic coding evaluations, which makes the approach relevant to future long-context model choices.

## Agent-Ready Context

Zyphra Hierarchical Attention (ZHA) builds a tree of compressed key summaries over the KV cache, then routes queries through it to select keys for sparse attention. The authors describe its total cost as **subquadratic**, and report applying it to pretrained models with a short KL distillation stage over **a few billion tokens**.

For coding agents that work with very long histories, the method points to an alternative to repeatedly attending over every cached token. The paper reports maintained performance on long-context retrieval and agentic coding evaluations, which makes the approach relevant to future long-context model choices.

The authors’ performance model projects an advantage beyond **1 million tokens** and suggests potential contexts in the hundreds of millions or billions. Those longer ranges are projections; the material does not establish that such context lengths are available in deployed models.

## Connected Context

Feed7 judgment across 974 accumulated Signals:

ZHA moves long-context scaling into the attention mechanism by routing queries through compressed summaries of cached keys. Its reported coding and retrieval results add a model-level alternative to harness-level compaction or retrieval, while the million-token advantage remains a projection rather than a demonstrated deployed context length.

- [ConvMem: Convolutional Memory for Long-Context Reasoning](https://feed7.dev/p/2609-10441v1-066ygq8) — Both use hierarchical structure to handle long context, but ConvMem builds parallel summaries for reasoning while ZHA indexes cached keys to route sparse attention.
- [Multi-State Linear Attention with Generalized Second-Order Routing](https://feed7.dev/p/2610-37645v1-12yb9up) — Both alter model-side routing over compressed long-context representations; MERLIN adds moment estimates to segment routing, whereas ZHA uses a tree index over KV keys.
- [CompactionRL: Reinforcement Learning with Context Compaction for Long-Horizon Agents](https://feed7.dev/p/2607-05378v1-0j3vikd) — CompactionRL teaches an agent to manage a bounded context, while ZHA changes attention to scale over a long KV cache; the approaches address different layers of the long-horizon constraint.
- [The Illusion of Robustness: Aggregate Accuracy Hides Prediction Flips under Task-Irrelevant Context](https://feed7.dev/p/2607-12963v1-1oc0qmr) — ZHA reports maintained aggregate performance on long-context tasks, while this evaluation warns that aggregate accuracy can hide per-task flips under irrelevant context; item-level noise checks would qualify robustness claims.

## Context Map

- Layer: model
- Domains: coding, research
- Topics: context-engineering, agent-evals

## Uncertainty

- The authors’ performance model projects an advantage beyond **1 million tokens** and suggests potential contexts in the hundreds of millions or billions. Those longer ranges are projections; the material does not establish that such context lengths are available in deployed models.

## Agent Instruction

Use this item as source-backed context. Do not invent claims beyond the linked source. If this item conflicts with another source, call out the conflict.