Why AI Agents Need Million-Token Context — Thomas Wolf & Olive Song, MiniMax
MiniMax argues that long-running, multimodal agents need enough context for repeated tool responses, while sparse attention makes that context more tractable.
MiniMax M3 has roughly **400B total parameters**, activates **20B**, and supports text, images, and video. Its **1M-token context** uses MiniMax Sparse Attention: an index branch selects relevant blocks, then a sparse branch performs attention over them.
For agent builders, the practical claim is that multi-round tool responses and long, unstructured inputs can exceed short contexts. M3’s native multimodal training also points toward agents that inspect reports, presentations, tutorials, images, and video before acting with tools.
MiniMax M3 has roughly **400B total parameters**, activates **20B**, and supports text, images, and video. Its **1M-token context** uses MiniMax Sparse Attention: an index branch selects relevant blocks, then a sparse branch performs attention over them. For agent builders, the practical claim is that multi-round tool responses and long, unstructured inputs can exceed short contexts. M3’s native multimodal training also points toward agents that inspect reports, presentations, tutorials, images, and video before acting with tools. A large context window does not establish useful recall across its full length, and the discussion supplies no retrieval or agent-task measurements. Trillion-token attention is presented only as a research direction requiring architecture and hardware work.
This adds an architectural explanation for how a 1M-token multimodal model can make large contexts computationally practical and clarifies why tool-heavy agents may need them. Against the candidate model releases, it confirms that million-token capacity is becoming common but does not resolve model selection: neither sparse attention nor native multimodality demonstrates reliable recall or better agent performance without workload-level evaluation.