ReContext: Recursive Evidence Replay as LLM Harness for Long-Context Reasoning
ReContext is a training-free harness that replays query-relevant evidence from long inputs before answering, taking the best average rank across 8 long-context benchmarks up to 128K on Qwen3-4B/8B and Llama3-8B.
**The gist** ReContext uses the model's own **attention-based relevance signals** to build a query-conditioned evidence pool from the full input, then replays it before generation — **training-free**, no external memory, no context pruning. It takes the **best average rank on 8 long-context datasets up to 128K** across **Qwen3-4B, Qwen3-8B, and Llama3-8B**.
**Why it matters** Big context windows don't mean the model uses what's in them; this is harness-level evidence that restructuring when evidence is seen beats trusting raw attention — a pattern worth stealing for retrieval and context-assembly steps in agent loops.
**The gist** ReContext uses the model's own **attention-based relevance signals** to build a query-conditioned evidence pool from the full input, then replays it before generation — **training-free**, no external memory, no context pruning. It takes the **best average rank on 8 long-context datasets up to 128K** across **Qwen3-4B, Qwen3-8B, and Llama3-8B**. **Why it matters** Big context windows don't mean the model uses what's in them; this is harness-level evidence that restructuring when evidence is seen beats trusting raw attention — a pattern worth stealing for retrieval and context-assembly steps in agent loops. **Watch out** Gains were shown on **small open models (4B–8B)**; whether frontier models with stronger long-context behavior benefit, and what the recursive replay costs in extra tokens and latency, isn't established.