arXivPaperNeeds Review

Structured Memory for Edge Language Models: Persistent Context and Corpus Retrieval via O(1) SSM State Injection

PRECOG precomputes SSM corpus states and injects them at query time, avoiding context re-ingestion. On a 1.2B edge model, reported prefill fell from about 27 seconds to under 6 ms.

arXiv · Aug 3, 2026

Open Source Open Markdown Open JSON

Source Summary

PRECOG encodes documents offline as fixed-size SSM states, retrieves the best match, and injects it at query time. On a **1.2B-parameter** model with a **192 KB hidden state**, prefill dropped from about 27 seconds to **under 6 ms**.

Practical Implication

For edge agents with repeated corpus access, benchmark state injection against conventional RAG. The accompanying memory scheme combines episodic, semantic, and retrieved states with **O(1) session initialization** and an adjustable storage-fidelity tradeoff.

Agent-Ready Context

PRECOG encodes documents offline as fixed-size SSM states, retrieves the best match, and injects it at query time. On a **1.2B-parameter** model with a **192 KB hidden state**, prefill dropped from about 27 seconds to **under 6 ms**.

For edge agents with repeated corpus access, benchmark state injection against conventional RAG. The accompanying memory scheme combines episodic, semantic, and retrieved states with **O(1) session initialization** and an adjustable storage-fidelity tradeoff.

The approach depends on SSM architecture and cannot be transferred directly to position-entangled Transformer KV caches. The material says answer quality matched in-context RAG, but gives no task breakdown or evidence across other models and hardware.

Connected Context · Feed7 Judgment

This introduces an architecture-specific alternative to repeatedly prefilling retrieved documents: store corpus items as reusable SSM states and inject one at session start. It could materially change edge-agent retrieval economics and memory initialization, while narrowing applicability to compatible SSMs. Claims of RAG-equivalent quality remain provisional without task-level results, cross-model tests, or provenance controls.

ReContext: Recursive Evidence Replay as LLM Harness for Long-Context ReasoningReContext replays selected evidence as tokens for long-context reasoning, whereas PRECOG injects an offline-encoded state; they are competing ways to surface query-relevant evidence with different architecture and prefill costs.UniMem: Complementary Episodic-to-Parametric Memory for Boundary-Agnostic Task StreamsPRECOG supplies a fixed-state mechanism for episodic, semantic, and retrieved memory, while UniMem addresses routing recurring experience between retrieval and learned parameters; neither supplied result resolves inspectability or governance.Citation Needed: Provenance for LLM-Built Knowledge Graphs — Daniel Chalef, Zep AIState injection accelerates access but does not preserve the claim-level provenance needed when stored knowledge merges, changes, or must be deleted, making provenance metadata an implementation consequence rather than a solved feature.ReToken: One Token to Improve Vision-Language Models for Visual RetrievalBoth retrieve compact cached representations instead of processing the full context at query time, but PRECOG targets text in SSM state space while ReToken selects visual tokens from cached multimodal context.

Context Map

contextresearchdata#retrieval#agent-memory #context-engineering

Uncertainty

The approach depends on SSM architecture and cannot be transferred directly to position-entangled Transformer KV caches. The material says answer quality matched in-context RAG, but gives no task breakdown or evidence across other models and hardware.