Flide: Accelerating Decision Inference with Exact State Forking
Flide targets decision-model inference, where ordinary generation engines can recompute shared prefixes or alter output probabilities. Its reported speedups suggest a specialized serving path for agent workloads with branching decisions.
Flide flattens decision-request trees into a token stream and uses GPU-resident indices to navigate branches across standard attention and recurrent Gated DeltaNet layers. It reports up to **9.5×** faster answers than vLLM and SGLang, with outputs closest to a float32 reference in total variation distance.
If your agent workload makes repeated categorical decisions over shared history, consider whether a decision model and specialized inference engine could reduce latency. Flide also supports resuming expanding histories from earlier checkpoints.
Flide flattens decision-request trees into a token stream and uses GPU-resident indices to navigate branches across standard attention and recurrent Gated DeltaNet layers. It reports up to **9.5×** faster answers than vLLM and SGLang, with outputs closest to a float32 reference in total variation distance. If your agent workload makes repeated categorical decisions over shared history, consider whether a decision model and specialized inference engine could reduce latency. Flide also supports resuming expanding histories from earlier checkpoints. The results are from evaluations on public hybrid and decision models; the material does not establish production performance across serving setups. Exactness is stated for exact arithmetic, while the reported comparison uses a float32 reference.
LSA adds a learned latent-space index for reducing attention work at very long context lengths, with results at 10M tokens for a single layer. This is a distinct route from tree-indexed sparse attention, and the layer-level speedups and bounded accuracy evidence make end-to-end serving validation necessary before inferring agent-level gains.