# Spool: Low-Latency Streaming Inference for Hybrid Vision-Language Models

Source: [arXiv](https://arxiv.org/abs/2611.23228v1)  
Feed7 permalink: https://feed7.dev/p/2611-23228v1-0oc1ofk  
Published: 2026-11-17T18:59:27.000Z  
Trust: Needs Review (needs_review)

## Why Included

Spool removes expired video frames from hybrid VLM recurrent state without replaying the window through the backbone. Across 20 model-benchmark tests, it cut p95 time-to-first-token by a geometric mean of 6.6× versus the faster vLLM baseline.

## Source Summary

Spool caches per-frame state transformations and composes them as a video window advances, avoiding a full replay when old frames expire. In **20 tests** across five open-source hybrid VLMs and four video reasoning benchmarks, it reduced p95 time-to-first-token by a geometric mean of **6.6×** versus the faster of two recent vLLM baselines; average accuracy also rose **2.3 percentage points**.

## Practical Implication

Builders working on streaming video agents could consider this approach when they need bounded frame history and faster answers. The method is training-free and uses a dedicated data structure and GPU kernels.

## Agent-Ready Context

Spool caches per-frame state transformations and composes them as a video window advances, avoiding a full replay when old frames expire. In **20 tests** across five open-source hybrid VLMs and four video reasoning benchmarks, it reduced p95 time-to-first-token by a geometric mean of **6.6×** versus the faster of two recent vLLM baselines; average accuracy also rose **2.3 percentage points**.

Builders working on streaming video agents could consider this approach when they need bounded frame history and faster answers. The method is training-free and uses a dedicated data structure and GPU kernels.

Frame removal is exact within each recurrent layer, but expired frames can still influence later layers through cached inputs. The reported results cover the tested models and benchmarks; the material does not establish performance for other workloads.

## Connected Context

Feed7 judgment across 1116 accumulated Signals:

Spool adds a concrete streaming-inference result to the efficiency discussion: caching frame-state transformations can cut p95 time to first token while slightly improving average accuracy on the tested hybrid VLMs. Its exactness is limited to removal within each recurrent layer, so the result supports workload-specific measurement rather than a general claim that cached state is equivalent to full replay.

- [Metacognitive Decoding: Efficient Inference for Diffusion Language Agents](https://feed7.dev/p/2610-37642v1-1vbgjq8) — Both report inference-time efficiency gains while tracking task quality, but Spool targets bounded video history and latency, whereas Metacognitive Decoding adapts token generation and reports lower FLOPs.
- [You Only Pass Once: Answering and Abstaining Together in a Single Forward Pass of a Frozen Language Model](https://feed7.dev/p/2608-14465v1-0tpy8xd) — Both describe training-free inference mechanisms with efficiency and reliability implications; Spool accelerates streaming video processing, while YOPO combines answering and abstention in one pass and reports a model-internal intervention constraint.

## Context Map

- Layer: infra
- Domains: video, research
- Topics: agent-reliability

## Uncertainty

- Frame removal is exact within each recurrent layer, but expired frames can still influence later layers through cached inputs. The reported results cover the tested models and benchmarks; the material does not establish performance for other workloads.

## Agent Instruction

Use this item as source-backed context. Do not invent claims beyond the linked source. If this item conflicts with another source, call out the conflict.
