{
  "schema_version": "1.1",
  "id": "archive:https://arxiv.org/abs/2611.09825v1",
  "slug": "2611-09825v1-0g2rcid",
  "url": "https://feed7.dev/p/2611-09825v1-0g2rcid",
  "title": "BitDecoding-FP4: Native FP4 KV Cache Decoding via Algorithm-System Co-Design",
  "why_included": "BitDecoding-FP4 co-designs KV-cache quantization and attention kernels, reporting near-BF16 accuracy and 2.4–3.0× peak throughput on NVIDIA B300. Its results are tied to specific formats, models, and hardware.",
  "summary": "BitDecoding-FP4 combines three quantization changes with fused probability quantization and attention operations on paged KV caches. It reports accuracy within **0.5 points** of BF16 with NVFP4 and within **1 point** with MXFP4; on NVIDIA B300 with Qwen3-4B and Qwen3-30B-A3B, peak serving throughput is **2.4–3.0×** BF16 KV.",
  "practical_implication": "For long-context agent serving, KV-cache formats and the kernels around them are a joint capacity, accuracy, and throughput decision. The paper's implementation is available on GitHub for builders with a compatible stack to inspect.",
  "agent_context": "BitDecoding-FP4 combines three quantization changes with fused probability quantization and attention operations on paged KV caches. It reports accuracy within **0.5 points** of BF16 with NVFP4 and within **1 point** with MXFP4; on NVIDIA B300 with Qwen3-4B and Qwen3-30B-A3B, peak serving throughput is **2.4–3.0×** BF16 KV.\n\nFor long-context agent serving, KV-cache formats and the kernels around them are a joint capacity, accuracy, and throughput decision. The paper's implementation is available on GitHub for builders with a compatible stack to inspect.\n\nThe figures are reported for specified models and B300 hardware; the material does not establish the same gains on other GPUs or serving systems. The accuracy numbers are relative to BF16 KV, not a general quality guarantee.",
  "source": {
    "name": "arXiv",
    "url": "https://arxiv.org/abs/2611.09825v1",
    "published_at": "2026-11-06T18:13:54.000Z"
  },
  "source_class": "blog_post",
  "content_type": "Paper",
  "layer": "infra",
  "domains": [],
  "topics": [],
  "verification": {
    "status": "needs_review",
    "label": "Needs Review",
    "method": "unverified",
    "verified_at": null
  },
  "uncertainty": [
    "The figures are reported for specified models and B300 hardware; the material does not establish the same gains on other GPUs or serving systems. The accuracy numbers are relative to BF16 KV, not a general quality guarantee."
  ],
  "connected_context": {
    "meaning": "This makes KV-cache quantization a concrete serving option: co-designed FP4 formats and kernels reportedly raise throughput while keeping accuracy near BF16 on the tested B300 setup. It adds implementation-level evidence to the candidates’ broader claims that cache movement and capacity constrain inference, while the hardware and model limits argue for workload-specific validation.",
    "corpus_size": 1032,
    "generated_at": "2026-11-09T11:01:49.982Z",
    "connections": [
      {
        "title": "Flash-dLLM: IO-Aware KV Caching and Parallel Decoding for Fast, Memory-Efficient Diffusion LLMs",
        "source_name": "arXiv",
        "source_url": "https://arxiv.org/abs/2609.26796v1",
        "feed7_url": "https://feed7.dev/p/2609-26796v1-0t57w0i",
        "reason": "Both target KV-cache costs in inference, but Flash-dLLM combines cache kernels with parallel decoding for diffusion models, while this paper focuses on quantized KV formats and fused attention; the gains address related costs and are not directly comparable."
      },
      {
        "title": "What's New in Inference Engineering — Philip Kiely, Baseten",
        "source_name": "AI Engineer",
        "source_url": "https://www.youtube.com/watch?v=75ckHC2LU_0",
        "feed7_url": "https://feed7.dev/p/what-s-new-in-inference-engineering-philip-kiely-baseten-1mlen9m",
        "reason": "The talk frames cache management and quantization as joint inference-tuning levers; this paper supplies a specific co-designed FP4 implementation and measured accuracy-throughput tradeoff on named hardware."
      },
      {
        "title": "Deep dive on LLM Inference at Scale — Harshul Jain, Audible & Tanmay Sah, Independent AI Researcher",
        "source_name": "AI Engineer",
        "source_url": "https://www.youtube.com/watch?v=y2W4FNAuPEA",
        "feed7_url": "https://feed7.dev/p/deep-dive-on-llm-inference-at-scale-harshul-jain-audible-tanmay-sah-inde-04utjkh",
        "reason": "The deep dive identifies KV-cache size as a possible GPU capacity bottleneck for long contexts; this paper tests reducing that burden through FP4 cache formats, with results bounded to its reported models and B300."
      }
    ]
  },
  "lifecycle": "Current",
  "published_at": "2026-11-06T18:13:54.000Z",
  "modified_at": "2026-11-06T18:13:54.000Z",
  "supersedes": [],
  "expires_at": null,
  "formats": {
    "html": "https://feed7.dev/p/2611-09825v1-0g2rcid",
    "json": "https://feed7.dev/p/2611-09825v1-0g2rcid.json",
    "markdown": "https://feed7.dev/p/2611-09825v1-0g2rcid.md"
  }
}