{
  "schema_version": "1.1",
  "id": "s13:https://arxiv.org/abs/2609.02846v1",
  "slug": "2609-02846v1-014naa0",
  "url": "https://feed7.dev/p/2609-02846v1-014naa0",
  "title": "UE5M3 FP4 Block Scaling for Stable Language Model Pretraining",
  "why_included": "A UE5M3 block-scaling recipe trained an 8B model in FP4 without Hadamard transforms or BF16 final layers, while reporting better losses and downstream estimates than the compared recipe.",
  "summary": "The authors paired E2M1 payloads with **UE5M3 block scales**, used selective stochastic rounding, removed the randomized Hadamard transform, and applied FP4 to eligible internal linears. They pretrained an **8B model on nearly 190B tokens**.",
  "practical_implication": "For low-precision training work, test whether wider block scales can replace auxiliary transforms and higher-precision exemptions. The simpler recipe may reduce work outside FP4 matrix multiplications.",
  "agent_context": "The authors paired E2M1 payloads with **UE5M3 block scales**, used selective stochastic rounding, removed the randomized Hadamard transform, and applied FP4 to eligible internal linears. They pretrained an **8B model on nearly 190B tokens**.\n\nFor low-precision training work, test whether wider block scales can replace auxiliary transforms and higher-precision exemptions. The simpler recipe may reduce work outside FP4 matrix multiplications.\n\nResults come from software-emulated training and recipe-specific inference policies. The reported **21.2% throughput increase** is a native execution ablation that jointly removed two components, not a direct end-to-end UE5M3 hardware result.",
  "source": {
    "name": "arXiv",
    "url": "https://arxiv.org/abs/2609.02846v1",
    "published_at": "2026-09-02T17:32:07.000Z"
  },
  "source_class": "blog_post",
  "content_type": "Paper",
  "layer": "model",
  "domains": [],
  "topics": [
    "model-selection"
  ],
  "verification": {
    "status": "needs_review",
    "label": "Needs Review",
    "method": "unverified",
    "verified_at": null
  },
  "uncertainty": [
    "Results come from software-emulated training and recipe-specific inference policies. The reported **21.2% throughput increase** is a native execution ablation that jointly removed two components, not a direct end-to-end UE5M3 hardware result."
  ],
  "connected_context": {
    "meaning": "This adds numerical format and scaling design to model-selection judgment: a wider FP4 block scale may simplify low-precision pretraining by removing transforms and exemptions. The evidence remains recipe- and execution-specific, so the throughput figure should motivate native hardware validation rather than be treated as an end-to-end deployment gain.",
    "corpus_size": 669,
    "generated_at": "2026-09-03T10:01:29.442Z",
    "connections": [
      {
        "title": "Adaption Labs: Gradient-Free Continual Learning — Sara Hooker, Adaption",
        "source_name": "AI Engineer",
        "source_url": "https://www.youtube.com/watch?v=XEd_SRVHBgU",
        "feed7_url": "https://feed7.dev/p/adaption-labs-gradient-free-continual-learning-sara-hooker-adaption-1pozjjc",
        "reason": "The FP4 configuration is a concrete example of the interacting architecture and training-recipe choices that broader automated adaptation systems would need to search and validate together."
      },
      {
        "title": "When Does Bigger Help? A Controlled Study of LLM Scale for Ontology Learning",
        "source_name": "arXiv",
        "source_url": "https://arxiv.org/abs/2608.31118v1",
        "feed7_url": "https://feed7.dev/p/2608-31118v1-1abyqwd",
        "reason": "The ontology study’s task-dependent scaling results reinforce the same selection discipline: neither parameter count nor a precision-format throughput ablation substitutes for matched quality and deployment measurements."
      }
    ]
  },
  "lifecycle": "Current",
  "published_at": "2026-09-02T17:32:07.000Z",
  "modified_at": "2026-09-02T17:32:07.000Z",
  "supersedes": [],
  "expires_at": null,
  "formats": {
    "html": "https://feed7.dev/p/2609-02846v1-014naa0",
    "json": "https://feed7.dev/p/2609-02846v1-014naa0.json",
    "markdown": "https://feed7.dev/p/2609-02846v1-014naa0.md"
  }
}