Mixture-of-Kittens: our open-source MoE megakernel for NVL72s
Cursor open-sourced a deterministic MoE training megakernel for GB300 NVL72 racks. It fuses communication with compute and reports 1.41× end-to-end training throughput in production.
Cursor open-sourced **Mixture-of-Kittens**, a deterministic MoE training megakernel for GB300 NVL72 systems. It reports up to **2.37× MXFP8 forward throughput** versus the fastest public baseline and **1.41× end-to-end tokens per second** in production.
Teams training MoE coding models should profile communication as part of the layer, not optimize matrix compute alone. MoK uses pull dispatch, push combine, a reusable device-side schedule, and no CPU-GPU synchronization in that path.
Cursor open-sourced **Mixture-of-Kittens**, a deterministic MoE training megakernel for GB300 NVL72 systems. It reports up to **2.37× MXFP8 forward throughput** versus the fastest public baseline and **1.41× end-to-end tokens per second** in production. Teams training MoE coding models should profile communication as part of the layer, not optimize matrix compute alone. MoK uses pull dispatch, push combine, a reusable device-side schedule, and no CPU-GPU synchronization in that path. The design targets DeepSeek-V3-style MoE layers and NVL72 hardware, where 72 GPUs share an NVLink domain. The supplied results come from Cursor, and the material does not establish gains on other topologies or workloads.
This adds production-oriented evidence that MoE training performance depends on integrating communication and compute rather than optimizing kernels in isolation. The reported end-to-end gain is more informative than forward throughput alone, but the result remains narrowly scoped to DeepSeek-V3-style layers on a 72-GPU NVLink domain. It therefore informs specialized training infrastructure without establishing a general MoE speedup.