# Mixture-of-Kittens: our open-source MoE megakernel for NVL72s

Source: [Cursor](https://cursor.com/blog/mixture-of-kittens)  
Feed7 permalink: https://feed7.dev/p/mixture-of-kittens-0u8yi4u  
Published: 2026-08-04T12:00:00.000Z  
Trust: Official Source (official_source)

## Why Included

Cursor open-sourced a deterministic MoE training megakernel for GB300 NVL72 racks. It fuses communication with compute and reports 1.41× end-to-end training throughput in production.

## Source Summary

Cursor open-sourced **Mixture-of-Kittens**, a deterministic MoE training megakernel for GB300 NVL72 systems. It reports up to **2.37× MXFP8 forward throughput** versus the fastest public baseline and **1.41× end-to-end tokens per second** in production.

## Practical Implication

Teams training MoE coding models should profile communication as part of the layer, not optimize matrix compute alone. MoK uses pull dispatch, push combine, a reusable device-side schedule, and no CPU-GPU synchronization in that path.

## Agent-Ready Context

Cursor open-sourced **Mixture-of-Kittens**, a deterministic MoE training megakernel for GB300 NVL72 systems. It reports up to **2.37× MXFP8 forward throughput** versus the fastest public baseline and **1.41× end-to-end tokens per second** in production.

Teams training MoE coding models should profile communication as part of the layer, not optimize matrix compute alone. MoK uses pull dispatch, push combine, a reusable device-side schedule, and no CPU-GPU synchronization in that path.

The design targets DeepSeek-V3-style MoE layers and NVL72 hardware, where 72 GPUs share an NVLink domain. The supplied results come from Cursor, and the material does not establish gains on other topologies or workloads.

## Connected Context

Feed7 judgment across 353 accumulated Signals:

This adds production-oriented evidence that MoE training performance depends on integrating communication and compute rather than optimizing kernels in isolation. The reported end-to-end gain is more informative than forward throughput alone, but the result remains narrowly scoped to DeepSeek-V3-style layers on a 72-GPU NVLink domain. It therefore informs specialized training infrastructure without establishing a general MoE speedup.

- [Accelerating Transformers Fine-Tuning with NVIDIA NeMo AutoModel](https://feed7.dev/p/accelerating-fine-tuning-nvidia-nemo-automodel-0dj2ywc) — NeMo AutoModel reports broader fine-tuning acceleration behind a familiar API, while Mixture-of-Kittens isolates a lower-level communication-aware kernel optimization; together they address different layers of MoE training performance.
- [The Messy Reality of Scale: Synthetic Data and Pre-Training — Marah Abdin & Robert McHardy, poolside](https://feed7.dev/p/the-messy-reality-of-scale-synthetic-data-and-pre-training-marah-abdin-r-15307vg) — Poolside’s distributed-training and numerical-consistency warnings reinforce Mixture-of-Kittens’ central implementation consequence: communication, scheduling, precision, and compute must be profiled as one system before scaling.

## Context Map

- Layer: infra
- Domains: coding
- Topics: open-models

## Uncertainty

- The design targets DeepSeek-V3-style MoE layers and NVL72 hardware, where 72 GPUs share an NVLink domain. The supplied results come from Cursor, and the material does not establish gains on other topologies or workloads.

## Agent Instruction

Use this item as source-backed context. Do not invent claims beyond the linked source. If this item conflicts with another source, call out the conflict.