Cursor's Megakernel Improves GPU Training Throughput 1.41x on Blackwell Racks

On August 4, 2026, Cursor open-sourced Mixture-of-Kittens (MoK), a megakernel that raised end-to-end training throughput 1.41x on 512 GB300 GPUs, while the Latent Space Inference Engineering Masterclass published arguments for why megakernels are obsolete. The collision clarified a real tension: whether megakernels are dead depends on your hardware and workload.

The case against: megakernels eliminate kernel launch overhead and inter-kernel synchronization lag, but writing one is hard. Companies that build them often don't run them in production—per-kernel tuning plus scheduler overlap outperforms a monolithic fused kernel, since each component can be independently optimized and pipelined in parallel. Tensor parallelism imposes a ceiling: when half a matrix lives on GPU 1 and half on GPU 2, operations like softmax require an all-reduce before proceeding. Fusion cannot eliminate that communication barrier. NVIDIA is also solving synchronization in hardware. Kyle Kranen announced that Rubin introduces tile-level dependency triggers, allowing kernel N+1 to launch CTAs for a tile the moment kernel N finishes it, without waiting for stragglers. This is what megakernels previously did in CUDA code.

Cursor's release argued the opposite for MoE layers on NVL72 racks. MoE consumes more than half of end-to-end training time. A GB300 NVL72 is 72 GPUs in a single NVLink domain—communication topology differs qualitatively from a DGX cluster. Integrated Grace CPUs are slow relative to the GPUs: GPU streams regularly waited on CPU work (logging, metrics), leaving the GPU idle. Only a megakernel can eliminate CPU-GPU synchronization entirely; individual kernels cannot.

MoK's numbers: MXFP8 forward runs 2.37x faster than the fastest public baseline (DeepEP+TransformerEngine, HybridEP+Megatron) at EP degree 64 with 2,048 tokens per GPU. BF16 is 1.92x. End-to-end, tokens per second per GPU rose from 760.9 to 1,070.2—a 1.41x gain. Pull-based dispatch rather than DeepEP's push-based design accounts for part of the gap: pull leaves reverse NVLink lanes mostly idle; pull achieves 29% higher NVLink bandwidth utilization under expert imbalance. Signalling overhead drops from 103 µs to 18 µs. A ring token buffer called macrobatch cycles through a fixed-size buffer at minibatch granularity without stalling the GPU or dropping tokens.

MoK requires NVIDIA Blackwell SM100 or SM103—meaning GB200 NVL72 or GB300 NVL72 racks specifically. Needs CUDA 13.0+, PyTorch 2.10+, Python 3.12+. Inter-GPU buffers use PyTorch symmetric memory, which is Blackwell-only. Runs on H100 or B200 DGX nodes will not build. The project is Apache-2.0 licensed and already powers Cursor's Composer model training across tens of thousands of GPUs.

On commodity clusters and Rubin hardware, kernel-splitting plus hardware-level CTA scheduling is increasingly viable with far lower maintenance burden—no 67,000-line hand-fused forward pass to debug. On NVL72 racks, where Grace CPU bottleneck and NVLink topology create distinct constraints, fusion still wins by a margin that justifies the engineering cost. If MoE throughput is your training bottleneck and you own NVL72 capacity, evaluate MoK against your current DeepEP stack before Rubin ships.